scan

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 24, 2026 License: AGPL-3.0 Imports: 40 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// MaxPostDataSize is the maximum POST data size that Chrome DevTools will capture
	MaxPostDataSize = 64 * 1024 // 64KB

	// InitialBufferSize is the initial size for scanner buffers
	InitialBufferSize = 64 * 1024 // 64KB

	// MaxBufferSize is the maximum size for scanner buffers.
	// Minified JS bundles are frequently emitted as a single multi-megabyte
	// line, so this must be large enough to hold an entire bundle as one token;
	// otherwise bufio.Scanner aborts with ErrTooLong and the whole file is
	// skipped. The scanner grows the buffer on demand up to this cap, so the
	// value is a ceiling, not a pre-allocation.
	MaxBufferSize = 64 * 1024 * 1024 // 64MB

	// MaxResponseBodyBytes caps how much of a fetched HTTP response the crawler
	// reads into memory before scanning it. A crawl fetches arbitrary, attacker-
	// influenced hosts at scale, so an unbounded read lets a single hostile or
	// misconfigured server that streams a body of any length exhaust the
	// scanner's memory and take the whole crawl down. The cap matches
	// MaxBufferSize — the scanner's own per-token ceiling — so bytes past it
	// would be dropped by the scanner anyway; bounding the read just refuses to
	// buffer them first.
	MaxResponseBodyBytes = MaxBufferSize
)

Network and buffer sizes

View Source 
const (
	// MaxRedirects is the maximum number of HTTP redirects to follow
	MaxRedirects = 5

	// MaxParameterDisplayLength is the maximum length for parameter display in output
	MaxParameterDisplayLength = 100
)

Other limits

View Source 
const (
	// DOMTypeFlow is an observed source-to-sink flow: an attacker-controllable
	// input reached a security-sensitive browser sink.
	DOMTypeFlow = "dom_flow"

	// DOMTypeSink is a dangerous sink observed executing without any evidence of
	// controllable input reaching it (observe mode). It is intelligence, not a
	// vulnerability.
	DOMTypeSink = "dom_sink"

	// DOMTypeWebMessage reports postMessage listener/message analysis: what the
	// page listens for, whether it inspects origin/source, and whether message
	// data reached a sink.
	DOMTypeWebMessage = "web_message"

	// DOMTypeSummary is the final scan-summary record emitted once per scan in
	// streaming output.
	DOMTypeSummary = "scan_summary"
)

DOM finding types. These strings are stable public identifiers: automated triage keys off them, so their spellings must not change. New analyses (client-side prototype pollution, DOM clobbering) are added as new type constants rather than by overloading an existing one.

View Source 
const (
	// ConfidenceLow is a static or incomplete indication.
	ConfidenceLow = "low"
	// ConfidenceMedium is a runtime sink observation with uncertain source control.
	ConfidenceMedium = "medium"
	// ConfidenceHigh is a unique canary correlated from a specific source to a
	// specific sink.
	ConfidenceHigh = "high"
	// ConfidenceCertain is controlled execution confirmed.
	ConfidenceCertain = "certain"
)

Confidence reflects the quality of the evidence behind a DOM finding, kept deliberately separate from severity (which reflects impact). A high-severity sink reached only by a weak static signal is high severity, low confidence.

View Source 
const (
	TriggerPageLoad    = "page_load"
	TriggerInteraction = "interaction"
	TriggerPostMessage = "post_message"
)

DOM trigger categories describe when a flow was observed. They are used both as evidence in the finding and, folded to a category, as part of dedup so the same flow reached several ways collapses to one record with combined triggers.

View Source 
const (
	PhaseInitialLoad      = "initial_load"
	PhaseStateExploration = "state_exploration"
)

Scan phase distinguishes a flow seen during the initial page load from one that only appears after the scanner explores further application state.

View Source 
const (
	DOMTriageConfirmed    = "confirmed"
	DOMTriageWorthReview  = "worth_reviewing"
	DOMTriageLikelyBenign = "likely_benign"
	DOMTriageInfo         = "informational"
)
View Source 
const (
	DOMModeObserve = "observe"
	DOMModeCanary  = "canary"
	DOMModeConfirm = "confirm"
)

DOM scanning modes.

View Source 
const (
	SourceURLQuery       = "url_query"
	SourceURLFragment    = "url_fragment"
	SourceURLFull        = "url_full"
	SourceLocation       = "location"
	SourceReferrer       = "referrer"
	SourceWindowName     = "window_name"
	SourceFormInput      = "form_input"
	SourceCookie         = "cookie"
	SourceLocalStorage   = "local_storage"
	SourceSessionStorage = "session_storage"
	SourceWebMessage     = "web_message"
)

Source-family identifiers. These are the source.kind values in findings and the tokens accepted by -dom-sources, so they are stable public strings.

View Source 
const (
	DOMHintJavaScriptAccess  = "javascript_access"
	DOMHintJavaScriptURL     = "javascript_url"
	DOMHintJavaScriptRequest = "javascript_request"
	DOMHintPassiveWayback    = "passive_wayback"
	DOMHintPassiveCommon     = "passive_commoncrawl"
	DOMHintDOMForm           = "dom_form"
)
View Source 
const (
	ReflectionContextHTMLText    = "html_text"
	ReflectionContextHTMLAttr    = "html_attribute"
	ReflectionContextHTMLComment = "html_comment"
	ReflectionContextScript      = "script"
	ReflectionContextUnknown     = "unknown"
)

Reflection contexts classify where in the response body the marker landed. They are stable public strings emitted in findings.

View Source 
const (
	SeverityHigh   = "high"
	SeverityMedium = "medium"
	SeverityLow    = "low"
	SeverityInfo   = "info"
)

Severity levels rank a finding by how likely it is to be a real, directly exploitable secret, so the output can lead with what matters.

  • High: distinctive credential formats (provider tokens, cloud keys, JWTs) whose signature alone makes a match almost certainly a live secret.
  • Medium: keyword-anchored credentials (`api_key=...`, `password: ...`) that are probably secrets but carry more false positives and warrant review.
  • Low: findings that only occasionally reveal something sensitive — HTTP headers, for instance, are usually mundane and worth a look, not an alarm.
  • Info: non-secret intelligence — endpoints, URLs, emails, paths, IPs and generic high-entropy strings — that is useful context, not a leak.
View Source 
const DOMSchemaVersion = "dom.1.2"

DOMSchemaVersion identifies the DOM finding output schema. It is emitted in structured output so downstream consumers can detect an incompatible change. Bump the minor version when adding fields, the major version when changing or removing an existing field's meaning.

View Source 
const GatheredURLPattern = "gathered_url"

GatheredURLPattern is the Match.Pattern used for the crawler's "gathered URL" findings: in-scope URLs the crawl confirmed as live, annotated with the HTTP request methods that worked against them (and, for parameter replay, the parameters that produced the hit). They are surfaced as their own segment in the output, beneath the normal JavaScript findings.

View Source 
const GraphQLIntrospectionPattern = "graphql_introspection"

GraphQLIntrospectionPattern is the Match.Pattern for a confirmed GraphQL endpoint whose introspection is enabled. Introspection exposes the whole schema — every query, mutation and type — to any client, so besides mapping the API's surface it is a finding in its own right: production endpoints are expected to disable it.

View Source 
const ReflectionSchemaVersion = "reflection.1.0"

ReflectionSchemaVersion identifies the reflection-finding output schema. Bump the minor version when adding fields, the major version when changing an existing field's meaning.

View Source 
const ReflectionType = "reflection"

ReflectionType is the stable finding-type identifier for a reflected input, distinct from the DOM finding types so downstream triage can key off it.

Variables

View Source 
var (
	// BrowserDownloadBaseURL is the Chrome-for-Testing storage base. It is a var so
	// tests can point provisioning at a local server instead of the internet.
	BrowserDownloadBaseURL = "https://storage.googleapis.com/chrome-for-testing-public"

	// BrowserVersionURL is the Chrome-for-Testing "last known good versions"
	// endpoint used to discover the latest stable version to download. A var so
	// tests can redirect it; an empty value skips the lookup and uses the fallback.
	BrowserVersionURL = "https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions.json"

	// PinnedChromeVersion is the Chrome-for-Testing build JSMiner falls back to when
	// the latest-version lookup is unavailable (e.g. offline). Normal operation
	// always resolves and downloads the current latest stable instead.
	PinnedChromeVersion = "131.0.6778.204"

	// AutoDownloadBrowser controls whether ResolveBrowser may download a browser
	// when none is found locally. It is on by default so rendering works out of the
	// box; -no-download-browser turns it off for air-gapped or bundle-only setups.
	AutoDownloadBrowser = true

	// BrowserNotice, when set, receives short human-readable status messages about
	// browser provisioning — chiefly the first-run download, which is large enough
	// (~150MB) that a user should be told it is happening rather than seeing the
	// scan appear to hang. The CLI wires it to stderr; it is unset (silent) for
	// library callers.
	BrowserNotice func(msg string)
)
View Source 
var (
	// RenderTimeout is the timeout for page rendering operations
	RenderTimeout = 15 * time.Second

	// RenderSleepDuration is the wait time for dynamic content to load
	RenderSleepDuration = 8 * time.Second

	// MaxExploreStates bounds how many additional application states the renderer
	// reaches by interacting with a page — clicking client-side navigation
	// controls and filling and submitting forms — beyond the initial load. A
	// single-page app hides most of its surface behind event handlers, so without
	// this the crawler sees only the shell it first rendered. Zero disables
	// interaction and restores the plain "render once" behaviour.
	MaxExploreStates = 12

	// ExploreSettleDuration is how long to wait after each interaction for the
	// resulting state to render (client-side route change, XHR-driven update or
	// form submission) before it is snapshotted. It is deliberately much shorter
	// than RenderSleepDuration, which is paid once for the initial load.
	ExploreSettleDuration = 1500 * time.Millisecond

	// HTTPClientTimeout is the timeout for HTTP requests
	HTTPClientTimeout = 10 * time.Second

	// SkipTLSVerification controls whether HTTPS certificate verification is skipped
	// Defaults to true so invalid certificates are accepted unless explicitly disabled
	SkipTLSVerification = true

	// FollowRedirects controls whether HTTP clients follow 3xx responses. It is
	// deliberately independent from external-resource discovery: -external
	// decides which page-referenced scripts/imports are selected, while this
	// setting decides whether any selected URL may follow a redirect. Redirects
	// are disabled by default so scanning a target never silently moves elsewhere.
	FollowRedirects = false
)

Timeouts and delays

View Source 
var ChromePath string

ChromePath, when set, is the explicit path to the Chrome/Chromium executable used for rendering. It is empty by default, in which case chromedp auto-detects a browser on PATH. Setting it lets JSMiner render in environments where Chrome is installed at a known location that is not on PATH — common in CI images and containers (Playwright/Puppeteer browser caches, custom installs) — where auto-detection would otherwise fail and rendering would silently fall back to a static fetch.

View Source 
var FetchRetries = 2

FetchRetries is how many extra attempts the shared HTTP fetch path makes when a request fails with a transport error — a connection reset, DNS blip or timeout, the kind of transient failure an enterprise crawl of thousands of requests hits routinely. Only safe, bodyless GET/HEAD/OPTIONS requests are retried, so active mutation probes and discovered parameter replays are never double-submitted against a target. Zero disables retries.

View Source 
var Verbosity int

Verbosity controls how much diagnostic detail the scan package writes to the verbose log (stderr by default). It is 0 — silent — unless the caller raises it with SetVerbosity, and higher levels are cumulative: 

1 (-v)   crawl narrative: matches found per page, in-scope targets
         discovered, queue growth, calibration and template-dedup skips.
2 (-vv)  network and render activity: every HTTP request with its method,
         status and size; every page render with the scripts and application
         states it surfaced.
3 (-vvv) per-item trace: individual target enqueue/skip decisions, method
         probes, parameter replays, permutations, followed imports and
         recovered source maps.

Diagnostics go to stderr so they never contaminate the machine-readable results written to stdout.

Functions

func DOMSeedURLsFromMatches 

func DOMSeedURLsFromMatches(seed string, matches []Match, allowExternal bool, max int) []string

DOMSeedURLsFromMatches turns routes found by the static/rendered crawl into browser instrumentation seeds. It keeps query names, drops fragments and obvious non-document assets, and respects the original target scope.

func DOMSinkFamilies 

func DOMSinkFamilies() []string

DOMSinkFamilies returns the selectable sink families for -dom-sinks validation.

func DOMSourceAliases 

func DOMSourceAliases() map[string][]string

DOMSourceAliases maps convenience source names to the concrete families that implement them. url_full and location both read from the URL, which the query and fragment canaries cover.

func DOMSourceFamilies 

func DOMSourceFamilies() []string

DOMSourceFamilies returns the directly-selectable source families for -dom-sources validation.

func FetchURL 

func FetchURL(url string) (io.ReadCloser, error)

FetchURL retrieves the content at url with timeouts. Redirects are followed only when FollowRedirects is enabled.

func ProvisionBundle 

func ProvisionBundle(dir string) (string, error)

ProvisionBundle downloads the pinned Chromium and extracts it into dir/chromium, the layout bundledBrowserPath detects, so dir (holding the jsminer binary and this chromium/ directory) can be shipped as one self-contained archive that renders without any separate browser install or runtime download.

func RegisterRule 

func RegisterRule(r Rule)

RegisterRule adds r to the global rule registry. Plugin init functions should call this to make their rules available.

func RenderURL 

func RenderURL(urlStr string) ([]byte, []string, error)

RenderURL loads the page at urlStr in headless Chrome and returns the rendered HTML along with JavaScript URLs fetched during the page load.

func ResetThrottle 

func ResetThrottle()

ResetThrottle clears all accumulated per-host pacing state, restoring the throttle to its configured base gap. It exists mainly so tests start from a known state.

func ResolveBrowser 

func ResolveBrowser() string

ResolveBrowser returns the path to a Chrome/Chromium executable to render with, provisioning one if necessary. An explicit -chrome-path / $JSMINER_CHROME override always wins. Otherwise, when downloads are enabled (the default), it always provisions the latest stable Chrome-for-Testing build — reusing that version if already cached, downloading it if not — so renders use an up-to-date browser. When downloads are disabled or fail (e.g. offline) it falls back to any previously cached build, a Chromium bundled next to the jsminer executable, or a browser on PATH. It returns "" only when every option is exhausted, leaving chromedp to try its own detection so behaviour is never worse than before.

func SetAutoDownloadBrowser 

func SetAutoDownloadBrowser(on bool)

SetAutoDownloadBrowser toggles on-demand browser provisioning.

func SetChromePath 

func SetChromePath(path string)

SetChromePath configures an explicit Chrome/Chromium executable path for rendering. An empty value restores chromedp's PATH-based auto-detection.

func SetExtraHeaders 

func SetExtraHeaders(h http.Header)

SetExtraHeaders replaces the global extra headers used for all outgoing HTTP requests. It makes a copy of the provided header map.

func SetFetchRetries 

func SetFetchRetries(n int)

SetFetchRetries configures how many extra attempts a transient transport error earns on the safe, bodyless read path. A negative value is treated as zero.

func SetFollowRedirects 

func SetFollowRedirects(follow bool)

SetFollowRedirects configures whether HTTP 3xx responses are followed. When disabled, the redirect response itself is returned without sending the next request, regardless of whether its Location is on the same host, a subdomain, or an unrelated domain.

func SetHTTPTimeout 

func SetHTTPTimeout(seconds int)

SetHTTPTimeout configures the per-request timeout for the shared HTTP fetch path (page and script fetches, calibration probes, method probes, sitemaps). A non-positive value restores the default. Enterprise crawls of large bundles over slow links need this raised; interactive scans of flaky hosts may want it lowered so a single stalled request cannot hold up the whole crawl.

func SetHostRateFloor 

func SetHostRateFloor(host string, gap time.Duration)

SetHostRateFloor records a minimum inter-request gap for a single host, used to honour that site's robots.txt Crawl-delay. The floor is combined with (never lowers) the global base gap and is the level adaptive decay eases back toward, so the crawl never paces faster than the site asked for that host. A larger floor replaces a smaller one; a non-positive gap is ignored. Safe to call before or during a scan.

func SetMaxBackoff 

func SetMaxBackoff(d time.Duration)

SetMaxBackoff overrides the ceiling on the adaptive gap and any honoured Retry-After / reset hold. A non-positive value restores the default.

func SetMaxExploreStates 

func SetMaxExploreStates(n int)

SetMaxExploreStates configures how many additional application states the renderer reaches through interaction. A non-positive value disables interaction-based exploration, rendering each page exactly once.

func SetRateLimit 

func SetRateLimit(perSecond float64)

SetRateLimit configures proactive request spacing for the shared HTTP path, expressed as a maximum number of requests per second per host. A value <= 0 disables proactive spacing (the default), leaving only adaptive backoff and budget-aware pre-emption active. It is safe to call before a scan starts.

func SetRateLimitJitter 

func SetRateLimitJitter(fraction float64)

SetRateLimitJitter sets the fraction (e.g. 0.2 for ±20%) by which each computed inter-request gap is randomised, breaking up the perfectly regular cadence a paced crawl would otherwise produce. A non-positive value disables jitter (the default). Values above 1 are clamped to 1.

func SetRenderSleepDuration 

func SetRenderSleepDuration(seconds int)

SetRenderSleepDuration allows customizing the sleep duration for page rendering

func SetSkipTLSVerification 

func SetSkipTLSVerification(skip bool)

SetSkipTLSVerification configures whether HTTPS certificate verification should be skipped

func SetVerboseWriter 

func SetVerboseWriter(w io.Writer)

SetVerboseWriter redirects verbose diagnostics away from stderr, mainly so tests can capture them. Passing nil restores the default (os.Stderr).

func SetVerbosity 

func SetVerbosity(level int)

SetVerbosity sets the global verbose logging level (see Verbosity). Negative values are clamped to 0.

func SeverityRank 

func SeverityRank(sev string) int

SeverityRank exposes the severity ordering to callers outside the package — chiefly the CLI's -fail-on threshold check. Higher ranks are more severe; an unrecognised label ranks 0, which callers use to detect an invalid threshold.

func SortBySeverity 

func SortBySeverity(ms []Match)

SortBySeverity orders matches from highest to lowest severity, preserving the original relative order within each band so discovery order is kept for ties.

func SortDOMFindings 

func SortDOMFindings(findings []DOMFinding)

SortDOMFindings orders findings by severity (desc) then fingerprint (asc), matching DedupDOMFindings' ordering so any list can be presented consistently.

func SortReflectionFindings 

func SortReflectionFindings(findings []ReflectionFinding)

SortReflectionFindings orders findings by severity (desc) then fingerprint (asc) for stable, reproducible output.

func WalkDir 

func WalkDir(root string) (map[string]io.ReadCloser, error)

WalkDir walks directory and returns list of readers with their filenames

func WarmBrowser 

func WarmBrowser()

WarmBrowser resolves (and, if necessary, downloads) the render browser now, caching the result for later renders. The CLI calls it once at startup when rendering is enabled so any first-run download happens up front — with a visible notice — instead of silently stalling the first page render mid-scan.

Types

type CrawlOptions 

type CrawlOptions struct {
	// MaxDepth is the number of link hops to follow beyond the seed page. A
	// depth of 0 scans only the seed (matching a plain ScanURL); 1 also scans
	// endpoints found on the seed, and so on. A negative value means unlimited
	// depth: the crawl follows links until the link graph is exhausted (or the
	// page budget/scope stops it), which is what -crawl-all requests.
	MaxDepth int

	// MaxPages caps the total number of pages fetched by the crawl, protecting
	// against runaway link graphs and parameterised URL explosions. Zero means
	// no cap (bounded only by MaxDepth and scope).
	MaxPages int

	// SameScopeOnly restricts crawling to the seed host and its subdomains (see
	// sameScope). Off-scope endpoints are still reported when they surface as
	// matches, they are simply not crawled. This is the expected default: the
	// user asked to follow discovered URLs only when they match the host.
	SameScopeOnly bool

	// Permute turns on cross-level path permutation: every discovered relative
	// path is reused under every directory level the crawl has seen, so a path
	// found in one place is also tried under other levels on the same origin (see
	// permuter). A bounded set of useful suffix variants is considered too, and
	// candidates are ranked before enqueue. It is off by default because it
	// multiplies requests; PermuteMax bounds it.
	Permute bool

	// PermuteMax caps the number of path-permutation URLs successfully admitted
	// to the crawl. Already-known, duplicate and template-rejected URLs do not
	// consume it. Zero means no cap (bounded only by MaxPages and scope).
	PermuteMax int

	// AutoCalibrate turns on ffuf-style auto-calibration: before crawling, the
	// target is probed with random paths to learn its catch-all/soft-404
	// fingerprint, and pages matching that fingerprint — or duplicating a page
	// already scanned — are skipped so the crawl stays on unique, useful pages.
	// It defaults to on (see DefaultCrawlOptions); the CLI always enables it and
	// exposes no toggle. The field is retained so library callers and tests can
	// opt out.
	AutoCalibrate bool

	// ProbeMethods turns on multi-method probing: every page the crawl visits is
	// requested with each verb in RequestMethods, and the verbs that work — judged
	// against the per-method, per-level error logic learned by auto-calibration —
	// are reported as a gathered-URL finding. It defaults to on (see
	// DefaultCrawlOptions) and is off in a zero-value CrawlOptions so library
	// callers and existing tests are unaffected.
	ProbeMethods bool

	// RequestMethods lists the HTTP methods used by ProbeMethods. Empty means the
	// default set (GET, POST, PUT, PATCH, DELETE, OPTIONS).
	RequestMethods []string

	// ParamReplay turns on cross-level parameter replay: parameter bodies
	// discovered on POST/PUT/PATCH endpoints are replayed against every directory
	// level the crawl has seen, and replays that work against a level's learned
	// per-method error logic are reported as gathered-URL findings. It requires
	// ProbeMethods and defaults to on (see DefaultCrawlOptions); ParamReplayMax
	// bounds it.
	ParamReplay bool

	// ParamReplayMax caps the number of (level, parameter) replays generated when
	// ParamReplay is set. Zero means no cap (bounded only by scope and the crawl).
	ParamReplayMax int

	// TemplateDedup collapses templated duplicate pages — pages that are
	// structurally the same and differ only in data, such as /product/1 vs
	// /product/2, paginated listings and calendar/faceted URLs — so the crawl
	// fetches only a representative few of each class instead of every instance,
	// spending its page budget on genuinely distinct pages. It works on two
	// levels: discovered URLs are grouped by a normalised URL template before they
	// are fetched (see templateClasser), so a suppressed instance costs no request
	// at all; and fetched pages are additionally grouped by a structural body
	// signature (see structuralSig), catching templated pages whose URLs do not
	// reveal the pattern. It defaults to on (see DefaultCrawlOptions) and is off in
	// a zero-value CrawlOptions so library callers and existing tests are
	// unaffected. TemplateSampleMax bounds how many representatives are kept.
	TemplateDedup bool

	// TemplateSampleMax caps how many representative pages are crawled from each
	// template class when TemplateDedup is set. Zero selects a sensible default
	// (defaultTemplateSampleMax).
	TemplateSampleMax int

	// DiscoverWellKnown seeds the crawl from the URLs the site declares about
	// itself: robots.txt (Allow/Disallow directories and Sitemap: pointers) and
	// the XML sitemaps they and convention advertise (see discoverWellKnownURLs).
	// These are real, server-published paths, so they reach pages and API roots
	// that nothing links to and that static JS scanning never reveals. It defaults
	// to on (see DefaultCrawlOptions) and is off in a zero-value CrawlOptions so
	// library callers and existing tests are unaffected.
	DiscoverWellKnown bool

	// DiscoverPassive asks public web indexes for URLs historically observed on
	// the exact seed host. Historical URLs are path hints, not trusted targets:
	// query values are discarded, paths are rebased onto the current seed origin,
	// and each is accepted only when its live status and catch-all fingerprint
	// prove it still exists. A validated path becomes an ordinary real source for
	// Permute; rejected hints never enter its dictionary. It is opt-in because it
	// contacts third-party archives and adds validation requests to the target.
	DiscoverPassive bool

	// PassiveSources selects public indexes used by DiscoverPassive. Supported
	// values are "wayback" and "commoncrawl"; empty selects both.
	PassiveSources []string

	// PassiveMax caps the number of sanitized historical path hints admitted for
	// live validation. Values <= 0 select defaultPassiveMax; passive discovery is
	// never unbounded independently of the crawl's own page budget.
	PassiveMax int

	// OnDOMSourceHints receives sanitized source names learned outside response
	// bodies (currently archived query names). Values are never included. The CLI
	// uses this only when a later DOM scan is enabled.
	OnDOMSourceHints func([]DOMSourceHint)

	// ResumeFile, when non-empty, turns on crawl checkpointing: the crawl reloads
	// its state from this file at start (when it holds a checkpoint for the same
	// seed) and periodically writes its state back to it, so a run killed part way
	// through — the risk on a large -crawl-all — can be resumed instead of started
	// over. The file is removed on clean completion. Empty (the default) disables
	// checkpointing entirely.
	ResumeFile string

	// Concurrency is how many pages the crawl fetches and scans in parallel. A
	// crawl is dominated by per-page I/O — the HTTP fetch and, when rendering is
	// on, a headless-Chrome render that can take seconds — so processing several
	// pages at once is the main throughput win. Values <= 1 run the crawl serially
	// (the original, fully deterministic behaviour); higher values dispatch that
	// many pages to a worker pool. Per-host pacing and adaptive backoff (see
	// requestThrottle) still bound the load any single host sees, so raising this
	// speeds up render-heavy and multi-host crawls without abandoning politeness.
	// Note that with rendering on each worker may run its own browser, so the
	// setting also trades memory for speed. Zero means serial in a zero-value
	// CrawlOptions; DefaultCrawlOptions sets it to defaultCrawlConcurrency.
	Concurrency int

	// Progress, when non-nil, is invoked once per fetched page with the page
	// URL, its depth from the seed and the running page count. It lets the CLI
	// surface crawl progress without the scan package depending on the output
	// layer. During a concurrent crawl it is called as each page is dispatched, so
	// the page numbers stay monotonic even though pages then complete out of order.
	Progress func(pageURL string, depth, pageNum int)

	// OnCalibrated, when non-nil, is invoked once after auto-calibration with
	// the number of wildcard signatures learned.
	OnCalibrated func(wildcardSigs int)

	// OnComplete, when non-nil, is invoked once when the crawl finishes with a
	// summary of what it did (see CrawlStats). It lets the CLI print an
	// end-of-run report — pages fetched, targets discovered, errors, duration —
	// without the scan package depending on the output layer.
	OnComplete func(CrawlStats)
}

CrawlOptions configures a breadth-first crawl seeded from a single URL.

A crawl scans the seed page, harvests the endpoints it discovers (endpoint_url/endpoint_path and, for POST crawls, post_url/post_path), resolves the in-scope ones to absolute URLs and fetches them too, repeating until the depth or page budget is exhausted. Each fetched page is scanned with the normal rules, so crawling reaches JavaScript bundles — and the secrets in them — that are only linked from deeper pages or API responses.

func DefaultCrawlOptions 

func DefaultCrawlOptions() CrawlOptions

DefaultCrawlOptions returns sensible defaults for interactive use: follow two hops beyond the seed, stay on the seed host, stop after 200 pages, auto-calibrate to suppress catch-all/soft-404 and duplicate pages, and fetch several pages in parallel.

type CrawlStats 

type CrawlStats struct {
	// PagesFetched is the number of pages whose fetch completed without a
	// transport/read error, including passive hints subsequently rejected by live
	// validation. PagesErrored is the number whose fetch or scan returned an error.
	PagesFetched int
	PagesErrored int

	// WellKnownSeeds is how many URLs the crawl seeded from robots.txt/sitemaps.
	WellKnownSeeds int

	// PassiveFound is the bounded number of sanitized historical path hints,
	// PassiveEnqueued is how many survived normal crawl dedup/template admission,
	// and Validated/Rejected report the result of live status + catch-all checks.
	PassiveFound     int
	PassiveEnqueued  int
	PassiveValidated int
	PassiveRejected  int

	// TargetsFound is the total in-scope crawl targets discovered across all
	// pages (with repeats), and Enqueued is the number of distinct pages that
	// were actually queued for crawling over the whole run.
	TargetsFound int
	Enqueued     int

	// Matches is the number of deduplicated matches the crawl returned, and
	// WildcardSigs is how many catch-all/soft-404 signatures auto-calibration
	// learned for the root.
	Matches      int
	WildcardSigs int

	// PermuteConsidered is the number of combinations evaluated.
	// PermuteEnqueued counts only candidates admitted by normal crawl dedup and
	// template checks, so it is the value bounded by PermuteMax. PermuteFetched
	// and PermuteYielded report how many synthetic pages were successfully scanned
	// and how many produced at least one match or fresh crawl target.
	PermuteConsidered       int
	PermuteSkippedKnown     int
	PermuteSkippedAdmission int
	PermutePruned           int
	PermuteEnqueued         int
	PermuteFetched          int
	PermuteYielded          int

	// Duration is the wall-clock time the crawl took.
	Duration time.Duration
}

CrawlStats summarises a completed crawl. It is reported once through CrawlOptions.OnComplete so operators get the run-level accounting an enterprise crawl is expected to surface — throughput, reach and failures — rather than only the per-page verbose narrative.

type DOMFinding 

type DOMFinding struct {
	Type    string `json:"type"`
	Target  string `json:"target"`
	PageURL string `json:"page_url"`

	FrameURL  string `json:"frame_url,omitempty"`
	FramePath string `json:"frame_path,omitempty"`

	Source *DOMSource `json:"source,omitempty"`
	Sink   *DOMSink   `json:"sink,omitempty"`

	ProbeID string `json:"probe_id,omitempty"`

	// ValuePreview is a bounded, redaction-safe excerpt of the value seen at the
	// sink. It never contains full secrets, cookies, storage values or message
	// contents.
	ValuePreview string `json:"value_preview,omitempty"`

	// Context classifies the sink's parse context: html, js, url or attribute.
	Context string `json:"context,omitempty"`

	Stack []DOMStackFrame `json:"stack,omitempty"`

	// Trigger is the primary trigger category (page_load, interaction,
	// post_message). Triggers holds every distinct trigger a deduplicated flow was
	// observed through.
	Trigger  string   `json:"trigger,omitempty"`
	Triggers []string `json:"triggers,omitempty"`

	// Interaction describes the specific interaction that drove the flow (e.g. a
	// form submission or a clicked control), when applicable.
	Interaction string `json:"interaction,omitempty"`

	// Transform records an observed source transformation or sanitization between
	// source and sink (e.g. "url_decoded", "html_encoded").
	Transform string `json:"transform,omitempty"`

	// Phase records whether the flow occurred during initial loading or later
	// state exploration.
	Phase string `json:"phase,omitempty"`

	Severity   string `json:"severity"`
	Confidence string `json:"confidence"`
	Confirmed  bool   `json:"confirmed"`

	// Message carries postMessage-specific evidence for web_message findings.
	Message *DOMMessageInfo `json:"message,omitempty"`

	// URL carries structural destination evidence for URL/navigation sinks.
	URL *DOMURLEvidence `json:"url,omitempty"`

	// Triage explains, in plain terms, how much attention the current evidence
	// deserves. It never upgrades severity and does not replace manual review.
	Triage *DOMTriage `json:"triage,omitempty"`

	// Fingerprint is a deterministic dedup identity derived only from stable
	// properties (never from random canaries, timestamps or transient ids).
	Fingerprint string `json:"fingerprint,omitempty"`

	// Notes carries any diagnostic annotations (e.g. that instrumentation appeared
	// to break page execution). It is advisory context, not part of identity.
	Notes string `json:"notes,omitempty"`
}

DOMFinding is the richer, DOM-specific evidence model. It is intentionally not compressed into the generic Match.Params string so automated triage has every field it needs. Unset optional fields are omitted from output.

func DedupDOMFindings 

func DedupDOMFindings(findings []DOMFinding) []DOMFinding

DedupDOMFindings collapses findings that share a fingerprint into a single record, combining their trigger evidence and keeping the strongest severity, confidence and confirmation. The result is deterministically ordered (severity desc, then fingerprint asc) so identical scans emit identical output. Each returned finding carries its computed fingerprint.

type DOMMessageInfo 

type DOMMessageInfo struct {
	// ListenerCount is how many message listeners were observed on the frame.
	ListenerCount int `json:"listener_count,omitempty"`

	// OriginChecked reports whether a listener's source appears to inspect
	// event.origin. This is evidence the listener looks at origin, NOT proof that
	// origin validation is correct or that it was bypassed.
	OriginChecked bool `json:"origin_checked"`

	// OriginCheckedListeners is the number of observed listeners whose source
	// appears to inspect event.origin. The aggregate OriginChecked boolean is
	// retained for backwards compatibility.
	OriginCheckedListeners int `json:"origin_checked_listeners,omitempty"`

	// SourceChecked reports whether a listener's source appears to inspect
	// event.source. As with OriginChecked, this is evidence, not proof.
	SourceChecked bool `json:"source_checked"`

	// SourceCheckedListeners is the number of observed listeners whose source
	// appears to inspect event.source.
	SourceCheckedListeners int `json:"source_checked_listeners,omitempty"`

	// DataShape is the expected message-data shape when determinable (the property
	// names a listener reads off event.data), e.g. "{cmd, payload}".
	DataShape string `json:"data_shape,omitempty"`

	// ReachesSink reports whether message data was observed reaching a dangerous
	// sink.
	ReachesSink bool `json:"reaches_sink"`

	// ProbeGenerated distinguishes the scanner's deliberately injected message
	// shapes from messages the application generated on its own.
	ProbeGenerated bool `json:"probe_generated"`

	// SentToOrigin, when set, records that the page sent URL-derived data to a
	// different origin via postMessage — a potential cross-origin data leak.
	SentToOrigin string `json:"sent_to_origin,omitempty"`

	// Identity is a stable grouping key for duplicate messages (origin + shape),
	// so a page that emits the same message repeatedly is reported once.
	Identity string `json:"identity,omitempty"`

	// ListenerLocations points to the application code that registered the
	// observed message listeners, when a useful stack frame was available.
	ListenerLocations []DOMStackFrame `json:"listener_locations,omitempty"`
}

DOMMessageInfo carries postMessage-specific evidence.

type DOMScanConfig 

type DOMScanConfig struct {
	Mode        string
	MaxPages    int
	MaxProbes   int
	Workers     int
	PageTimeout time.Duration

	// Sources and Sinks, when non-nil, restrict the enabled source families and
	// sink families respectively. Nil means all enabled.
	Sources map[string]bool
	Sinks   map[string]bool

	// SourceHints are parameter/storage/cookie names mined by the static crawl or
	// passive indexes. MaxSourceHintsPerPage bounds how many are applied to one
	// rendered route; zero selects the default rather than removing the bound.
	SourceHints           []DOMSourceHint
	MaxSourceHintsPerPage int

	// Messages enables postMessage analysis as a separately controllable feature.
	Messages bool

	// Crawl follows in-scope links discovered in the rendered DOM to reach more
	// pages, bounded by MaxPages and MaxDepth. Off scans only the seed targets.
	Crawl    bool
	MaxDepth int

	// AllowExternal permits navigation to and probing of third-party origins. Off
	// by default: generated probes must not be sent off-scope, and a client-side
	// redirect out of scope is not followed for probing.
	AllowExternal bool

	// CollectRenderedArtifacts makes the instrumented DOM navigation the shared
	// browser pass for ordinary rendered discovery too. The resulting HTML,
	// scripts and live requests are scanned after the browser phase and returned
	// as ordinary Matches, avoiding a preceding RenderURLWithStates navigation.
	CollectRenderedArtifacts bool
	ArtifactEndpoints        bool
	ArtifactPosts            bool
	ArtifactExternal         bool

	// Progress, when set, receives short human-readable status lines (stderr).
	Progress func(msg string)
}

DOMScanConfig configures a DOM vulnerability scan. Every limit is independent of the crawl's own limits so DOM scanning can be bounded on its own terms.

func DefaultDOMScanConfig 

func DefaultDOMScanConfig() DOMScanConfig

DefaultDOMScanConfig returns the default DOM scan configuration: canary mode, bounded pages and probes, a modest worker pool and postMessage analysis on.

type DOMScanResult 

type DOMScanResult struct {
	Findings []DOMFinding
	// Matches are ordinary findings and endpoint discoveries collected from the
	// same instrumented page loads when CollectRenderedArtifacts is enabled.
	Matches []Match
	// SourceHints are additional scoped names learned from scripts that only the
	// shared browser pass discovered. They feed later non-rendering reflection
	// probes without leaking names between separate CLI targets.
	SourceHints []DOMSourceHint
	Summary     DOMScanSummary
}

DOMScanResult is the deduplicated findings and the summary of a DOM scan.

type DOMScanSummary 

type DOMScanSummary struct {
	SchemaVersion string `json:"schema_version"`
	Mode          string `json:"mode"`
	PagesScanned  int    `json:"pages_scanned"`
	PagesFailed   int    `json:"pages_failed"`
	ProbesSent    int    `json:"probes_sent"`
	ProbesLimit   int    `json:"probes_limit"`
	MaxPages      int    `json:"max_pages"`
	Findings      int    `json:"findings"`
	// SuppressedMessages counts web_message observations dropped as chatter: a
	// message with no listener to receive it and no security-sensitive effect
	// (framework, analytics and third-party-iframe traffic).
	SuppressedMessages int            `json:"suppressed_messages"`
	FindingsBySeverity map[string]int `json:"findings_by_severity"`
	Partial            bool           `json:"partial"`
	TimedOut           bool           `json:"timed_out"`
	DurationMS         int64          `json:"duration_ms"`
	Errors             []string       `json:"errors,omitempty"`
	SourceHints        int            `json:"source_hints"`
	HintProbesSent     int            `json:"hint_probes_sent"`
}

DOMScanSummary is the machine-readable end-of-scan record.

type DOMSink 

type DOMSink struct {
	Name     string `json:"name"`
	Argument int    `json:"argument"`
}

DOMSink identifies the security-sensitive browser API a flow reached and which argument carried the controllable data.

type DOMSource 

type DOMSource struct {
	Kind         string   `json:"kind"`
	Name         string   `json:"name,omitempty"`
	DiscoveredBy []string `json:"discovered_by,omitempty"`
}

DOMSource identifies the attacker-controllable input family (kind) and the specific input (name, e.g. a query parameter name or cookie name) a flow originated from.

type DOMSourceHint 

type DOMSourceHint struct {
	Kind       string   `json:"kind"`
	Name       string   `json:"name"`
	ScopeHost  string   `json:"scope_host,omitempty"`
	Discovered []string `json:"discovered_by,omitempty"`
}

DOMSourceHint is passive/static intelligence that can be turned into a unique DOM canary. ScopeHost keeps hints from separate CLI targets isolated; an empty scope lets library callers deliberately apply a hint to every seed.

type DOMStackFrame 

type DOMStackFrame struct {
	Function string `json:"function,omitempty"`
	URL      string `json:"url,omitempty"`
	Line     int    `json:"line,omitempty"`
	Column   int    `json:"column,omitempty"`
}

DOMStackFrame is one frame of a captured JavaScript call stack, locating the code that drove data into a sink.

type DOMTriage 

type DOMTriage struct {
	Verdict string `json:"verdict"`
	Reason  string `json:"reason"`
}

DOMTriage is a conservative, evidence-backed hint for deciding whether a finding deserves manual investigation. It is not a vulnerability verdict.

type DOMURLEvidence 

type DOMURLEvidence struct {
	Resolved          bool   `json:"resolved"`
	Scheme            string `json:"scheme,omitempty"`
	DestinationOrigin string `json:"destination_origin,omitempty"`
	SameOrigin        bool   `json:"same_origin"`
	CanaryComponent   string `json:"canary_component,omitempty"`
	InputKind         string `json:"input_kind,omitempty"`
	ExecutableScheme  bool   `json:"executable_scheme"`
}

DOMURLEvidence explains what a URL/navigation sink would actually target. It intentionally exposes only structural details, not a second copy of the potentially sensitive full URL (ValuePreview remains bounded and redacted).

type Extractor 

type Extractor struct {
	// contains filtered or unexported fields
}

Extractor holds compiled regex patterns

func NewExtractor 

func NewExtractor(safe bool, longSecret bool) *Extractor

NewExtractor creates an Extractor

func (*Extractor) AddDOMSourceHints 

func (e *Extractor) AddDOMSourceHints(hints []DOMSourceHint)

AddDOMSourceHints merges externally discovered hints (currently passive web indexes) into the same per-target corpus as JavaScript-derived hints.

func (*Extractor) LoadAllowlist 

func (e *Extractor) LoadAllowlist(path string) error

LoadAllowlist loads allowed domain suffixes

func (*Extractor) LoadRulesFile 

func (e *Extractor) LoadRulesFile(path string) error

LoadRulesFile loads additional regex patterns from a YAML file

func (*Extractor) ScanDOM 

func (e *Extractor) ScanDOM(ctx context.Context, targets []string, cfg DOMScanConfig) (DOMScanResult, error)

ScanDOM runs the opt-in DOM vulnerability scan over the given URL targets. It reuses the package's browser provisioning, headers/cookies, TLS, redirect, throttle and timeout configuration. It never changes any behaviour unless called, and returns findings plus a summary. A single failed page does not discard findings from successful pages; context cancellation stops active browser work cleanly and yields a clearly-marked partial result.

func (*Extractor) ScanDir 

func (e *Extractor) ScanDir(root string, workers int) ([]Match, error)

ScanDir scans all supported files under root directory using workers to limit concurrency.

func (*Extractor) ScanReader 

func (e *Extractor) ScanReader(source string, r io.Reader) ([]Match, error)

ScanReader scans an io.Reader and returns matches

func (*Extractor) ScanReaderAST 

func (e *Extractor) ScanReaderAST(source string, r io.Reader) ([]Match, error)

ScanReaderAST scans JavaScript source using an AST and applies regex patterns to all discovered string values. Only JavaScript files are processed when safe mode is enabled.

func (*Extractor) ScanReaderPostRequests 

func (e *Extractor) ScanReaderPostRequests(source string, r io.Reader) ([]Match, error)

ScanReaderPostRequests extracts HTTP POST request endpoints from r. Matches use the pattern name "post_url" for absolute URLs and "post_path" for relative paths. Only JavaScript files are processed when safe mode is enabled.

func (*Extractor) ScanReaderWithEndpoints 

func (e *Extractor) ScanReaderWithEndpoints(source string, r io.Reader) ([]Match, error)

ScanReaderWithEndpoints scans r like ScanReader and also extracts HTTP endpoints from JavaScript sources. Endpoint matches use the pattern name "endpoint_url" for absolute URLs and "endpoint_path" for relative paths.

func (*Extractor) ScanReflections 

func (e *Extractor) ScanReflections(ctx context.Context, targets []string, cfg ReflectionScanConfig) (ReflectionScanResult, error)

ScanReflections replays gathered parameters against the given URL targets and reports server-side reflections. It reuses the package's HTTP client, headers, TLS, redirect, throttle and timeout configuration, stays within each target's scope (unless AllowExternal), and honours the URL, parameter and probe budgets. Context cancellation stops the scan cleanly and marks the result partial.

func (*Extractor) ScanURL 

func (e *Extractor) ScanURL(urlStr string, endpoints bool, external bool, render bool) ([]Match, error)

ScanURL scans urlStr and any discovered script or import references. Cross-domain resources are followed by default. Set external to false to restrict scanning to the same domain. JavaScript files are scanned using the configured rules. ScanURL scans urlStr and any discovered script or import references. When endpoints is true, only endpoint matches are returned.

func (*Extractor) ScanURLCrawl 

func (e *Extractor) ScanURLCrawl(urlStr string, endpoints, external, render bool, opts CrawlOptions) ([]Match, error)

ScanURLCrawl scans urlStr and then crawls the in-scope endpoints it discovers, returning the deduplicated union of all matches. When endpoints is true only endpoint matches are produced (as with ScanURL). external controls whether off-scope script/import references are followed while scanning an individual page; the page-to-page crawl itself is governed by opts.

func (*Extractor) ScanURLPosts 

func (e *Extractor) ScanURLPosts(urlStr string, external bool, render bool) ([]Match, error)

ScanURLPosts scans urlStr and discovered script/import references returning only HTTP POST request endpoints found in JavaScript sources.

func (*Extractor) ScanURLPostsCrawl 

func (e *Extractor) ScanURLPostsCrawl(urlStr string, external, render bool, opts CrawlOptions) ([]Match, error)

ScanURLPostsCrawl behaves like ScanURLCrawl but scans each page for HTTP POST request endpoints, following the discovered endpoints to reach deeper pages.

func (*Extractor) SetCalibrator 

func (e *Extractor) SetCalibrator(c *autoCalibrator)

SetCalibrator installs (or clears, when nil) an auto-calibrator used during crawls to skip catch-all/soft-404 and duplicate pages. It is nil by default, leaving non-crawl scans unaffected.

func (*Extractor) SetCollectDOMSourceHints 

func (e *Extractor) SetCollectDOMSourceHints(on bool)

SetCollectDOMSourceHints enables the hidden intelligence pass used by -dom and -full. Keeping it opt-in avoids extra POST-expression parsing for ordinary secret-only scans.

func (*Extractor) SetRecoverSourceMaps 

func (e *Extractor) SetRecoverSourceMaps(on bool)

SetRecoverSourceMaps toggles recovery of original source from JavaScript source maps. When on (the default), a scanned JS bundle that advertises a source map has its original, pre-bundled sources recovered and scanned so their secrets and endpoints surface as ordinary matches. Disabling it skips all source-map fetching and decoding.

func (*Extractor) SetSnippet 

func (e *Extractor) SetSnippet(on bool)

SetSnippet toggles capture of a raw source window around each match so the output layer can render a code excerpt. It is disabled by default because locating every value in the source adds work proportional to the input size.

func (*Extractor) TakeDOMSourceHints 

func (e *Extractor) TakeDOMSourceHints() []DOMSourceHint

TakeDOMSourceHints returns the current deterministic corpus and clears it so the CLI can associate the hints with exactly one target before scanning the next target.

type FilterRegexRule 

type FilterRegexRule struct {
	Name     string
	RE       *regexp.Regexp
	Severity string
	Filter   func(string) bool
}

FilterRegexRule implements Rule with an optional post-match filter.

func (FilterRegexRule) Find 

func (r FilterRegexRule) Find(data []byte) []Match

func (FilterRegexRule) MatchName 

func (r FilterRegexRule) MatchName() string

type HTTPRequest 

type HTTPRequest struct {
	URL  string
	Body string
}

HTTPRequest represents a captured HTTP request.

func RenderURLWithRequests 

func RenderURLWithRequests(urlStr string) ([]byte, []string, []HTTPRequest, error)

RenderURLWithRequests loads the page and captures POST requests made during rendering. It returns the rendered HTML, JavaScript URLs and POST requests.

func RenderURLWithStates 

func RenderURLWithStates(urlStr string) ([][]byte, []string, []HTTPRequest, []string, error)

RenderURLWithStates loads the page and then explores application state that only appears after interaction: it clicks client-side navigation controls and fills forms with plausible valid values and submits them, snapshotting each distinct DOM state it reaches. It returns those state snapshots (the initial load first), the union of JavaScript URLs seen across all states, the POST requests captured throughout, and the URLs of the XHR/fetch API calls the page made — so a single-page app whose surface lives behind event handlers is scanned in every state, not just the shell it first renders, and the API endpoints its dynamic navigation calls are discovered even when no bundle mentions them literally.

Interaction is bounded by MaxExploreStates; when that is zero the result is a single state and this behaves like RenderURLWithRequests.

type Match 

type Match struct {
	Source   string `json:"source"`
	Pattern  string `json:"pattern"`
	Value    string `json:"value"`
	Params   string `json:"params,omitempty"`
	Severity string `json:"severity"`

	// Snippet holds a raw source window surrounding the matched value. It is
	// only populated when snippet capture is enabled (see SetSnippet) and is
	// consumed by the output layer to render a prettified, highlighted code
	// excerpt. It is intentionally excluded from the default JSON encoding.
	Snippet string `json:"-"`
}

Match represents a single regex hit

func FilterEndpointMatches 

func FilterEndpointMatches(ms []Match) []Match

FilterEndpointMatches returns only endpoint matches from ms.

func FilterGatheredMatches 

func FilterGatheredMatches(ms []Match) []Match

FilterGatheredMatches returns only the gathered-URL findings from ms, preserving order. It lets the CLI keep the gathered-URL segment when the endpoint-only filter would otherwise drop it.

func FilterPostMatches 

func FilterPostMatches(ms []Match) []Match

FilterPostMatches returns only the matches relevant to POST-request output: the post_url/post_path endpoints and the crawl's gathered-URL findings. It exists so a -posts crawl can harvest HTML markup links to follow the link graph (emitted as endpoint_url matches for navigation) without those navigation-only links leaking into the POST-endpoint results.

func UniqueMatches 

func UniqueMatches(ms []Match) []Match

UniqueMatches returns a new slice containing only the first occurrence of each pattern/value pair from ms. The original order is preserved for the first occurrence.

type ReflectionFinding 

type ReflectionFinding struct {
	Type      string `json:"type"`
	Target    string `json:"target"`   // scheme://host
	PageURL   string `json:"page_url"` // route probed (path + query name), marker redacted
	Parameter string `json:"parameter"`
	Method    string `json:"method"`

	// Context classifies where the marker landed: html_text, html_attribute,
	// html_comment, script or unknown.
	Context string `json:"context"`

	// Occurrences is how many times the marker was reflected in the response.
	Occurrences int `json:"occurrences,omitempty"`

	// Unfiltered lists the breakout metacharacters that were reflected without
	// encoding. An empty list on a reflected finding means the dangerous
	// characters were encoded (or their survival could not be determined).
	Unfiltered []string `json:"unfiltered,omitempty"`

	// ValuePreview is a bounded, redaction-safe excerpt of the response around the
	// reflection point.
	ValuePreview string `json:"value_preview,omitempty"`

	// DiscoveredBy records how the parameter name was learnt (JS access, request
	// body, passive archive, or the route's own query string).
	DiscoveredBy []string `json:"discovered_by,omitempty"`

	Severity   string     `json:"severity"`
	Confidence string     `json:"confidence"`
	Triage     *DOMTriage `json:"triage,omitempty"`

	// Fingerprint is a deterministic dedup identity derived only from stable
	// properties (never from the random marker).
	Fingerprint string `json:"fingerprint,omitempty"`

	// Notes carries advisory diagnostics (e.g. that the reflection boundary was
	// altered so character filtering could not be determined).
	Notes string `json:"notes,omitempty"`
}

ReflectionFinding is one parameter whose value was reflected into a target's HTTP response. It is intentionally a distinct model from DOMFinding: this is a server-side reflection observed without a browser, not a DOM source-to-sink flow.

func DedupReflectionFindings 

func DedupReflectionFindings(findings []ReflectionFinding) []ReflectionFinding

DedupReflectionFindings collapses findings that share a fingerprint into one record, keeping the strongest severity/confidence and the union of surviving characters and discovery sources. The result is deterministically ordered (severity desc, then fingerprint asc).

type ReflectionScanConfig 

type ReflectionScanConfig struct {
	// MaxURLs bounds how many distinct routes are probed (0 = unlimited).
	MaxURLs int
	// MaxParams bounds how many parameter names are tested per route.
	MaxParams int
	// MaxProbes bounds the total number of HTTP requests the whole scan may send
	// (0 = unlimited), so a large parameter corpus cannot cause unbounded traffic.
	MaxProbes int
	// Workers is the number of routes probed in parallel.
	Workers int

	// ParamHints are the parameter names mined for the DOM scan. Only url_query
	// hints are used; each route also tests its own existing query-string names.
	ParamHints []DOMSourceHint

	// AllowExternal permits a probe's redirects to leave the target's scope. Off
	// by default so a reflection probe can never become a redirect-driven SSRF.
	AllowExternal bool

	// Progress, when set, receives short human-readable status lines (stderr).
	Progress func(msg string)
}

ReflectionScanConfig configures a reflection scan. Its bounds are independent of the DOM scan's so the two can be tuned separately, and it reuses the package's shared HTTP client, headers, TLS, redirect, throttle and timeout configuration.

func DefaultReflectionScanConfig 

func DefaultReflectionScanConfig() ReflectionScanConfig

DefaultReflectionScanConfig returns conservative defaults mirroring the DOM scan's page/param/probe bounds and worker count.

type ReflectionScanResult 

type ReflectionScanResult struct {
	Findings []ReflectionFinding
	Summary  ReflectionScanSummary
}

ReflectionScanResult is the deduplicated findings and the summary of a scan.

type ReflectionScanSummary 

type ReflectionScanSummary struct {
	SchemaVersion string `json:"schema_version"`
	URLsScanned   int    `json:"urls_scanned"`
	URLsFailed    int    `json:"urls_failed"`
	ProbesSent    int    `json:"probes_sent"`
	ProbesLimit   int    `json:"probes_limit"`
	ParamsTested  int    `json:"params_tested"`
	Findings      int    `json:"findings"`
	// SuppressedEchoes counts candidates dropped because their reflection was
	// indistinguishable from an arbitrary-name (whole-query) echo — i.e. the
	// parameter was not distinctly processed by the application.
	SuppressedEchoes   int            `json:"suppressed_echoes"`
	FindingsBySeverity map[string]int `json:"findings_by_severity"`
	Partial            bool           `json:"partial"`
	DurationMS         int64          `json:"duration_ms"`
	Errors             []string       `json:"errors,omitempty"`
}

ReflectionScanSummary is the machine-readable end-of-scan record.

type RegexRule 

type RegexRule struct {
	Name     string
	RE       *regexp.Regexp
	Severity string

	// Filter, when non-nil, is applied to every regex hit. Returning false drops
	// the match. It is used to reject the large volume of false positives that
	// broad keyword/credential patterns produce on minified bundles (e.g.
	// `token:e`, `password:!0`) without weakening the patterns themselves.
	Filter func(string) bool

	// ContextFilter, when non-nil, receives the whole input and the match bounds
	// so it can inspect the bytes surrounding a hit that the string-only Filter
	// cannot see — e.g. the character immediately after a `path` match, which
	// tells a real route from the head of a JS regex literal (`= /checked\s*…/`).
	// Returning false drops the match.
	ContextFilter func(data []byte, start, end int) bool
	// contains filtered or unexported fields
}

RegexRule implements Rule using a regular expression.

func (RegexRule) Find 

func (r RegexRule) Find(data []byte) []Match

func (RegexRule) MatchName 

func (r RegexRule) MatchName() string

type Rule 

type Rule interface {
	MatchName() string
	Find(data []byte) []Match
}

Rule defines an interface for matchers used by Extractor.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.