vulndb

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 5, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package vulndb is the offline, normalized advisory store behind the vuln module. It defines a single internal advisory schema that every feed (NVD, GHSA, OSV, distro security trackers) normalizes *into*, so vulnerability matching never depends on a feed's quirks. The store is a plain JSON file — no external database engine — that loads deterministically, carries a build timestamp for staleness reporting, and ships with an embedded snapshot so the tool works fully air-gapped out of the box.

Version comparison is the part of a scanner that "lives or dies" on accuracy: ecosystems disagree on version semantics (Debian epochs, RPM evr, Alpine apk suffixes, Go pseudo-versions, PEP 440, semver pre-releases, Maven qualifiers), so comparison is delegated per-scheme rather than assuming one ordering — the classic source of false matches. See version*.go.

Index

Constants

View Source 
const DefaultEPSSURL = "https://epss.empiricalsecurity.com/epss_scores-current.csv.gz"

DefaultEPSSURL is FIRST's daily full EPSS scoreset (gzipped CSV). The endpoint 302-redirects to the current dated file, so the client must follow redirects (net/http does by default).

View Source 
const DefaultKEVURL = "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json"

DefaultKEVURL is CISA's machine-readable KEV catalog.

Variables

This section is empty.

Functions

func Compare 

func Compare(sch VersionScheme, a, b string) int

Compare returns -1, 0, or +1 as version a is less than, equal to, or greater than b, using the ordering rules of the given scheme. Comparison is total and deterministic; unparseable input degrades to a byte-wise comparison rather than panicking, so hostile version strings can never crash a scan.

func DistroEOL 

func DistroEOL(name, version string, now time.Time) (time.Time, bool, bool)

DistroEOL reports whether the given distro release is past end-of-life at `now`. known=false means the release is not in the table (no claim made). Version matching uses the major (debian/centos) or major.minor (alpine/ubuntu) prefix so "3.16.9" matches "alpine 3.16".

func EcosystemFeedURL 

func EcosystemFeedURL(eco string) string

EcosystemFeedURL returns the public OSV export URL for one ecosystem (e.g. "Debian", "Alpine", "PyPI", "npm", "Go"). The ecosystem is path-escaped so a stray '#', '?', '%', or '/' in a user-supplied --ecosystems value can't be parsed as a URL fragment/query and silently truncate the path.

func EmbeddedBytes 

func EmbeddedBytes() []byte

EmbeddedBytes returns the raw embedded snapshot, for tooling that wants to write it to disk as a starting point.

func NormalizePyPI 

func NormalizePyPI(name string) string

NormalizePyPI applies PEP 503 name normalization (lowercase, collapse runs of "-", "_", "." to a single "-"), matching how PyPI advisory feeds key names.

func ParseEPSSCSV 

func ParseEPSSCSV(r io.Reader) (map[string]float64, error)

ParseEPSSCSV parses the EPSS CSV (pure, testable). The file starts with a `#model_version,score_date` comment line, then a `cve,epss,percentile` header, then one row per CVE. It tolerates the comment/header in any leading position and skips malformed rows. Returns CVE(upper)→epss.

func ParseKEV 

func ParseKEV(data []byte) ([]string, error)

ParseKEV parses the CISA KEV JSON (pure, testable): {"vulnerabilities":[{"cveID":"CVE-..."}]}. Returns sorted, unique, uppercased CVE IDs.

func Vulnerable 

func Vulnerable(sch VersionScheme, installed string, ranges []Range) bool

Vulnerable reports whether an installed version falls inside any of the advisory's affected ranges, comparing with the given scheme.

A range is "[Introduced, Fixed)" (Fixed exclusive) or "[Introduced, LastAffected]" (LastAffected inclusive) or, with neither upper bound, an open "[Introduced, ∞)". Introduced of "" or "0" means unbounded below. An empty installed version can never be judged and is treated as not-vulnerable, so a component we could not version never raises a (false) finding.

Types

type Advisory 

type Advisory struct {
	ID        string    `json:"id"`                // primary id, e.g. CVE-2023-45853 / GHSA-xxxx
	Aliases   []string  `json:"aliases,omitempty"` // other ids used for de-dup and enrichment lookups
	Summary   string    `json:"summary,omitempty"`
	Ecosystem Ecosystem `json:"ecosystem"`
	Package   string    `json:"package"`
	Ranges    []Range   `json:"ranges"`
	Severity  Severity  `json:"severity,omitempty"`
	CVSS      *CVSS     `json:"cvss,omitempty"`
	CWEs      []string  `json:"cwes,omitempty"`
	// Symbols lists the specific vulnerable symbols/functions when a feed
	// declares them (as OSV's ecosystem_specific imports do). It powers
	// symbol-level reachability: an advisory with symbols is only "reached" when
	// one of them is actually used. Empty means "unknown", treated as reachable.
	Symbols    []string `json:"symbols,omitempty"`
	References []string `json:"references,omitempty"`
	Source     string   `json:"source,omitempty"`    // feed name: nvd/ghsa/osv/distro
	Published  string   `json:"published,omitempty"` // kept verbatim; never sampled from the clock
	Modified   string   `json:"modified,omitempty"`
}

Advisory is the single normalized record every feed maps into. One Advisory describes one (ecosystem, package) pair; a feed entry that affects several packages normalizes into several Advisories sharing an ID/aliases.

func (Advisory) FixedVersion 

func (a Advisory) FixedVersion(sch VersionScheme) string

FixedVersion returns the smallest fixed version across the advisory's ranges, or "" if none is known (an unfixed vulnerability). It is the concrete upgrade target an operator — or an AI agent — should move to.

type CVSS 

type CVSS struct {
	Version string  `json:"version,omitempty"` // "2.0", "3.0", "3.1", "4.0"
	Vector  string  `json:"vector,omitempty"`
	Score   float64 `json:"score,omitempty"`
}

CVSS carries a single CVSS metric as reported by a feed.

func ParseCVSSVector 

func ParseCVSSVector(vector string) (CVSS, bool)

ParseCVSSVector parses a CVSS v3.0/3.1 vector string and returns the metric CVSS with its computed base score. ok is false for a vector that is not a recognizable v3 vector (v2/v4 vectors are returned verbatim with score 0 by the caller). Doing the arithmetic ourselves — rather than trusting a feed's self-reported number — means the score is reproducible and consistent across feeds that disagree.

type Coord 

type Coord struct {
	Ecosystem Ecosystem
	Package   string
	Version   string
	Scheme    VersionScheme
}

Coord is the normalized lookup key derived from an SBOM component: the feed ecosystem an advisory would live under, the package name as that feed spells it, the installed version, and the scheme to compare versions with.

func FromCataloger 

func FromCataloger(foundBy, name, version, distroID string) (Coord, bool)

FromCataloger builds a Coord from an SBOM component that lacks a usable PURL, using the cataloger (found_by) name to pick an ecosystem. This keeps coverage for components a cataloger recorded without a PURL rather than dropping them.

func ParsePURL 

func ParsePURL(purl string) (Coord, bool)

ParsePURL turns a Package URL into a Coord. It reconstructs ecosystem-correct package names (scoped npm "@scope/name", Go module paths, Maven "group:artifact") so the name matches how advisory feeds key that ecosystem — the difference between a hit and a silent miss. ok is false for a PURL we cannot classify.

type DB 

type DB struct {
	Schema     int                `json:"schema"`
	BuiltAt    time.Time          `json:"built_at"`
	Source     string             `json:"source,omitempty"`
	Advisories []Advisory         `json:"advisories"`
	EPSS       map[string]float64 `json:"epss,omitempty"` // CVE id → exploit probability [0,1]
	KEV        []string           `json:"kev,omitempty"`  // CVEs in CISA's Known Exploited catalog

	// SkippedRecords counts per-record parse failures encountered while
	// building this DB via Update (malformed OSV documents that normalizeOSV
	// rejected). It's a diagnostic counter for the rebuild that produced this
	// *DB value, not persisted state — deliberately excluded from JSON so it
	// never affects Marshal's byte-reproducible output, and zero (not present)
	// for any DB loaded via Open/LoadJSON rather than freshly built.
	SkippedRecords int `json:"-"`
	// contains filtered or unexported fields
}

DB is a loaded, indexed advisory database. The JSON fields are the on-disk format; the unexported index fields are rebuilt on load and never serialized, so a re-serialized DB is byte-stable.

func Default 

func Default() (*DB, error)

Default loads the embedded bootstrap advisory database.

func LoadJSON 

func LoadJSON(data []byte) (*DB, error)

LoadJSON parses and indexes a DB from its JSON bytes.

func Open 

func Open(path string) (*DB, error)

Open loads a DB from a JSON file on disk.

func Update 

func Update(ctx context.Context, opts Options) (*DB, error)

Update rebuilds an advisory DB from the configured sources, normalizing every feed into the internal schema and de-duplicating by alias. It is safe to run air-gapped (Fetcher nil) against a committed snapshot directory.

func UpdateNVD 

func UpdateNVD(ctx context.Context, fetcher Fetcher, source string, now time.Time, epss map[string]float64, kev []string) (*DB, error)

UpdateNVD builds a DB from an NVD Fetcher, normalizing every fetched `cve` document and merging duplicate CVEs across (ecosystem, package) the same way every other feed does (see mergeByAlias). A per-document normalization failure is counted in SkippedRecords rather than aborting the whole update.

func (*DB) Age 

func (db *DB) Age(now time.Time) time.Duration

Age returns how old the database is relative to now. Callers inject now so staleness never depends on the ambient clock inside analysis.

func (*DB) At 

func (db *DB) At(i int) *Advisory

At returns a pointer to the advisory at index i (as returned by IndicesFor), avoiding a struct copy. It panics on an out-of-range i, exactly like a slice index — callers are expected to only pass indices obtained from IndicesFor on the same db.

func (*DB) Count 

func (db *DB) Count() int

Count returns the number of advisories, for diagnostics.

func (*DB) EPSSFor 

func (db *DB) EPSSFor(ids []string) (float64, bool)

EPSSFor returns the EPSS exploit probability for the first of ids that has a score, and whether one was found. EPSS is keyed by CVE, which may be an advisory alias rather than its primary id.

func (*DB) IndicesFor 

func (db *DB) IndicesFor(c Coord) []int

IndicesFor returns the raw indices into Advisories that apply to a component coordinate, without copying any Advisory. The returned slice aliases the DB's internal index (db.byPkg) — callers must treat it as read-only and must not mutate it, and it is only valid for the lifetime of db (which is never mutated after LoadJSON/Update build it, so this is safe to hold as long as db itself is reachable).

func (*DB) IsKEV 

func (db *DB) IsKEV(ids []string) bool

IsKEV reports whether any of ids is in CISA's Known Exploited Vulnerabilities catalog — the single strongest "fix this now" signal, independent of CVSS.

func (*DB) Marshal 

func (db *DB) Marshal() ([]byte, error)

Marshal serializes the DB back to canonical JSON: advisories sorted by (ecosystem, package, id) and KEV sorted, so `db update` output is byte-reproducible for the same inputs.

func (*DB) Query 

func (db *DB) Query(c Coord) []Advisory

Query returns the advisories that apply to a component coordinate. It returns a fresh slice; callers may reorder it freely.

This copies every candidate Advisory by value, which is convenient for callers that want an owned, reorderable slice (tests, CLI diagnostics) but is wasted allocation/copy work in the vuln module's per-component hot loop, which only reads fields and never needs ownership. That loop should prefer IndicesFor + At instead (see match.go).

func (*DB) Stale 

func (db *DB) Stale(now time.Time, maxAge time.Duration) bool

Stale reports whether the DB is older than maxAge as of now. A zero now or BuiltAt disables the check (returns false) so tests stay deterministic.

type EPSSFetcher 

type EPSSFetcher struct {
	URL    string
	Client *http.Client
}

EPSSFetcher pulls the full EPSS scoreset over HTTP(S). Zero value is unusable; build one with NewEPSSFetcher.

func NewEPSSFetcher 

func NewEPSSFetcher(url string) *EPSSFetcher

NewEPSSFetcher returns a fetcher for url (DefaultEPSSURL when empty) with a bounded timeout.

func (*EPSSFetcher) Fetch 

func (f *EPSSFetcher) Fetch(ctx context.Context) (map[string]float64, error)

Fetch downloads and parses the scoreset into a CVE→probability map.

type Ecosystem 

type Ecosystem string

Ecosystem is the feed namespace an advisory applies to. For OS packages it is the distro id (alpine, debian, ubuntu, rhel, …); for language dependencies it is the package ecosystem (npm, pypi, go, maven, cargo, rubygems, composer, nuget). It selects *which* advisories apply to a component; the VersionScheme (see scheme) selects *how* versions within them are compared.

type Fetcher 

type Fetcher interface {
	Fetch(ctx context.Context) ([][]byte, error)
}

Fetcher retrieves raw OSV record documents from a remote source. Each returned element is one OSV JSON document.

type HTTPFetcher 

type HTTPFetcher struct {
	URL    string
	Client *http.Client
}

HTTPFetcher pulls advisory records from a self-hosted OSV mirror over HTTP(S). It expects the URL to return a JSON array of OSV records (the shape a mirror of osv.dev exports). It is only constructed when the operator opts into network access, keeping the default path fully air-gapped.

func NewHTTPFetcher 

func NewHTTPFetcher(url string) *HTTPFetcher

NewHTTPFetcher builds a fetcher with a bounded default timeout.

func (*HTTPFetcher) Fetch 

func (f *HTTPFetcher) Fetch(ctx context.Context) ([][]byte, error)

Fetch downloads the feed and splits it into individual OSV record documents.

type KEVFetcher 

type KEVFetcher struct {
	URL    string
	Client *http.Client
}

KEVFetcher pulls the CISA KEV catalog over HTTP(S).

func NewKEVFetcher 

func NewKEVFetcher(url string) *KEVFetcher

NewKEVFetcher returns a fetcher for url (DefaultKEVURL when empty).

func (*KEVFetcher) Fetch 

func (f *KEVFetcher) Fetch(ctx context.Context) ([]string, error)

Fetch downloads and parses the catalog into a sorted, de-duplicated list of KEV CVE IDs (uppercased).

type MultiFetcher 

type MultiFetcher struct{ Fetchers []Fetcher }

MultiFetcher concatenates the records of several fetchers (one per ecosystem feed). A failure of any feed fails the whole update — a silently partial DB is worse than an error.

func (*MultiFetcher) Fetch 

func (m *MultiFetcher) Fetch(ctx context.Context) ([][]byte, error)

type NVDFetcher 

type NVDFetcher struct {
	APIKey   string
	Client   *http.Client
	Since    time.Time // zero = unbounded (no lastModStartDate/EndDate filter)
	Until    time.Time
	MaxPages int // 0 = unlimited
	PageSize int // capped at 2000; 0 = default 2000
	Sleep    time.Duration
	BaseURL  string
}

NVDFetcher retrieves raw per-CVE documents from the live NVD REST API, paginating through resultsPerPage-sized pages and rate-limiting itself per NVD's published guidance (much slower without an API key).

func (NVDFetcher) Fetch 

func (f NVDFetcher) Fetch(ctx context.Context) ([][]byte, error)

Fetch implements Fetcher, paginating the NVD CVE API and returning one raw `cve` document per CVE across all pages.

type Options 

type Options struct {
	FromDir string             // directory of OSV JSON documents (+ optional epss.json/kev.json)
	Fetcher Fetcher            // optional network source; nil = air-gapped
	Now     time.Time          // build timestamp
	Source  string             // human label recorded in the DB
	EPSS    map[string]float64 // enrichment overlay (merged over any epss.json)
	KEV     []string           // enrichment overlay (merged over any kev.json)
}

Options configures a database rebuild. The rebuild is offline by default: it reads raw feed documents from FromDir. Network access is opt-in and provided by a Fetcher (net/http lives behind that interface so tests never touch the wire). Now is injected so the build timestamp is deterministic in tests.

type Range 

type Range struct {
	Scheme       VersionScheme `json:"scheme,omitempty"` // override; empty = ecosystem default
	Introduced   string        `json:"introduced,omitempty"`
	Fixed        string        `json:"fixed,omitempty"`
	LastAffected string        `json:"last_affected,omitempty"`
}

Range is one affected version interval for a package. Semantics:

  • Introduced=="" or "0" means "from the beginning of time".
  • Fixed is the first version that is NOT affected (exclusive upper bound). A non-empty Fixed also tells us the vulnerability is fixable and where.
  • LastAffected is an inclusive upper bound used when there is no fix (open range with a known last-vulnerable version).

An advisory may carry several ranges (multiple affected branches).

type Severity 

type Severity string

Severity is a normalized qualitative risk level. Feeds report severity in incompatible ways (NVD bands, GHSA labels, distro-specific words); they all normalize into this small ordered set. 

const (
	SevCritical   Severity = "critical"
	SevHigh       Severity = "high"
	SevMedium     Severity = "medium"
	SevLow        Severity = "low"
	SevNegligible Severity = "negligible"
	SevUnknown    Severity = "unknown"
)

func NormalizeSeverityWord 

func NormalizeSeverityWord(s string) Severity

NormalizeSeverityWord maps assorted feed severity spellings to the normalized band, for feeds that report a word rather than a vector.

func SeverityFromScore 

func SeverityFromScore(score float64) Severity

SeverityFromScore maps a CVSS base score to the normalized qualitative band (CVSS v3 ranges).

func (Severity) Rank 

func (s Severity) Rank() int

Rank returns an ascending numeric order for a severity, so callers can sort or gate without a switch. Higher is worse.

type VersionScheme 

type VersionScheme string

VersionScheme names a per-ecosystem version ordering. Matching dispatches on it so, e.g., a Debian epoch never gets compared with semver rules. 

const (
	SchemeSemver  VersionScheme = "semver"
	SchemeDeb     VersionScheme = "deb"
	SchemeRPM     VersionScheme = "rpm"
	SchemeAPK     VersionScheme = "apk"
	SchemePEP440  VersionScheme = "pep440"
	SchemeGo      VersionScheme = "go"
	SchemeMaven   VersionScheme = "maven"
	SchemeGem     VersionScheme = "gem"
	SchemeGeneric VersionScheme = "generic"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.