store

package
v0.0.0-...-9e498b3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 17, 2024 License: BSD-3-Clause, CC-BY-4.0 Imports: 17 Imported by: 0

Documentation

Overview

Package store supports permanent data storage for the vuln worker.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CVERecord 

type CVERecord struct {
	// ID is the CVE ID, which is the same as the filename base. E.g. "CVE-2020-0034".
	ID string
	// Path is the path to the CVE file in the repo.
	Path string
	// Blobhash is the hash of the CVE's blob in repo, for quick change detection.
	BlobHash string
	// CommitHash is the commit of the cvelist repo from which this information came.
	CommitHash string
	// CommitTime is the time of the above commit.
	// If zero, it has not been populated.
	CommitTime time.Time
	// CVEState is the value of the metadata.STATE field.
	CVEState string
	// TriageState is the state of our triage processing on the CVE.
	TriageState TriageState
	// TriageStateReason is an explanation of TriageState.
	TriageStateReason string

	// Module is the Go module path that might be affected.
	Module string

	// Package is the Go package path that might be affected.
	Package string

	// CVE is a copy of the CVE, for the NeedsIssue triage state.
	CVE *cveschema.CVE

	// ReferenceURLs is a list of the URLs in the CVE references,
	// for the FalsePositive triage state.
	ReferenceURLs []string

	// IssueReference is a reference to the GitHub issue that was filed.
	// E.g. golang/vulndb#12345.
	// Set only after a GitHub issue has been successfully created.
	IssueReference string

	// IssueCreatedAt is the time when the issue was created.
	// Set only after a GitHub issue has been successfully created.
	IssueCreatedAt time.Time

	// History holds previous states of a CVERecord,
	// from most to least recent.
	History []*CVERecordSnapshot
}

A CVERecord contains information about a CVE.

func NewCVERecord 

func NewCVERecord(cve *cveschema.CVE, path, blobHash string, commit *object.Commit) *CVERecord

NewCVERecord creates a CVERecord from a CVE, its path and its blob hash.

func (*CVERecord) GetID 

func (r *CVERecord) GetID() string

func (*CVERecord) GetIssueCreatedAt 

func (r *CVERecord) GetIssueCreatedAt() time.Time

func (*CVERecord) GetIssueReference 

func (r *CVERecord) GetIssueReference() string

func (*CVERecord) GetUnit 

func (r *CVERecord) GetUnit() string

func (*CVERecord) Snapshot 

func (r *CVERecord) Snapshot() *CVERecordSnapshot

func (*CVERecord) Validate 

func (r *CVERecord) Validate() error

Validate returns an error if the CVERecord is not valid.

type CVERecordSnapshot 

type CVERecordSnapshot struct {
	CommitHash        string
	CVEState          string
	TriageState       TriageState
	TriageStateReason string
}

CVERecordSnapshot holds a previous state of a CVERecord. The fields mean the same as those of CVERecord.

type CommitUpdateRecord 

type CommitUpdateRecord struct {
	// The ID of this record in the DB. Needed to modify the record.
	ID string
	// When the update started and completed. If EndedAt is zero,
	// the update is in progress (or it crashed).
	StartedAt, EndedAt time.Time
	// The repo commit hash that this update is working on.
	CommitHash string
	// The time the commit occurred.
	CommitTime time.Time
	// The total number of CVEs being processed in this update.
	NumTotal int
	// The number currently processed. When this equals NumTotal, the
	// update is done.
	NumProcessed int
	// The number of CVEs added to the DB.
	NumAdded int
	// The number of CVEs modified.
	NumModified int
	// The error that stopped the update.
	Error string
	// The last time this record was updated.
	UpdatedAt time.Time `firestore:",serverTimestamp"`
}

A CommitUpdateRecord describes a single update operation, which reconciles a commit in the CVE list repo with the DB state.

type FireStore 

type FireStore struct {
	// contains filtered or unexported fields
}

FireStore is a Store implemented with Google Cloud Firestore.

A Firestore DB is a set of documents. Each document has its own unique ID (primary key). Documents are grouped into collections, and each document can have sub-collections. A document can be referred to by a path of the form top-level-collection/doc/sub-collection/doc/...

In this layout, there is a single top-level collection called Namespaces, with documents for each development environment. Within each namespace, there are some collections: - CVEs for CVERecords - CommitUpdates for CommitUpdateRecords - DirHashes for directory hashes - GHSAs for GHSARecords. - ModuleScans for ModuleScanRecords.

func NewFireStore 

func NewFireStore(ctx context.Context, projectID, namespace, impersonate string) (_ *FireStore, err error)

NewFireStore creates a new FireStore, backed by a client to Firestore. Since each project can have only one Firestore database, callers must provide a non-empty namespace to distinguish different virtual databases (e.g. prod and testing). If non-empty, the impersonate argument should be the name of a service account to impersonate.

func (*FireStore) Clear 

func (s *FireStore) Clear(ctx context.Context) (err error)

Clear removes all documents in the namespace.

func (*FireStore) CreateCommitUpdateRecord 

func (fs *FireStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)

CreateCommitUpdateRecord implements Store.CreateCommitUpdateRecord. On successful return, r.ID is set to the record's ID.

func (*FireStore) CreateModuleScanRecord 

func (fs *FireStore) CreateModuleScanRecord(ctx context.Context, r *ModuleScanRecord) (err error)

CreateModuleScanRecord implements Store.CreateModuleScanRecord.

func (*FireStore) GetCVERecord 

func (fs *FireStore) GetCVERecord(ctx context.Context, id string) (_ *CVERecord, err error)

GetCVERecord implements store.GetCVERecord.

func (*FireStore) GetDirectoryHash 

func (fs *FireStore) GetDirectoryHash(ctx context.Context, dir string) (_ string, err error)

GetDirectoryHash implements Transaction.GetDirectoryHash.

func (*FireStore) GetModuleScanRecord 

func (fs *FireStore) GetModuleScanRecord(ctx context.Context, path, version string, dbTime time.Time) (_ *ModuleScanRecord, err error)

GetModuleScanRecord implements store.GetModuleScanRecord.

func (*FireStore) ListCVERecordsWithTriageState 

func (fs *FireStore) ListCVERecordsWithTriageState(ctx context.Context, ts TriageState) (_ []*CVERecord, err error)

ListCVERecordsWithTriageState implements Store.ListCVERecordsWithTriageState.

func (*FireStore) ListCommitUpdateRecords 

func (fs *FireStore) ListCommitUpdateRecords(ctx context.Context, limit int) (_ []*CommitUpdateRecord, err error)

ListCommitUpdateRecords implements Store.ListCommitUpdateRecords.

func (*FireStore) ListModuleScanRecords 

func (fs *FireStore) ListModuleScanRecords(ctx context.Context, limit int) (_ []*ModuleScanRecord, err error)

ListModuleScanRecords implements Store.ListModuleScanRecords.

func (*FireStore) RunTransaction 

func (fs *FireStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) (err error)

RunTransaction implements Store.RunTransaction.

func (*FireStore) SetCommitUpdateRecord 

func (fs *FireStore) SetCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) (err error)

SetCommitUpdateRecord implements Store.SetCommitUpdateRecord.

func (*FireStore) SetDirectoryHash 

func (fs *FireStore) SetDirectoryHash(ctx context.Context, dir, hash string) (err error)

SetDirectoryHash implements Transaction.SetDirectoryHash.

type GHSARecord 

type GHSARecord struct {
	// GHSA is the advisory.
	GHSA *ghsa.SecurityAdvisory
	// TriageState is the state of our triage processing on the CVE.
	TriageState TriageState
	// TriageStateReason is an explanation of TriageState.
	TriageStateReason string
	// IssueReference is a reference to the GitHub issue that was filed.
	// E.g. golang/vulndb#12345.
	// Set only after a GitHub issue has been successfully created.
	IssueReference string
	// IssueCreatedAt is the time when the issue was created.
	// Set only after a GitHub issue has been successfully created.
	IssueCreatedAt time.Time
}

A GHSARecord holds information about a GitHub security advisory.

func (*GHSARecord) GetID 

func (r *GHSARecord) GetID() string

func (*GHSARecord) GetIssueCreatedAt 

func (r *GHSARecord) GetIssueCreatedAt() time.Time

func (*GHSARecord) GetIssueReference 

func (r *GHSARecord) GetIssueReference() string

func (*GHSARecord) GetUnit 

func (r *GHSARecord) GetUnit() string

type MemStore 

type MemStore struct {
	// contains filtered or unexported fields
}

MemStore is an in-memory implementation of Store, for testing.

func NewMemStore 

func NewMemStore() *MemStore

NewMemStore creates a new, empty MemStore.

func (*MemStore) CVERecords 

func (ms *MemStore) CVERecords() map[string]*CVERecord

CVERecords return all the CVERecords of the store.

func (*MemStore) Clear 

func (ms *MemStore) Clear(context.Context) error

Clear removes all data from the MemStore.

func (*MemStore) CreateCommitUpdateRecord 

func (ms *MemStore) CreateCommitUpdateRecord(ctx context.Context, r *CommitUpdateRecord) error

CreateCommitUpdateRecord implements Store.CreateCommitUpdateRecord.

func (*MemStore) CreateModuleScanRecord 

func (ms *MemStore) CreateModuleScanRecord(_ context.Context, r *ModuleScanRecord) error

CreateModuleScanRecord implements Store.CreateModuleScanRecord.

func (*MemStore) GetCVERecord 

func (ms *MemStore) GetCVERecord(ctx context.Context, id string) (*CVERecord, error)

GetCVERecord implements store.GetCVERecord.

func (*MemStore) GetDirectoryHash 

func (ms *MemStore) GetDirectoryHash(_ context.Context, dir string) (string, error)

GetDirectoryHash implements Transaction.GetDirectoryHash.

func (*MemStore) GetModuleScanRecord 

func (ms *MemStore) GetModuleScanRecord(_ context.Context, path, version string, dbTime time.Time) (*ModuleScanRecord, error)

GetModuleScanRecord implements store.GetModuleScanRecord.

func (*MemStore) ListCVERecordsWithTriageState 

func (ms *MemStore) ListCVERecordsWithTriageState(_ context.Context, ts TriageState) ([]*CVERecord, error)

ListCVERecordsWithTriageState implements Store.ListCVERecordsWithTriageState.

func (*MemStore) ListCommitUpdateRecords 

func (ms *MemStore) ListCommitUpdateRecords(_ context.Context, limit int) ([]*CommitUpdateRecord, error)

ListCommitUpdateRecords implements Store.ListCommitUpdateRecords.

func (*MemStore) ListModuleScanRecords 

func (ms *MemStore) ListModuleScanRecords(ctx context.Context, limit int) ([]*ModuleScanRecord, error)

ListModuleScanRecords implements Store.ListModuleScanRecords.

func (*MemStore) RunTransaction 

func (ms *MemStore) RunTransaction(ctx context.Context, f func(context.Context, Transaction) error) error

RunTransaction implements Store.RunTransaction. A transaction runs with a single lock on the entire DB.

func (*MemStore) SetCommitUpdateRecord 

func (ms *MemStore) SetCommitUpdateRecord(_ context.Context, r *CommitUpdateRecord) error

SetCommitUpdateRecord implements Store.SetCommitUpdateRecord.

func (*MemStore) SetDirectoryHash 

func (ms *MemStore) SetDirectoryHash(_ context.Context, dir, hash string) error

SetDirectoryHash implements Transaction.SetDirectoryHash.

type ModuleScanRecord 

type ModuleScanRecord struct {
	Path       string
	Version    string
	DBTime     time.Time // last-modified time of the vuln DB
	Error      string    // if non-empty, error while scanning
	VulnIDs    []string
	FinishedAt time.Time // when the scan completed (successfully or not)
}

A ModuleScanRecord holds information about a vulnerability scan of a module.

func (*ModuleScanRecord) Validate 

func (r *ModuleScanRecord) Validate() error

Validate returns an error if the ModuleScanRecord is not valid.

type Store 

type Store interface {
	// CreateCommitUpdateRecord creates a new CommitUpdateRecord. It should be called at the start
	// of an update. On successful return, the CommitUpdateRecord's ID field will be
	// set to a new, unique ID.
	CreateCommitUpdateRecord(context.Context, *CommitUpdateRecord) error

	// SetCommitUpdateRecord modifies the CommitUpdateRecord. Use the same record passed to
	// CreateCommitUpdateRecord, because it will have the correct ID.
	SetCommitUpdateRecord(context.Context, *CommitUpdateRecord) error

	// ListCommitUpdateRecords returns some of the CommitUpdateRecords in the store, from most to
	// least recent.
	ListCommitUpdateRecords(ctx context.Context, limit int) ([]*CommitUpdateRecord, error)

	// GetCVERecord returns the CVERecord with the given id. If not found, it returns (nil, nil).
	GetCVERecord(ctx context.Context, id string) (*CVERecord, error)

	// ListCVERecordsWithTriageState returns all CVERecords with the given triage state,
	// ordered by ID.
	ListCVERecordsWithTriageState(ctx context.Context, ts TriageState) ([]*CVERecord, error)

	// GetDirectoryHash returns the hash for the tree object corresponding to dir.
	// If dir isn't found, it succeeds with the empty string.
	GetDirectoryHash(ctx context.Context, dir string) (string, error)

	// SetDirectoryHash sets the hash for the given directory.
	SetDirectoryHash(ctx context.Context, dir, hash string) error

	// CreateModuleScanRecord adds a ModuleScanRecord to the DB.
	CreateModuleScanRecord(context.Context, *ModuleScanRecord) error

	// GetModuleScanRecord returns the most recent ModuleScanRecord matching the
	// given module path, version and DB time. If not found, it returns (nil,
	// nil).
	GetModuleScanRecord(ctx context.Context, path, version string, dbTime time.Time) (*ModuleScanRecord, error)

	// ListModuleScanRecords returns some of the ModuleScanRecords in the store
	// from most to least recent. If limit is zero, all records are returned.
	ListModuleScanRecords(ctx context.Context, limit int) ([]*ModuleScanRecord, error)

	// RunTransaction runs the function in a transaction.
	RunTransaction(context.Context, func(context.Context, Transaction) error) error
}

A Store is a storage system for the CVE database.

type Transaction 

type Transaction interface {
	// CreateCVERecord creates a new CVERecord. It is an error if one with the same ID
	// already exists.
	CreateCVERecord(*CVERecord) error

	// SetCVERecord sets the CVE record in the database. It is
	// an error if no such record exists.
	SetCVERecord(r *CVERecord) error

	// GetCVERecords retrieves CVERecords for all CVE IDs between startID and
	// endID, inclusive.
	GetCVERecords(startID, endID string) ([]*CVERecord, error)

	// CreateGHSARecord creates a new GHSARecord. It is an error if one with the same ID
	// already exists.
	CreateGHSARecord(*GHSARecord) error

	// SetGHSARecord sets the GHSA record in the database. It is
	// an error if no such record exists.
	SetGHSARecord(*GHSARecord) error

	// GetGHSARecord returns a single GHSARecord by GHSA ID.
	// If not found, it returns (nil, nil).
	GetGHSARecord(id string) (*GHSARecord, error)

	// GetGHSARecords returns all the GHSARecords in the database.
	GetGHSARecords() ([]*GHSARecord, error)
}

Transaction supports store operations that run inside a transaction.

type TriageState 

type TriageState string

TriageState is the state of our work on the CVE or GHSA. It is implemented as a string rather than an int so that stored values are immune to renumbering. 

const (
	// No action is needed on the CVE or GHSA (perhaps because it is rejected, reserved or invalid).
	TriageStateNoActionNeeded TriageState = "NoActionNeeded"
	// The CVE needs to have an issue created.
	TriageStateNeedsIssue TriageState = "NeedsIssue"
	// An issue has been created in the issue tracker.
	// The IssueReference and IssueCreatedAt fields have more information.
	TriageStateIssueCreated TriageState = "IssueCreated"
	// This vulnerability has already been handled under an alias (i.e., a CVE
	// or GHSA that refers to the same vulnerability).
	TriageStateAlias TriageState = "Alias"
	// The CVE state was changed after the CVE was created.
	TriageStateUpdatedSinceIssueCreation TriageState = "UpdatedSinceIssueCreation"
	// Although the triager might think this CVE is relevant to Go, it is not.
	TriageStateFalsePositive TriageState = "FalsePositive"
	// There is already an entry in the Go vuln DB that covers this CVE.
	TriageStateHasVuln TriageState = "HasVuln"
)

func (TriageState) Validate 

func (s TriageState) Validate() error

Validate returns an error if the TriageState is not one of the above values.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.