cli

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 6, 2026 License: MIT Imports: 46 Imported by: 0

Documentation

Overview

Package cli is the implementation behind cmd/audit. It is internal because the command is the contract, not the Go API of its subcommands.

Index

Constants

View Source 
const (
	Reads  = false
	Writes = true
)

Reads and Writes say whether a command puts objects. A command that only reads takes no --lock-mode: the lock is a property of what is written.

View Source 
const DeadLetterPrefix = "dlq/"

DeadLetterPrefix is where the writer keeps what it could not process.

View Source 
const LayerEnv = "AUDIT_CONFIG_LAYER"

LayerEnv is the variable a deployment on Lambda sets to the ARN of the layer version that carries the configuration. Nothing in the environment tells a function which layers it has, so the deployment that attached the layer says so, and the writer repeats it in its start-up record.

View Source 
const NotAnAction = "audit:not-an-action"

NotAnAction marks a line whose literal under the source's namespace is something other than an action name.

View Source 
const TokenFileEnv = "AUDIT_TOKEN_FILE"

TokenFileEnv names the file holding the token a process presents to the writer and the registry: in a cluster, its projected service-account token.

Variables

This section is empty.

Functions

func CatalogueFields 

func CatalogueFields(documents []string, from store.Store) (observe.Fields, error)

CatalogueFields loads catalogue documents and returns the Fields a reindex needs, falling back to the archive's own copy of a catalogue the files do not include. See observe.FieldsFrom for how a record is resolved.

func FindCatalogues 

func FindCatalogues(root string) ([]string, error)

FindCatalogues returns every catalogue document under a directory.

func LoadDeployment 

func LoadDeployment(path string) (*preset.Deployment, error)

LoadDeployment reads a deployment's profile configuration from a file.

func LoadGrants 

func LoadGrants(path string) (auth.Declarative, error)

LoadGrants reads the mapping. An empty path is not a default-allow: it is a deployment that has granted nobody anything, and every request is denied.

func OpenArchiveFrom added in v0.6.0 

func OpenArchiveFrom(ctx context.Context, a config.Archive, secrets *config.Secrets) (*s3store.Store, error)

OpenArchiveFrom opens the archive the configuration names.

func OpenExportsFrom added in v0.6.0 

func OpenExportsFrom(ctx context.Context, b config.Bucket, secrets *config.Secrets) (*s3store.Store, error)

OpenExportsFrom opens the exports bucket, which is a store of its own and has no lock: an export is a copy made to be taken away and then cleared.

func OpenKeysFrom added in v0.6.0 

func OpenKeysFrom(ctx context.Context, k *config.Keys, secrets *config.Secrets) (keys.Provider, error)

OpenKeysFrom opens the key provider the configuration names. It is nil where a deployment runs without one, which is the default.

func OpenSignerFrom added in v0.7.0 

func OpenSignerFrom(ctx context.Context, s config.Signer, secrets *config.Secrets) (keys.Signer, error)

OpenSignerFrom opens the key the configuration says seals are signed with. The private half of a managed key (kms, transit) never reaches this process; the file is the exception, and the one to leave to development.

func OpenStore added in v0.3.0 

func OpenStore(ctx context.Context, region string, o s3store.Options) (*s3store.Store, error)

OpenStore opens a store on the ambient AWS configuration, which in a cluster is the workload's own identity, or the static credentials the environment carries.

func ParseDay 

func ParseDay(v string) (time.Time, error)

ParseDay reads a date or a timestamp, so that an auditor may write either.

func Reset added in v0.7.0 

func Reset(ctx context.Context, cursors observe.Cursors, profile, tenant string) error

Reset forgets where the indexer had read a profile to (or one tenant of it), so that it reads that prefix again from the start and catches up. The index keeps what it has: a row that is there is left alone.

func SinkFrom added in v0.6.0 

func SinkFrom(s config.Sink, require string) (sink.Sink, error)

SinkFrom is a client to the writer the configuration names, presenting the token in the file it names. Every job that records through the writer builds its client here or in WriterClient, so that none of them is the one that forgot.

`expect` says what the writer is configured to give, and `require` is the floor the process holds it to: with a require, the client is wrapped in sink.Guard, which refuses at start-up when the expectation is below it and fails any acknowledgement weaker than it afterwards. Without one the client is returned as it is.

func WriterClient 

func WriterClient(url string) *sink.Client

WriterClient is a client to the writer at url, presenting the token named by AUDIT_TOKEN_FILE when it is set. Every command that records through the writer builds its client here, so that none of them is the one that forgot.

func WriterEvidence added in v0.11.0 

func WriterEvidence(src config.Source, deployment, workloads, catalogues string) (writer.Evidence, error)

WriterEvidence is what the writer says about its configuration when it starts: the digest of the file it read, of the documents and the directory of catalogues that file names, and on Lambda the layer. A path that is empty is a document the configuration does not have, and is left out. A document that cannot be read is an error, because the configuration was about to fail to load it anyway and a record that omitted it would be a record that lied.

Types

type Access 

type Access struct {
	Issuers []authn.Issuer
	Rules   auth.Declarative
}

Access is a grants file read and checked: who may authenticate, and what each of them may see.

func LoadAccess 

func LoadAccess(path string, profiles map[string]*preset.Profile) (Access, error)

LoadAccess reads a grants file with its issuers, and refuses a rule set that would let one issuer satisfy a rule meant for another. profiles is the deployment's, by name with the presets each is built from; a preset turns roles into profiles through it, and a file naming a preset is refused without it.

type ArchiveFlags added in v0.3.0 

type ArchiveFlags struct {
	Bucket, Prefix, Region, Endpoint *string
	PathStyle                        *bool
	// contains filtered or unexported fields
}

ArchiveFlags are the flags every command that opens the archive shares: where it is, which store it is on, and -- for a command that writes to it -- which lock it writes with.

They are one set so that the writer, the jobs and the query service take the same names and read the same environment (AUDIT_BUCKET, AUDIT_PREFIX, AWS_REGION, AUDIT_S3_ENDPOINT, AUDIT_S3_PATH_STYLE, AUDIT_LOCK_MODE), which is what lets the chart set them once per pod. Static credentials need no flag: the SDK reads AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY itself, and AWS_ENDPOINT_URL_S3 as well, so an endpoint may also arrive that way.

func NewArchiveFlags added in v0.3.0 

func NewArchiveFlags(fs *flag.FlagSet, lookup func(name, fallback string) string, writes bool) *ArchiveFlags

NewArchiveFlags registers the archive flags on a flag set. Lookup gives each flag's default from the environment; nil means no environment.

func (*ArchiveFlags) Lock added in v0.3.0 

func (a *ArchiveFlags) Lock() (s3store.LockMode, error)

Lock is the mode the command writes in. A command registered with Reads has none to give and answers compliance, which nothing it does depends on.

func (*ArchiveFlags) Open added in v0.3.0 

func (a *ArchiveFlags) Open(ctx context.Context) (*s3store.Store, error)

Open opens the archive the flags name, on the ambient AWS configuration.

func (*ArchiveFlags) Options added in v0.3.0 

func (a *ArchiveFlags) Options() (s3store.Options, error)

Options are the store options the flags describe.

func (*ArchiveFlags) SetLock added in v0.3.0 

func (a *ArchiveFlags) SetLock(mode s3store.LockMode) error

SetLock is what a deprecated alias of --lock-mode does: it sets the mode unless --lock-mode was given explicitly and says otherwise, which is two answers to one question and refused.

type CheckEmitters 

type CheckEmitters struct {
	Root      string
	Catalogue string
	Out       io.Writer
}

CheckEmitters finds the action names a source's code emits and holds them to the catalogue that describes them.

It reads string literals rather than resolving calls, so it catches the mistakes that actually happen — a typo, an action added in code and not in the catalogue, an action retired from the catalogue and still emitted — and says plainly that it cannot see a name assembled at run time. A catalogue that is only as true as the code remembers to keep it is not a contract.

Reading literals means it sometimes finds one that is not an action at all: a source's namespace is a good prefix for other things too, and a Postgres setting called audit.tenant_id looks exactly like an action called audit.tenant_id. A line carrying NotAnAction is left alone, which keeps the escape hatch narrow, visible at the declaration and greppable — rather than letting the checker guess, which is how a real missing emitter gets excused.

func (CheckEmitters) Run 

func (c CheckEmitters) Run() int

Run reports the number of problems found.

type ClockReport 

type ClockReport struct {
	Source   string `json:"source"`
	OffsetMS int64  `json:"offset_ms"`
	DelayMS  int64  `json:"delay_ms"`
	Stratum  uint8  `json:"stratum"`
	// Within is false when the offset exceeds what the deployment allows.
	Within bool `json:"within_tolerance"`
	// Recorded says whether the event reached a writer. A check nobody can see
	// the result of has not met the requirement, however good the offset was.
	Recorded bool `json:"recorded"`
	// Unreachable names the references that did not answer. One is survivable
	// and worth seeing; all of them is the error above.
	Unreachable []string `json:"unreachable,omitempty"`
}

ClockReport is what the check found.

func (ClockReport) String 

func (r ClockReport) String() string

String is the human form.

type ClockSync 

type ClockSync struct {
	// Sink is the writer. Without one the check runs and reports and records
	// nothing, which is a way to test the references and not a way to satisfy
	// the requirement.
	Sink sink.Sink
	// Catalogue describes audit.clock.synchronised. It is the common one.
	Catalogue *catalogue.Catalogue
	// Servers are the time references, tried in order of who answers.
	Servers []string
	// MaxOffset is how far out the clock may be before the run is a failure the
	// deployment hears about. Zero means any offset is accepted and only
	// recorded.
	MaxOffset time.Duration
	Timeout   time.Duration
	Version   string
	Instance  string
	JSON      bool
	Out       io.Writer
}

ClockSync compares this machine's clock with UTC and records the answer.

ETSI EN 319 401 §7.10 asks that the clock be synchronised and that the synchronisation be recorded. The recording is the half that belongs here: every timestamp in the archive is this machine's opinion of the time, and an auditor who cannot see that the opinion was checked has to take the ordering of the whole trail on faith.

It does not set the clock. Whatever runs the machine does that, and a component that both set the time and recorded the times of things would be marking its own paper.

func (ClockSync) Run 

func (c ClockSync) Run(ctx context.Context) (ClockReport, error)

Run checks the clock, records the reading, and reports it.

An offset outside the tolerance is an error, so that a scheduled run fails where a deployment can see it. The reading is recorded first either way: an hour whose timestamps are suspect is exactly the hour an auditor will want the measurement from.

type Conformance 

type Conformance struct {
	Client   auditv1connect.QueryServiceClient
	Profiles []string
	// Max bounds how many records a profile's walk reads. Default 1000.
	Max int
	// PageSize is the limit each page asks for. Default 100.
	PageSize int32
	// VerifiedBefore, when set, requires every sampled record that occurred
	// longer ago than this to be covered by a verified digest: the digest and
	// verify jobs have run and found it intact.
	VerifiedBefore time.Duration
	Now            func() time.Time
	JSON           bool
	Out            io.Writer
}

Conformance holds a running query service to what the search contract promises, using nothing but the API a caller has.

It reads and never writes. A conformance run that wrote test records into a deployment would leave them in an Object-Locked archive for as long as the profile keeps anything — years, for evidence — so it checks the promises over whatever the deployment already holds. The reads it makes are recorded by the service like any other; that is the service doing its job.

What it checks, per profile: paging walks every record once, newest first; the last page still has a `next`; every record is the profile's copy; Get returns what Search returned, with its provenance; a filter returns only what matches it and the record it was built from; a cursor is refused with another query; an unknown id is not found. With VerifiedBefore, a record older than that must carry a verified_at, which seals set.

func (Conformance) Run 

func (c Conformance) Run(ctx context.Context) (int, error)

Run reports the number of checks that failed. An error is a run that could not be made at all — the service unreachable, the caller refused.

type ConformanceFinding 

type ConformanceFinding struct {
	Profile string `json:"profile"`
	Check   string `json:"check"`
	OK      bool   `json:"ok"`
	Detail  string `json:"detail,omitempty"`
}

ConformanceFinding is one check's result.

type ExportFlags added in v0.3.0 

type ExportFlags struct {
	Bucket, Endpoint *string
	PathStyle        *bool
	// contains filtered or unexported fields
}

ExportFlags are the flags for the bucket exports are written to: its own bucket, on a store of its own if need be, with no lock.

func NewExportFlags added in v0.3.0 

func NewExportFlags(fs *flag.FlagSet, lookup func(name, fallback string) string) *ExportFlags

NewExportFlags registers the export flags. The bucket is --exports, as it always was; the store it is on takes --exports-endpoint and --exports-path-style (AUDIT_EXPORTS_ENDPOINT, AUDIT_EXPORTS_PATH_STYLE), and credentials of its own, when the archive's do not reach it, from AUDIT_EXPORTS_ACCESS_KEY_ID, AUDIT_EXPORTS_SECRET_ACCESS_KEY and, if there is one, AUDIT_EXPORTS_SESSION_TOKEN.

func (*ExportFlags) Open added in v0.3.0 

func (e *ExportFlags) Open(ctx context.Context, region string, archive *ArchiveFlags) (*s3store.Store, error)

Open opens the exports bucket, without a lock.

It is a different store from the archive on purpose: an export is a copy of records made to be taken away and then cleared, and a lock would keep it. A bucket without Object Lock refuses a put that names a lock mode, so this is also the only way to write to one. The archive's flags give the endpoint and path style the exports inherit when they name none of their own.

type GrantRule 

type GrantRule struct {
	Name   string `json:"name"`
	Issuer string `json:"issuer,omitempty"`
	Claim  string `json:"claim,omitempty"`
	Value  string `json:"value,omitempty"`
	Grant  struct {
		AllTenants bool     `json:"all_tenants,omitempty"`
		Tenants    []string `json:"tenants,omitempty"`
		Profiles   []string `json:"profiles"`
		Operations []string `json:"operations"`
		// From and Until bound what the rule's holders may read, by when
		// records happened: an external assessor is given the period under
		// assessment, not the archive. RFC 3339; empty is unbounded.
		From  string `json:"from,omitempty"`
		Until string `json:"until,omitempty"`
	} `json:"grant"`
}

GrantRule maps a claim value to a grant.

type GrantsFile 

type GrantsFile struct {
	// APIVersion is `truvity.github.io/<kind>/v1`, or absent, which means the same.
	APIVersion string `json:"apiVersion,omitempty"`
	// Issuers are the token issuers trusted to say who a caller is. They live
	// in this file, beside the rules, because a rule is only as safe as the
	// issuers able to satisfy it; see auth.Rule.Issuer.
	Issuers []IssuerEntry `json:"issuers,omitempty"`
	// Presets read grants out of a claim's vocabulary rather than one value
	// each. Only access-roster's exists.
	Presets []PresetEntry `json:"presets,omitempty"`
	Rules   []GrantRule   `json:"rules,omitempty"`
}

GrantsFile is what a deployment writes to say who may read what.

It is a file rather than a call to a policy engine because a mapping an auditor can read is worth more here than one a service can compute, and because the engine is a dependency a deployment should choose rather than inherit.

type Hold 

type Hold struct {
	Store     store.Store
	Sink      sink.Sink
	Catalogue *catalogue.Catalogue
	// RetainUntil is how long a hold's own record is locked for.
	RetainUntil func(time.Time) time.Time

	Profile, Tenant, Reason, ID, By string
	Now                             func() time.Time
	JSON                            bool
	Out                             io.Writer
}

Hold places, releases and lists legal holds.

Placing one is an operator action and releasing one needs the deployment's break-glass role; neither is decided here. What this does is make the call and record that it was made, so that a hold and its release are as accounted for as the records they cover.

func (Hold) Run 

func (h Hold) Run(ctx context.Context, action string) error

Run dispatches place, release or list.

type IssuerEntry 

type IssuerEntry struct {
	URL      string `json:"url"`
	Audience string `json:"audience"`
}

IssuerEntry is one trusted issuer.

type KeyDestroy 

type KeyDestroy struct {
	Provider keys.Provider
	// Store is the archive, read to check for legal holds. It is required:
	// destroying a key whose copies are under hold destroys evidence that may
	// not be destroyed, and the operator should not be the one to remember.
	Store store.Store
	// Sink is where the record of the erasure goes. The catalogue declares this
	// action as block delivery, so it is required and the record is confirmed.
	Sink      sink.Sink
	Catalogue *catalogue.Catalogue

	Tenant  string
	Purpose string
	By      string
	Reason  string
	Out     io.Writer
}

KeyDestroy erases a tenant's pseudonymisation key for one purpose.

This is what erasure means here. The copies stay — they are under a lock and an auditor still needs to see that something happened — but the pseudonyms in them can never be recomputed, so nothing in the archive can be tied back to the person. It cannot be undone and there is nothing to restore from.

func (KeyDestroy) Run 

func (k KeyDestroy) Run(ctx context.Context) error

Run checks the holds, destroys the key, and records that it did.

type KeyFlags 

type KeyFlags struct {
	Provider *string
	// local
	Root *string
	Dir  *string
	// transit
	OpenBAO *OpenBAOFlags
	Prefix  *string
}

KeyFlags are the flags every binary names its key provider with, so that the writer, the query service's resolve and `audit key destroy` cannot be pointed at the same keys in three different spellings.

func NewKeyFlags 

func NewKeyFlags(fs *flag.FlagSet, lookup func(name, fallback string) string) *KeyFlags

NewKeyFlags registers the key flags on a flag set. Lookup gives each flag's default from the environment; nil means no environment.

func (*KeyFlags) Configured 

func (k *KeyFlags) Configured() bool

Configured reports whether the flags name a provider at all. The query service offers resolve only when they do.

func (*KeyFlags) Local 

func (k *KeyFlags) Local() bool

Local reports whether the provider is the local one, whose directory is the only copy of the keys and so carries rules the transit provider does not.

func (*KeyFlags) Open 

func (k *KeyFlags) Open(ctx context.Context) (keys.Provider, error)

Open builds the provider, which is nil where a deployment runs without one. A nil provider is the default: staff are kept in clear because that is what accountability is for, and people outside arrive as identifiers the application already minted. A profile that asks for a pseudonym anyway is refused by the writer, naming the profile — see docs/decisions/0013-no-pseudonymisation-keys-by-default.md.

type Marker 

type Marker interface {
	Purge(ctx context.Context, before time.Time) error
}

Marker is the part of the deduplication store a purge needs.

type Notary added in v0.7.0 

type Notary struct {
	Store  store.Store
	Signer keys.Signer
	// Profiles to seal; empty is every profile the archive has records for.
	Profiles []string
	// Settle is how long after an hour has ended it is sealed, so that a batch
	// put late in the hour it is keyed by is in the seal.
	Settle time.Duration
	// Now is the clock; nil is the wall clock.
	Now func() time.Time
	// Metrics, when given, receives the age of the newest seal per profile.
	Metrics *telemetry.Notary
	// Sink and Catalogue, when both are given, are where each seal is recorded.
	Sink      sink.Sink
	Catalogue *catalogue.Catalogue
	Version   string
	Instance  string
	JSON      bool
	Out       io.Writer
}

Notary seals the archive: for every profile, every tenant and every closed hour since the last seal, one signed statement of what the hour holds, chained to the one before (docs/reference/bucket-contract.md, docs/decisions/0019).

It is the one part of the system that signs, and it holds nothing the writer does: the key is a managed one the writer's role cannot use, and the bucket rights are to read records and put seals. Whoever writes the archive and can also sign for it can choose what to sign, which is the whole of why this is a separate job under a separate identity.

A run is idempotent. An hour already sealed is not sealed again, and a run over a bucket with nothing new does nothing but read.

func (Notary) Run added in v0.7.0 

func (n Notary) Run(ctx context.Context) (*NotaryReport, error)

Run seals what can be sealed and reports it. A tenant that cannot be sealed is a failure in the report and an error from Run, and the other tenants are sealed regardless.

type NotaryFailure added in v0.7.0 

type NotaryFailure struct {
	Profile string `json:"profile"`
	Tenant  string `json:"tenant"`
	Reason  string `json:"reason"`
}

NotaryFailure is a tenant the run could not seal further, and why. The chain stops there: a seal over a later hour would be chained to a seal that does not exist.

type NotaryReport added in v0.7.0 

type NotaryReport struct {
	// Key is the thumbprint of the key the seals were signed with.
	Key string `json:"key"`
	// Sealed are the seals written; Present counts the hours found sealed
	// already by another run, which is what makes a rerun a no-op.
	Sealed   []SealedHour    `json:"sealed"`
	Present  int             `json:"present"`
	Failures []NotaryFailure `json:"failures,omitempty"`
	// Ages is, per profile, the seconds since the end of the newest sealed hour
	// of the tenant furthest behind.
	Ages map[string]float64 `json:"ages,omitempty"`
}

NotaryReport is what a run did.

func (*NotaryReport) String added in v0.7.0 

func (r *NotaryReport) String() string

type OpenBAOFlags 

type OpenBAOFlags struct {
	Address   *string
	Mount     *string
	Namespace *string
	CAFile    *string
	TokenFile *string
	AuthMount *string
	AuthRole  *string
	JWTFile   *string
}

OpenBAOFlags are how a command reaches an OpenBAO transit engine: where it is, which namespace, which chain its certificate comes from, and how to sign in. The key provider and the digest signer share them, so that the writer's keys and the digest's signature cannot be reached two different ways.

func NewOpenBAOFlags 

func NewOpenBAOFlags(fs *flag.FlagSet, lookup func(name, fallback string) string) *OpenBAOFlags

NewOpenBAOFlags registers the connection flags. Lookup gives each default from the environment; nil means none. An operator's shell names the server the way the OpenBAO CLI does, so its variables are the fallback.

func (*OpenBAOFlags) Credentials 

func (o *OpenBAOFlags) Credentials() (login *keys.JWTLogin, token, tokenFile string)

Credentials are the one way to authenticate these flags name: a JWT login when an auth mount is given, else a token file, else BAO_TOKEN from the environment.

type PresetEntry 

type PresetEntry struct {
	Name   string `json:"name"`
	Issuer string `json:"issuer,omitempty"`
	Claim  string `json:"claim,omitempty"`
}

PresetEntry names a grant preset and the issuer whose claims it reads.

type Purge 

type Purge struct {
	Index    index.Indexer
	Dedupe   Marker
	Profiles map[string]*preset.Profile
	// IdentifyingAfter is how long the index keeps who an event happened to,
	// as opposed to what happened. It has no default and no framework number
	// behind it: the presets this repository ships cite retention for the
	// record, and none of them states a separate, shorter life for the actor
	// and subject columns. Left at zero, nothing is forgotten early — which is
	// not the same as nothing needing to be.
	IdentifyingAfter time.Duration
	// DedupeWindow is how long a written identifier is remembered. It wants to
	// be the widest window the deployment's profiles ask for; shorter and a
	// redelivery arrives after the memory of it has gone.
	DedupeWindow time.Duration
	// DryRun reports what would be purged and purges nothing.
	DryRun bool
	Now    func() time.Time
	JSON   bool
	Out    io.Writer
}

Purge brings the index and the deduplication table back within what the profiles allow.

Nothing here touches the archive. Those objects are under a compliance lock and are released by the lock expiring, which is the point: a retention that code could shorten would not be a retention. This keeps the projection from answering questions the archive no longer can, and keeps two tables that would otherwise grow without end from doing so.

func (Purge) Run 

func (r Purge) Run(ctx context.Context) (PurgeReport, error)

Run purges each profile and then the deduplication table.

type PurgeProfile 

type PurgeProfile struct {
	Profile string `json:"profile"`
	// Everything is the moment before which rows were removed, derived from the
	// profile's own retention.
	Everything time.Time `json:"everything_before"`
	// Identifying is the moment before which the actor, subject, address and
	// correlation columns were cleared, when a deployment sets a schedule.
	Identifying *time.Time `json:"identifying_before,omitempty"`
}

PurgeProfile is one profile's schedule.

type PurgeReport 

type PurgeReport struct {
	DryRun   bool           `json:"dry_run,omitempty"`
	Profiles []PurgeProfile `json:"profiles"`
	// Dedupe is the moment before which written identifiers were forgotten.
	Dedupe *time.Time `json:"dedupe_before,omitempty"`
}

PurgeReport is what a run forgot, or would have.

func (PurgeReport) String 

func (r PurgeReport) String() string

String is the human form.

type Reindex 

type Reindex struct {
	Store store.Store
	Index index.Indexer
	// Fields says which of an action's extension properties are indexed. It is
	// the catalogue's answer, resolved the same way the writer resolved it. A
	// reindex without it would quietly produce an index missing its data
	// columns, and because indexing is idempotent, a later run with the
	// catalogue would not repair it. Unset, the catalogues are read from the
	// archive, where the writer put each before any record that names it.
	Fields   observe.Fields
	Profile  string
	From, To time.Time
	// Batch is how many rows are indexed at a time. Default 500.
	Batch int
	JSON  bool
	Out   io.Writer
}

Reindex rebuilds a profile's index from the archive.

This command is what makes the index a projection rather than a second record. Everything a search answers with can be derived again from the objects, and the objects are the ones under an object lock that carry their own hashes. That is what lets the writer carry on when the index is unreachable, lets a deployment change the shape of the index without a migration of the trail, and lets an operator throw the database away.

It is a batch over a range of ingest days, for repairing one without waiting for the cursor to come round, and it shares what turns an object into rows with the indexer that follows the bucket (internal/observe): the two cannot disagree about what an object holds. The other repair is to reset the cursor (Reset) and let that indexer catch up.

It is safe to run over a range that is already indexed: an index counts a record once, and the line a row points at is the line the object has, so a second pass over the same objects changes nothing.

func (Reindex) Run 

func (r Reindex) Run(ctx context.Context) (ReindexReport, error)

Run walks the range and indexes what it finds.

type ReindexReport 

type ReindexReport struct {
	Profile string `json:"profile"`
	Objects int    `json:"objects"`
	Records int    `json:"records"`
	// Unreadable counts objects that could not be read or decoded. They are
	// reported rather than skipped in silence: an object the archive holds and
	// nothing can read is a finding, not a detail.
	Unreadable []string `json:"unreadable,omitempty"`
}

ReindexReport is what a reindex read and wrote.

func (ReindexReport) String 

func (r ReindexReport) String() string

String is the human form.

type Replay 

type Replay struct {
	Store store.Store
	// Sink is the writer. Nil means a dry run: the dead letters are read and
	// summarised and nothing is sent.
	Sink     sink.Sink
	From, To time.Time
	// Reason keeps only dead letters whose reason contains this text, which is
	// how an operator replays the cause they fixed and not the others.
	Reason string
	// Action keeps only dead letters of one action.
	Action string
	// Batch is how many records are sent at a time. Default 100.
	Batch int
	Out   io.Writer
}

Replay sends dead letters back to a writer.

Nothing the writer cannot process is dropped: it is kept under the dead-letter prefix with the reason, so that a fault upstream is a thing an operator fixes rather than a gap nobody can account for. Replay is the second half of that promise. Once the cause is fixed — a catalogue version registered, a profile configured, an emitter corrected — the records go back in, keeping their identifiers, and the writer's deduplication makes a replay of something that did get through cost nothing.

What replay does not do is repair a record. A record that was refused because it does not satisfy its schema will be refused again, and should be: the archive is not the place to correct what an emitter got wrong.

func (Replay) Run 

func (r Replay) Run(ctx context.Context) (ReplayReport, error)

Run reads the dead letters of the range and, unless this is a dry run, sends them back.

It reports how many failed again by listing the dead-letter prefix before and after. That works because a writer dead-letters within the call that carried the record, so once a blocking write returns, whatever it could not process is already back under the prefix. It is also the only way to tell: a writer accepts a record it dead-letters, and it is right to, because a record that can never be valid must not be retried forever by everything below.

type ReplayReport 

type ReplayReport struct {
	Read     int            `json:"read"`
	Sent     int            `json:"sent"`
	Skipped  int            `json:"skipped"`
	DeadEnd  int            `json:"dead_end"`
	Reasons  map[string]int `json:"reasons"`
	Rejected []string       `json:"rejected,omitempty"`
}

ReplayReport is what a replay found and did.

type SealCheck added in v0.7.0 

type SealCheck struct {
	// Roots are the thumbprints of the root keys the verifier pins. Nothing else
	// in the bucket is trusted, however it is signed.
	Roots []string
	// Settle is the notary's settle window and Grace how much later than that a
	// seal may still be missing: an hour is due once it has ended, Settle has
	// passed and Grace has passed.
	Settle, Grace time.Duration
}

SealCheck is what a verifier needs to check seals: the roots it trusts, and when a seal is late enough that its absence is a fault.

type SealedHour added in v0.7.0 

type SealedHour struct {
	Key     string    `json:"key"`
	Profile string    `json:"profile"`
	Tenant  string    `json:"tenant"`
	Hour    time.Time `json:"hour"`
	Objects int       `json:"objects"`
	Records int64     `json:"records"`
	Root    string    `json:"root"`
}

SealedHour is one seal this run wrote.

type Validate 

type Validate struct {
	PresetDirs   []string
	CatalogueDoc []string
	Deployment   string
	Out          io.Writer
}

Validate holds every preset and catalogue under the given paths to the contracts this repository publishes, and reports everything wrong at once rather than the first thing.

func (Validate) Run 

func (v Validate) Run() (problems int)

Run reports the number of problems found.

type Verify 

type Verify struct {
	Store   store.Store
	Profile string
	// From and To bound the range, of ingest time: the hours from the one From
	// is in up to but not including the hour To is in (an exact hour boundary
	// is excluded).
	From, To time.Time
	// MinimumRetention lets the check also say whether an object's lock is
	// shorter than its profile requires.
	MinimumRetention map[string]time.Duration
	// RequiredLock is the lock mode each profile demands, from the composed
	// deployment when the command was given one. With it an object with no
	// lock is INVALID under a profile that demands one and `unlocked` under
	// one that does not; without it locks are not spoken of.
	RequiredLock map[string]string
	// Sink and Catalogue, when both are given, are where this job records what
	// it checked. A verification that never ran and one that found nothing
	// wrong look identical in the archive; these events are the difference.
	Sink      sink.Sink
	Catalogue *catalogue.Catalogue
	Version   string
	Instance  string
	JSON      bool
	Out       io.Writer
	// Seals, when set, makes the check cover the seals of the range too.
	Seals *SealCheck
	// Now is the clock the missing-seal rule reads; nil is the wall clock.
	Now func() time.Time
}

Verify checks a profile's record objects over a range of ingest time against the bucket contract (docs/reference/bucket-contract.md): for every object, its key, its metadata, the sha256 of its stored bytes, and the hash on every one of its records.

It needs the archive and nothing else. An auditor runs it with read-only credentials and their own copy of this command, which is what makes the answer worth having: nothing in the result depends on trusting the operator of the archive. What the objects alone cannot say is that nothing was omitted or added: that is what seals vouch for (docs/decisions/0019-seals.md), and with Seals set this checks them too, against the roots it is given and nothing else.

func (Verify) Run 

func (v Verify) Run(ctx context.Context) (int, error)

Run reports the number of problems found.

type VerifyFinding added in v0.7.0 

type VerifyFinding struct {
	// Window is the hour of ingest time the object is in, as the prefix of its
	// hour: records/<profile>/<yyyy>/<mm>/<dd>/<hh>.
	Window string `json:"window"`
	Object string `json:"object,omitempty"`
	OK     bool   `json:"ok"`
	Reason string `json:"reason,omitempty"`
	// Unlocked is an object that is right and carries no retention, under a
	// profile that demands no lock: information, not a problem.
	Unlocked bool `json:"unlocked,omitempty"`
}

VerifyFinding is one thing wrong, or one object checked and right.

type VerifyReport added in v0.7.0 

type VerifyReport struct {
	Profile string    `json:"profile"`
	From    time.Time `json:"from"`
	To      time.Time `json:"to"`
	Objects int       `json:"objects"`
	Records int       `json:"records"`
	// Seals is how many seals were checked; zero when seals were not.
	Seals int `json:"seals,omitempty"`
	// Windows are the hours that had objects, in the order they were checked.
	// A caller that reports per window needs to know which were looked at, not
	// only which had something wrong.
	Windows  []string        `json:"windows,omitempty"`
	Findings []VerifyFinding `json:"findings"`
}

VerifyReport is what a verification found.

func (*VerifyReport) Problems added in v0.7.0 

func (r *VerifyReport) Problems() []VerifyFinding

Problems returns only what was wrong.

func (*VerifyReport) String added in v0.7.0 

func (r *VerifyReport) String() string

String renders the report as the command prints it.

func (*VerifyReport) Unlocked added in v0.7.0 

func (r *VerifyReport) Unlocked() int

Unlocked counts the objects that are right and carry no lock.

type WorkloadEntry 

type WorkloadEntry struct {
	Issuer  string `json:"issuer,omitempty"`
	Subject string `json:"subject"`
	Source  string `json:"source"`
}

WorkloadEntry is one service account and the source it speaks for.

type Workloads 

type Workloads struct {
	Issuers []authn.Issuer
	Map     auth.Workloads
}

Workloads is a workloads file read and checked.

func LoadWorkloads 

func LoadWorkloads(path string) (Workloads, error)

LoadWorkloads reads and checks a workloads file. It refuses one with no issuer, because a service configured with it could never admit anybody.

type WorkloadsFile 

type WorkloadsFile struct {
	// APIVersion is `truvity.github.io/<kind>/v1`, or absent, which means the same.
	APIVersion string          `json:"apiVersion,omitempty"`
	Issuers    []IssuerEntry   `json:"issuers"`
	Workloads  []WorkloadEntry `json:"workloads,omitempty"`
}

WorkloadsFile says which issuers the writer and the registry trust to name a calling workload, and which source each workload speaks for.

In a cluster the issuer is the cluster's own OIDC issuer and the subjects are service accounts: a workload presents its projected token, and the kubelet — not the workload — decides whose it is.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.