README
¶
audit
An audit trail an application owns: one record format, one write path, an immutable archive, and projections for security, billing and history.
An installation belongs to one application and runs in its namespace, rendered by its chart. The application emits through a library; the receiver, the writer and the query service are separate Deployments, so the application holds no credentials for the archive and a fix never rebuilds it.
flowchart LR
subgraph app["application pod"]
E["emit library<br/>validates, block or async"]
end
E -- "ack = durable" --> R["receiver"]
R -- "direct mode" --> W["writer"]
R -- "stream mode" --> N[("JetStream")] --> W
W --> S3[("Object-Locked bucket<br/>THE RECORD")]
W --> PG[("index — rebuildable")]
D["digest, hourly<br/>verify, nightly"] --> S3
Q["query service"] --> PG
Q --> S3
UI["Audit page in the<br/>application's console"] --> Q
The two shapes
| shape | for | how a record becomes durable |
|---|---|---|
| direct | an internal service, or a cluster with no stream | the receiver puts the object, then acknowledges |
| stream | a product: many pods, metering, quotas | the receiver publishes, writers consume and put |
Both write the same archive and are verified by the same command. There is no shape that puts the writer inside the application (why).
The two extensions
| extension | what it adds | needs |
|---|---|---|
| billing | rollups at index time, an immutable monthly statement | a metering profile |
| usage quotas | a usage consumer, a counter cache, an hourly reconciler | stream mode |
Start here
| you want to | read |
|---|---|
| see how it fits together | Architecture — the parts, the catalogue, what an acknowledgement means, what can be lost |
| connect an application | Integrating — the catalogue, one constructor per action, the CI check, the Audit page |
| run one in a cluster | Deployment — what to prepare, then direct or stream |
| record what your application does | Emitting — deliveries, registration, the Go emitter; examples/emit |
| search the trail, or audit it | Reading — grants, the API, the clients, audit verify; examples/read |
| know which presets to compose | Presets policy, then presets/ |
| understand why it is built this way | why, concepts, the decisions |
| work on this repository | layout, CONTRIBUTING |
Consumers
| repo | surface |
|---|---|
| truvity/gitops | chart audit |
| A second, non-AWS estate | chart audit |
| truvity/cloudflare | Go, from r2broker |
Neighbours
- access-roster ↔ openbao ↔ audit. access-roster is the issuer; openbao
is a relying party (its own
docs/integrations/access-roster.md); audit is the record every one of them writes.
Status
| part | state |
|---|---|
Record, catalogues, presets, audit validate / check-emitters |
built |
Go emitter: block and async; Connect and JetStream sinks |
built |
| Writer: split per profile, Object Lock where a profile demands it, index, dead letters, legal holds, retention addenda | built |
Two store tiers on any S3-compatible store: record (Object Lock in compliance mode) and attested (chain under a managed key, no lock) (0014) |
built |
| Query service: search, facets, get, export, tail; JWT with declarative grants | built |
Digest chain and audit verify; signing with a key file, AWS KMS or OpenBAO transit |
built |
Pseudonymisation keys: local and OpenBAO transit, off by default (0013) |
built |
Helm chart: mode, receiver, writer, query service, the four jobs, the extension toggles |
built; a golden per shape, and every documented example rendered |
@truvity/audit: query client, sentences, React hooks and view |
built; consumed from a release tag (github:truvity/audit#vX.Y.Z), not from a registry |
| TypeScript emitter | designed, not built |
| Billing statement, usage consumer, reconciler | designed, not built |
| Exporters (OCSF, ECS, Parquet), adapters | designed, not built |
What it is not
- Not an application log pipeline. Logs keep flowing through the observability stack. This records what happened, who did it, to what, with what outcome, and keeps it for as long as a framework requires.
- Not a SIEM. It exports to one.
- Not a wallet transaction log. A digital identity wallet's own log lives on the user's device and is invisible to the provider by law. This component sees operational events only.
Releasing
A pushed v* tag runs release.yaml, the
shared release-public workflow
(truvity/ci-workflows): it builds
the toolchain archives and, through ko, the three images the chart
deploys, and packages and pushes the audit chart to
oci://ghcr.io/truvity/charts/audit at the tag's version.
charts/audit/Chart.yaml's committed
version: 0.0.0 / appVersion: "0.0.0" are placeholders the release
stamps over — never bump them by hand. The CLI is also published as a Nix
flake release asset, for pinning audit verify the way other repositories
pin accessctl.
Licence
MIT. See LICENSE.
Documentation
¶
Overview ¶
Package audit carries the files this repository publishes as data: the framework presets, the meta-schemas that govern catalogues and presets, and the common catalogue of the component's own events.
They are embedded so that a deployment gets them from the binary it already runs, rather than from files it has to ship and keep in step.
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var Catalogue embed.FS
Catalogue is the common catalogue: the component's own meta-events, which every deployment carries.
View Source
var Generated embed.FS
Generated holds what this build published for readers outside Go: the record's JSON Schema, with the proto's comments carried as descriptions. The writer puts a copy of it in the archive beside the records it describes.
View Source
var Presets embed.FS
Presets are the framework presets under presets/.
View Source
var Proto embed.FS
Proto is the schema of record itself. The writer copies the proto of a record's major beside the record's JSON Schema on first use, so the archive keeps the meaning of every field and not only its shape.
View Source
var Schemas embed.FS
Schemas are the meta-schemas under schemas/: catalogue, preset, extension.
Functions ¶
This section is empty.
Types ¶
This section is empty.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package auth is where a deployment plugs in who is asking and what they may see.
|
Package auth is where a deployment plugs in who is asking and what they may see. |
|
Package catalogue reads and validates the action catalogues that describe what a source emits.
|
Package catalogue reads and validates the action catalogues that describe what a source emits. |
|
cmd
|
|
|
audit
command
Command audit is the tool that holds a deployment to the contracts this repository publishes: it validates catalogues and presets, explains what a profile keeps, and checks that the code and the catalogue still agree.
|
Command audit is the tool that holds a deployment to the contracts this repository publishes: it validates catalogues and presets, explains what a profile keeps, and checks that the code and the catalogue still agree. |
|
audit-query
command
Command audit-query answers questions about what was written.
|
Command audit-query answers questions about what was written. |
|
audit-writer
command
Command audit-writer is an installation's front door and its write path, which are one process in a small installation and two in a busy one.
|
Command audit-writer is an installation's front door and its write path, which are one process in a small installation and two in a busy one. |
|
protoc-gen-audit-jsonschema
command
Command protoc-gen-audit-jsonschema is the buf plugin that writes the record's JSON Schema.
|
Command protoc-gen-audit-jsonschema is the buf plugin that writes the record's JSON Schema. |
|
deploy
|
|
|
pulumi
module
|
|
|
e2e
|
|
|
fixture
Package fixture stands in for the platform on the local kind box (see truvity/policy's hack/kind/README.md and docs/decisions/0005-kind-is-the-gate.md there): a Postgres database and its two roles, a JetStream stream, an S3 bucket and the two signing keys the digest and verify jobs need — the things a real deployment's platform would already have provisioned before `helm install audit` ever runs.
|
Package fixture stands in for the platform on the local kind box (see truvity/policy's hack/kind/README.md and docs/decisions/0005-kind-is-the-gate.md there): a Postgres database and its two roles, a JetStream stream, an S3 bucket and the two signing keys the digest and verify jobs need — the things a real deployment's platform would already have provisioned before `helm install audit` ever runs. |
|
fixture/cmd/resolve
command
Command resolve prints e2e/fixture's names as shell variable assignments, so apply.sh and every hack script that installs onto the same names can `eval` this rather than repeating them.
|
Command resolve prints e2e/fixture's names as shell variable assignments, so apply.sh and every hack script that installs onto the same names can `eval` this rather than repeating them. |
|
Package emit is what an application imports to record what it did.
|
Package emit is what an application imports to record what it did. |
|
examples
|
|
|
emit
command
Command emit is an example application that records what it does into the audit trail.
|
Command emit is an example application that records what it does into the audit trail. |
|
read
command
Command read is an example of reading the audit trail from Go: a search, paged to the end with its cursor, and one record read with where it came from.
|
Command read is an example of reading the audit trail from Go: a search, paged to the end with its cursor, and one record read with where it came from. |
|
gen
|
|
|
hack
|
|
|
goreleaser-snapshot-config
command
Command goreleaser-snapshot-config derives a single-architecture copy of .goreleaser.yaml for the local and CI loops that build straight into the kind box's own registry (see hack/example-snapshot.sh).
|
Command goreleaser-snapshot-config derives a single-architecture copy of .goreleaser.yaml for the local and CI loops that build straight into the kind box's own registry (see hack/example-snapshot.sh). |
|
Package index is the projection a search reads.
|
Package index is the projection a search reads. |
|
indextest
Package indextest is the one corpus every searcher is asked about.
|
Package indextest is the one corpus every searcher is asked about. |
|
postgres
Package postgres is the default index and the shared deduplication store.
|
Package postgres is the default index and the shared deduplication store. |
|
s3scan
Package s3scan answers queries by reading the archive, with no index at all.
|
Package s3scan answers queries by reading the archive, with no index at all. |
|
internal
|
|
|
authtest
Package authtest is an identity provider for tests: a discovery document and a key set served over a real listener, so the code under test runs its actual discovery and key fetch rather than being handed a key.
|
Package authtest is an identity provider for tests: a discovery document and a key set served over a real listener, so the code under test runs its actual discovery and key fetch rather than being handed a key. |
|
cli
Package cli is the implementation behind cmd/audit.
|
Package cli is the implementation behind cmd/audit. |
|
clock
Package clock measures how far this machine's clock is from UTC.
|
Package clock measures how far this machine's clock is from UTC. |
|
corpus
Package corpus reads the example records in testdata/records, for tests that hold a transport or an encoder to them.
|
Package corpus reads the example records in testdata/records, for tests that hold a transport or an encoder to them. |
|
digest
Package digest builds and checks the chain that makes the archive tamper-evident.
|
Package digest builds and checks the chain that makes the archive tamper-evident. |
|
hold
Package hold places and releases legal holds over the archive.
|
Package hold places and releases legal holds over the archive. |
|
identity
Package identity is the way back from a pseudonym to the person, for the cases the law requires and only for them.
|
Package identity is the way back from a pseudonym to the person, for the cases the law requires and only for them. |
|
metaschema
Package metaschema validates a document against one of the meta-schemas this repository publishes: the catalogue format, the preset format, and the constraints every extension-slot schema must satisfy.
|
Package metaschema validates a document against one of the meta-schemas this repository publishes: the catalogue format, the preset format, and the constraints every extension-slot schema must satisfy. |
|
pgtest
Package pgtest gives a test its own Postgres schema.
|
Package pgtest gives a test its own Postgres schema. |
|
query
Package query is the read service: it turns a request into a closed query, narrows it to what the caller was granted, asks a searcher, and records that the read happened.
|
Package query is the read service: it turns a request into a closed query, narrows it to what the caller was granted, asks a searcher, and records that the read happened. |
|
registry
Package registry holds the catalogues a deployment has registered.
|
Package registry holds the catalogues a deployment has registered. |
|
s3test
Package s3test gives a test a real S3 to walk.
|
Package s3test gives a test a real S3 to walk. |
|
schemagen
Package schemagen turns the record's protobuf descriptor into a JSON Schema of the form this project writes.
|
Package schemagen turns the record's protobuf descriptor into a JSON Schema of the form this project writes. |
|
telemetry
Package telemetry is the OpenTelemetry metrics this repository's processes publish.
|
Package telemetry is the OpenTelemetry metrics this repository's processes publish. |
|
writer
Package writer turns records into the copies the profiles keep, and puts those copies where they are kept.
|
Package writer turns records into the copies the profiles keep, and puts those copies where they are kept. |
|
Package keys holds the two kinds of key this system needs and the interfaces a deployment plugs its own behind.
|
Package keys holds the two kinds of key this system needs and the interfaces a deployment plugs its own behind. |
|
Package preset reads the framework presets and composes them into profiles.
|
Package preset reads the framework presets and composes them into profiles. |
|
Package query is the query service as a library: search, facets, get, export, tail and resolve over what the writer wrote, behind the grants a caller holds, with every read recorded in the trail it reads.
|
Package query is the query service as a library: search, facets, get, export, tail and resolve over what the writer wrote, behind the grants a caller holds, with every read recorded in the trail it reads. |
|
Package record is the canonical audit record: the generated protobuf type plus the rules the type alone cannot express — identifiers, bounds, the negative list, and the canonical JSON form that every hash is taken over.
|
Package record is the canonical audit record: the generated protobuf type plus the rules the type alone cannot express — identifiers, bounds, the negative list, and the canonical JSON form that every hash is taken over. |
|
sdk
module
|
|
|
Package sink is the write contract, and it is the same contract at every hop.
|
Package sink is the write contract, and it is the same contract at every hop. |
|
natssink
Package natssink publishes records to a NATS JetStream stream, and consumes them again on the other side.
|
Package natssink publishes records to a NATS JetStream stream, and consumes them again on the other side. |
|
Package store is the object store the archive lives in.
|
Package store is the object store the archive lives in. |
|
s3store
Package s3store keeps the archive in an S3 bucket.
|
Package s3store keeps the archive in an S3 bucket. |
|
storetest
Package storetest holds the store a test writes to.
|
Package storetest holds the store a test writes to. |
|
Package wire is how this repository's services speak JSON.
|
Package wire is how this repository's services speak JSON. |
|
Package writer is the audit writer as a library: the thing that takes records, splits them into the copies their profiles keep, pseudonymises, and puts them into the locked archive.
|
Package writer is the audit writer as a library: the thing that takes records, splits them into the copies their profiles keep, pseudonymises, and puts them into the locked archive. |
Click to show internal directories.
Click to hide internal directories.