Documentation
¶
Overview ¶
Package outputconfig loads audit output configuration from a YAML document and returns ready-to-use audit.Option values for audit.New.
Registry Pattern ¶
Output modules register factories via audit.RegisterOutputFactory in their init() functions. This package constructs outputs from YAML type strings without importing the output modules directly. The consumer controls which output types are available via blank imports:
import (
"github.com/axonops/audit/outputconfig"
_ "github.com/axonops/audit/file"
_ "github.com/axonops/audit/syslog"
_ "github.com/axonops/audit/webhook"
)
If an output type's module is not blank-imported, Load returns an error for that output — no output is silently dropped.
YAML Schema ¶
The configuration document has the following top-level keys:
version: 1 # required, must be 1
app_name: "my-service" # required, application name (max 255 bytes)
host: "${HOSTNAME:-localhost}" # required, hostname (max 255 bytes; env vars supported)
timezone: "UTC" # optional, overrides auto-detected timezone
auditor: # optional, core auditor settings
enabled: true # default: true
queue_size: 10000 # default: 10,000 (max: 1,000,000)
shutdown_timeout: "5s" # default: "5s" (max: "60s")
validation_mode: strict # "strict" (default), "warn", "permissive"
omit_empty: false # default: false
outputs: # required, map of named outputs
# TLS policy is configured per-output
# (see each output type's docs).
audit_log:
type: file # registered output type
enabled: true # optional, default true
file: # output-specific config block
path: /var/log/audit.log
max_size_mb: 100
formatter: # optional per-output formatter
type: cef
vendor: MyCompany
product: MyApp
route: # optional per-output event filter
include_categories: [security]
exclude_labels: [pii] # optional sensitivity label filter
hmac: # optional per-output HMAC integrity
enabled: true
salt:
version: "v1"
value: "${HMAC_SALT}"
algorithm: HMAC-SHA-256
Environment Variables ¶
Values support ${VAR} and ${VAR:-default} substitution. Expansion happens after YAML parsing for injection safety — the raw YAML structure is validated first, then string values are expanded.
Secret References ¶
String values in the YAML configuration can contain ref+SCHEME://PATH#KEY URIs that are resolved from external secret backends (OpenBao, Vault) at load time. Register providers with WithSecretProvider:
loaded, err := outputconfig.Load(ctx, yamlData, taxonomy,
outputconfig.WithSecretProvider(provider),
outputconfig.WithSecretTimeout(30*time.Second),
)
WithSecretTimeout controls the overall timeout for all secret resolution network I/O. Default: DefaultSecretTimeout (10s). The caller's context deadline takes precedence when earlier.
Usage ¶
The primary entry point is New; use NewWithLoad when LoadOption values are needed. Both construct a ready-to-use audit.Auditor in a single call:
auditor, err := outputconfig.New(ctx, taxonomyYAML, "outputs.yaml")
if err != nil {
return fmt.Errorf("audit config: %w", err)
}
defer auditor.Close()
When the pre-built auditor is not what you want — for example, you want to inspect the parsed outputs before constructing the auditor — call Load directly:
loaded, err := outputconfig.Load(ctx, yamlData, taxonomy)
if err != nil {
return fmt.Errorf("audit config: %w", err)
}
opts := append([]audit.Option{audit.WithTaxonomy(taxonomy)}, loaded.Options()...)
auditor, err := audit.New(opts...)
if err != nil {
_ = loaded.Close() // clean up outputs the auditor would have owned
return err
}
Load fails hard on any configuration error — partial configurations are never returned. This ensures that a misconfigured output does not silently drop audit events.
Index ¶
- Constants
- Variables
- func New(ctx context.Context, taxonomyYAML []byte, outputsConfigPath string, ...) (*audit.Auditor, error)
- func NewWithLoad(ctx context.Context, taxonomyYAML []byte, outputsConfigPath string, ...) (*audit.Auditor, error)
- type LoadOption
- func WithCoreMetrics(m audit.Metrics) LoadOption
- func WithDiagnosticLogger(l *slog.Logger) LoadOption
- func WithFactory(typeName string, factory audit.OutputFactory) LoadOption
- func WithOutputMetrics(factory audit.OutputMetricsFactory) LoadOption
- func WithSecretProvider(p secrets.Provider) LoadOption
- func WithSecretTimeout(d time.Duration) LoadOption
- type Loaded
- func (l *Loaded) AppName() string
- func (l *Loaded) Close() error
- func (l *Loaded) Host() string
- func (l *Loaded) Options() []audit.Option
- func (l *Loaded) OutputMetadata() []OutputInfo
- func (l *Loaded) Outputs() []audit.Output
- func (l *Loaded) StandardFields() map[string]any
- func (l *Loaded) String() string
- func (l *Loaded) Timezone() string
- type OutputInfo
Examples ¶
Constants ¶
const DefaultSecretTimeout = 10 * time.Second
DefaultSecretTimeout is the default timeout for secret resolution when no explicit timeout is configured via WithSecretTimeout.
const MaxOutputConfigSize = 1 << 20 // 1 MiB
MaxOutputConfigSize is the maximum YAML input size accepted by Load.
const MaxOutputCount = 100
MaxOutputCount is the maximum number of outputs in a single config.
const MaxSecretValueSize = 64 << 10 // 64 KiB
MaxSecretValueSize is the maximum size in bytes for a resolved secret value. Values exceeding this limit are rejected.
Variables ¶
var ErrOutputConfigInvalid = fmt.Errorf("audit/outputconfig: output config validation failed: %w", audit.ErrConfigInvalid)
ErrOutputConfigInvalid is the sentinel error wrapped by output configuration validation failures. It itself wraps audit.ErrConfigInvalid so consumers can match at either level via errors.Is:
errors.Is(err, outputconfig.ErrOutputConfigInvalid) // specific errors.Is(err, audit.ErrConfigInvalid) // generic
The wrapping relationship mirrors stdlib's `fs.ErrNotExist` / `os.ErrNotExist` pattern.
Functions ¶
func New ¶ added in v0.1.11
func New(ctx context.Context, taxonomyYAML []byte, outputsConfigPath string, opts ...audit.Option) (*audit.Auditor, error)
New is the convenience facade that creates a ready-to-use audit.Auditor from embedded taxonomy YAML and a filesystem path to the outputs configuration. It combines audit.ParseTaxonomyYAML, Load, and audit.New into a single call — the 80 % case.
When outputsConfigPath is empty, New creates a stdout-only development auditor with app_name derived from os.Args and host from os.Hostname. Useful for local development and quick evaluation.
opts are applied after the options produced by Load, so they take precedence (last wins). Use this to add audit.WithMetrics, audit.WithDisabled, or other overrides.
For advanced cases that need LoadOption values — secret providers, per-output metrics factories, custom output factory registrations — use NewWithLoad instead.
Non-stdout output types require blank imports to register their factories:
import _ "github.com/axonops/audit/file" // file output import _ "github.com/axonops/audit/syslog" // syslog output import _ "github.com/axonops/audit/webhook" // webhook output import _ "github.com/axonops/audit/loki" // loki output
— or the convenience umbrella package:
import _ "github.com/axonops/audit/outputs" // all output types
Example ¶
ExampleNew shows the simplest consumer flow: embed the taxonomy, point New at a filesystem path for the outputs YAML, get back a ready-to-use *audit.Auditor. Additional audit.Option values override the Load-derived ones (last wins).
package main
import (
"context"
"fmt"
"os"
"github.com/axonops/audit"
"github.com/axonops/audit/outputconfig"
)
// writeTempYAML writes content to a temp file and returns its path.
// Only used by the runnable examples — production consumers use
// `go:embed` + a real filesystem path.
func writeTempYAML(content []byte) string {
f, err := os.CreateTemp("", "outputs-*.yaml")
if err != nil {
panic(err)
}
if _, wErr := f.Write(content); wErr != nil {
panic(wErr)
}
_ = f.Close()
return f.Name()
}
func removeFile(path string) { _ = os.Remove(path) }
var exampleTaxonomyYAML = []byte(`
version: 1
categories:
write:
- user_create
events:
user_create:
fields:
outcome: {required: true}
`)
var exampleOutputsYAML = []byte(`
version: 1
app_name: example-app
host: example-host
outputs:
console:
type: stdout
`)
func main() {
// In a real program you would load a separate YAML file from disk:
// auditor, err := outputconfig.New(ctx, taxonomyYAML, "outputs.yaml")
//
// This runnable example writes the YAML to a temp file so it can
// run under `go test`.
tmp := writeTempYAML(exampleOutputsYAML)
defer removeFile(tmp)
auditor, err := outputconfig.New(context.Background(), exampleTaxonomyYAML, tmp,
audit.WithDisabled(), // last-wins override — keeps the example silent
)
if err != nil {
fmt.Println("new:", err)
return
}
defer func() { _ = auditor.Close() }()
fmt.Println("disabled:", auditor.IsDisabled())
}
Output: disabled: true
func NewWithLoad ¶ added in v0.1.12
func NewWithLoad(ctx context.Context, taxonomyYAML []byte, outputsConfigPath string, loadOpts []LoadOption, opts ...audit.Option) (*audit.Auditor, error)
NewWithLoad is like New but forwards loadOpts to Load. Use when you need one of WithSecretProvider, WithCoreMetrics, WithOutputMetrics, WithFactory, WithSecretTimeout, or WithDiagnosticLogger. For the common no-LoadOption case, prefer New.
Example ¶
ExampleNewWithLoad shows the advanced form that accepts LoadOption values — used when the consumer needs a custom secret provider, a core-metrics recorder, or per-output metrics factory. For the simple no-LoadOption case use New.
package main
import (
"context"
"fmt"
"os"
"github.com/axonops/audit"
"github.com/axonops/audit/outputconfig"
)
// writeTempYAML writes content to a temp file and returns its path.
// Only used by the runnable examples — production consumers use
// `go:embed` + a real filesystem path.
func writeTempYAML(content []byte) string {
f, err := os.CreateTemp("", "outputs-*.yaml")
if err != nil {
panic(err)
}
if _, wErr := f.Write(content); wErr != nil {
panic(wErr)
}
_ = f.Close()
return f.Name()
}
func removeFile(path string) { _ = os.Remove(path) }
var exampleTaxonomyYAML = []byte(`
version: 1
categories:
write:
- user_create
events:
user_create:
fields:
outcome: {required: true}
`)
var exampleOutputsYAML = []byte(`
version: 1
app_name: example-app
host: example-host
outputs:
console:
type: stdout
`)
func main() {
tmp := writeTempYAML(exampleOutputsYAML)
defer removeFile(tmp)
auditor, err := outputconfig.NewWithLoad(context.Background(),
exampleTaxonomyYAML, tmp,
[]outputconfig.LoadOption{
outputconfig.WithCoreMetrics(nil), // nil metrics = no-op, illustrates threading
},
audit.WithDisabled(),
)
if err != nil {
fmt.Println("new:", err)
return
}
defer func() { _ = auditor.Close() }()
fmt.Println("disabled:", auditor.IsDisabled())
}
Output: disabled: true
Types ¶
type LoadOption ¶
type LoadOption func(*loadOptions)
LoadOption configures optional behaviour for Load.
func WithCoreMetrics ¶
func WithCoreMetrics(m audit.Metrics) LoadOption
WithCoreMetrics sets the core audit.Metrics implementation that is forwarded to output factories during construction. If m is nil, factories receive nil metrics (equivalent to not calling this option). This option replaces the former positional `coreMetrics` parameter on Load.
func WithDiagnosticLogger ¶ added in v0.1.11
func WithDiagnosticLogger(l *slog.Logger) LoadOption
WithDiagnosticLogger sets the diagnostic logger that is threaded through to every output constructed by Load. The logger reaches each output's [OutputFactory] via its logger parameter, so construction-time warnings (TLS policy, file permission mode) route to the consumer's configured handler rather than slog.Default.
Pair this with audit.WithDiagnosticLogger on the audit.Auditor so that construction-time AND runtime warnings route through the same handler. Passing nil is valid and equivalent to not calling this option — factories fall back to slog.Default.
func WithFactory ¶
func WithFactory(typeName string, factory audit.OutputFactory) LoadOption
WithFactory registers a per-call output factory override for the given type name. Per-call factories take precedence over globally registered factories. Multiple calls for the same type: last wins.
Choosing a registration path ¶
WithFactory is one of two registration paths. The other is github.com/axonops/audit.RegisterOutputFactory (typically invoked from init() via a blank-import of an output sub-module such as github.com/axonops/audit/outputs or github.com/axonops/audit/file), which mutates the global registry and applies process-wide.
Use WithFactory for tests, per-call overrides, or multiple auditors in one process with different factory bindings. Use RegisterOutputFactory (via blank-import) for default production setup. See the "Output Factory Registration" section of docs/output-configuration.md for full guidance on choosing between them.
func WithOutputMetrics ¶ added in v0.1.11
func WithOutputMetrics(factory audit.OutputMetricsFactory) LoadOption
WithOutputMetrics sets the audit.OutputMetricsFactory used to create per-output metrics during Load. The factory is called once per output with the output type name and YAML key name. Pass nil to disable per-output metrics.
The factory is invoked BEFORE output construction (#696). The resulting audit.OutputMetrics value is threaded into the output's constructor via audit.FrameworkContext, which means rotation / reconnect telemetry sees the correct sink from the very first event onward.
func WithSecretProvider ¶
func WithSecretProvider(p secrets.Provider) LoadOption
WithSecretProvider registers a secret provider for resolving ref+ URIs during config loading. Multiple providers can be registered for different schemes. Duplicate schemes cause Load to return an error.
func WithSecretTimeout ¶
func WithSecretTimeout(d time.Duration) LoadOption
WithSecretTimeout sets the overall timeout for secret resolution. This timeout applies to all provider Resolve calls combined during a single Load invocation. Default: DefaultSecretTimeout (10s). The caller's context deadline takes precedence when earlier.
type Loaded ¶ added in v0.1.12
type Loaded struct {
// contains filtered or unexported fields
}
Loaded is the result of parsing an outputs YAML configuration via Load. It aggregates the constructed outputs and the audit.Option values required to create an auditor. Fields are unexported — use the method accessors below.
The zero value is not usable; always obtain via Load.
func Load ¶
func Load(ctx context.Context, data []byte, taxonomy *audit.Taxonomy, opts ...LoadOption) (*Loaded, error)
Load parses a YAML output configuration, constructs all outputs via the registry, validates routes against the taxonomy, and returns audit.Option values ready for audit.New.
Load fails hard on any error — unknown output types, missing factory registrations, invalid YAML, unknown YAML keys, malformed routes, unresolvable environment variables, or route references to taxonomy entries that don't exist. Audit is a compliance function — silent misconfiguration is worse than refusing to start.
Environment variable substitution (${VAR} and ${VAR:-default}) runs on string values in the parsed YAML tree, NOT on raw bytes. This prevents YAML injection via env var values.
Secret reference resolution (ref+SCHEME://PATH#KEY) runs after env var expansion when providers are registered via WithSecretProvider or configured in the YAML secrets: section. The ctx parameter controls timeout for network I/O during secret resolution. Use WithSecretTimeout or the YAML secrets.timeout field to configure the resolution timeout.
When providers are configured in the YAML secrets: section, they are constructed, used for resolution, and closed within Load. They do not outlive the call.
Example ¶
ExampleLoad shows how to parse an outputs YAML configuration into a *Loaded and feed its options into audit.New directly — useful when the consumer needs the parsed outputs for inspection before constructing the auditor, or wants to mix in additional audit.Option values that are not expressible as LoadOption.
package main
import (
"context"
"fmt"
"github.com/axonops/audit"
"github.com/axonops/audit/outputconfig"
)
var exampleTaxonomyYAML = []byte(`
version: 1
categories:
write:
- user_create
events:
user_create:
fields:
outcome: {required: true}
`)
var exampleOutputsYAML = []byte(`
version: 1
app_name: example-app
host: example-host
outputs:
console:
type: stdout
`)
func main() {
tax, err := audit.ParseTaxonomyYAML(exampleTaxonomyYAML)
if err != nil {
fmt.Println("parse taxonomy:", err)
return
}
loaded, err := outputconfig.Load(context.Background(), exampleOutputsYAML, tax)
if err != nil {
fmt.Println("load:", err)
return
}
opts := append([]audit.Option{audit.WithTaxonomy(tax)}, loaded.Options()...)
auditor, err := audit.New(opts...)
if err != nil {
_ = loaded.Close() // clean up outputs the auditor would have owned
fmt.Println("new auditor:", err)
return
}
defer func() { _ = auditor.Close() }()
fmt.Println("outputs:", len(loaded.OutputMetadata()))
fmt.Println("app_name:", loaded.AppName())
}
Output: outputs: 1 app_name: example-app
func (*Loaded) AppName ¶ added in v0.1.12
AppName returns the application name parsed from the top-level `app_name:` key. Always non-empty after a successful Load.
func (*Loaded) Close ¶ added in v0.1.12
Close closes every output constructed by Load. Intended for use when the caller decides not to construct an auditor after a successful Load — for example, if audit.New returns an error and the outputs need cleanup. Returns the first non-nil Close error encountered; subsequent errors are swallowed.
Close is not safe to call more than once, and MUST NOT be called after the Loaded.Outputs have been handed to a live audit.Auditor — the auditor takes ownership in that case.
func (*Loaded) Host ¶ added in v0.1.12
Host returns the hostname parsed from the top-level `host:` key. Always non-empty after a successful Load.
func (*Loaded) Options ¶ added in v0.1.12
Options returns the slice of audit.Option values ready to pass to audit.New, including framework-field options (app_name, host, timezone), config-equivalent options (audit.WithQueueSize, etc.), standard-field defaults, and one audit.WithNamedOutput per configured output.
The returned slice aliases the internal storage with capacity trimmed to length, so a caller's `append(loaded.Options(), extra)` cannot silently mutate the internal slice — append will allocate a new backing array. The caller must still not mutate the returned elements themselves.
The caller must also supply audit.WithTaxonomy separately; it is not part of the returned slice because the same taxonomy is typically needed independently by the application.
func (*Loaded) OutputMetadata ¶ added in v0.1.12
func (l *Loaded) OutputMetadata() []OutputInfo
OutputMetadata returns a diagnostic snapshot of each configured output — name, type, route, formatter, HMAC config, exclude labels — in YAML declaration order. Useful for tests asserting that a YAML config parsed as intended.
The returned slice is a shallow copy; the OutputInfo values themselves reference the same underlying pointers as the auditor pipeline, so callers MUST NOT mutate the referenced values.
func (*Loaded) Outputs ¶ added in v0.1.12
Outputs returns the constructed audit.Output instances in YAML declaration order. Intended for cleanup when the caller decides not to construct an auditor after a successful Load — call Loaded.Close or iterate and close each output individually.
The returned slice is a shallow copy; mutating it does not affect subsequent calls to Outputs, OutputMetadata, or the auditor.
func (*Loaded) StandardFields ¶ added in v0.1.12
StandardFields returns the reserved-standard-field defaults parsed from the top-level `standard_fields:` section, or nil when not specified. The returned map is the internal one; callers MUST NOT mutate it.
Values carry the YAML-decoded Go type (int for YAML integers, string for YAML strings, time.Time for ISO8601 timestamps decoded natively). Type mismatches against the reserved-field declared type are reported by audit.WithStandardFieldDefaults at audit.New time.
type OutputInfo ¶ added in v0.1.12
type OutputInfo struct {
// Name is the config-level name of the output, as declared in the
// YAML outputs map key.
Name string
// Type is the output type name (e.g. "file", "syslog", "webhook",
// "loki", "stdout") as declared in the YAML type: field.
Type string
// Output is the constructed output instance.
Output audit.Output
// Route is the optional per-output event filter. Nil means all
// events are delivered to this output.
Route *audit.EventRoute
// Formatter is the optional per-output formatter override. Nil
// means the auditor's default formatter is used.
Formatter audit.Formatter
// HMACConfig is the optional per-output HMAC configuration.
// Nil means no HMAC for this output.
HMACConfig *audit.HMACConfig
// ExcludeLabels lists sensitivity label names whose fields are
// stripped from events before delivery to this output. Nil or
// empty means no field stripping.
ExcludeLabels []string
}
OutputInfo is a diagnostic snapshot of a single configured output, returned by Loaded.OutputMetadata. Fields mirror the YAML declaration; modifying them has no effect on the auditor pipeline.
func (*OutputInfo) String ¶ added in v0.1.12
func (o *OutputInfo) String() string
String returns a safe representation of OutputInfo that never includes credentials, header values, or resolved environment variable values.
Source Files
Directories