Documentation
¶
Overview ¶
Package flightrecorder wraps Go 1.25's runtime/trace.FlightRecorder with a clean lifecycle API and composable trigger conditions.
A flight recorder buffers the last few seconds of execution trace in memory. When a problem is detected (slow operation, error, panic), the program can snapshot exactly the problematic window of time for offline analysis with `go tool trace`.
Process-global constraint ¶
Go's runtime/trace allows only ONE active FlightRecorder per process. Calling Recorder.Start when another recorder is already running returns ErrAlreadyEnabled. Do not create multiple Recorder instances and call Start on all of them — design your application around a single recorder (typically created at startup, shared across middleware and host hooks).
Quick start ¶
recorder, _ := flightrecorder.New(
flightrecorder.WithSnapshotDir("/var/lib/app/traces"),
flightrecorder.WithCompression(gzip.BestSpeed), // 10x smaller files
flightrecorder.WithMaxSnapshots(50), // retain newest 50
flightrecorder.WithMinAge(10 * time.Second),
flightrecorder.WithMaxBytes(1 << 20), // 1 MiB
)
if err := recorder.Start(); err != nil {
log.Fatal(err)
}
defer recorder.Close()
// Later, when something goes wrong:
path, _ := recorder.SnapshotToDir(context.Background())
Decompress before analysing (go tool trace does not read .gz directly):
gunzip snapshot-*.trace.gz && go tool trace snapshot-*.trace
Trigger integration ¶
// Fire only when an operation exceeds 100ms:
trigger := flightrecorder.OnLatency(100*time.Millisecond)
recorder.SnapshotIf(ctx, flightrecorder.TriggerContext{
Kind: "command",
Type: "user.create",
Duration: 150 * time.Millisecond,
}, trigger)
Analyze the captured trace with: go tool trace snapshot.trace
Snapshot-to-directory, compression, and retention ¶
For auto-triggered captures, write timestamped snapshots to a directory, compress them, and retain only the newest:
recorder, _ := flightrecorder.New(
flightrecorder.WithSnapshotDir("/var/lib/app/traces"),
flightrecorder.WithCompression(gzip.BestSpeed),
flightrecorder.WithMaxSnapshots(50),
)
path, _ := recorder.SnapshotToDir(context.Background())
Non-blocking capture and graceful drain ¶
SnapshotIfAsync captures in a background goroutine so trace I/O does not block hot paths. Stop and Close drain all in-flight captures before shutting down.
Observability hooks (no dependencies) ¶
WithMetrics and WithLogger register callbacks so consumers wire their own Prometheus, OpenTelemetry, or log/slog backend without the library importing any of them. The SnapshotEvent passed to the metrics hook carries the TriggerContext.Kind and TriggerContext.Type so dashboards can label by operation (e.g. "http.request" vs "event.handler").
Error handling ¶
The package returns typed errors so callers can handle failure modes programmatically. All error types implement the standard [error] interface and support errors.Is and errors.As.
- ErrAlreadyEnabled / *AlreadyEnabledError — another recorder is active. - *ConfigError — invalid option passed to New. - *SnapshotError — IO failure during snapshot or close.
Example: distinguish error categories after Start.
recorder, err := flightrecorder.New(opts...)
if err != nil {
var cfgErr *flightrecorder.ConfigError
if errors.As(err, &cfgErr) {
log.Printf("bad config: %s %s", cfgErr.Field, cfgErr.Constraint)
}
return err
}
if err := recorder.Start(); err != nil {
if errors.Is(err, flightrecorder.ErrAlreadyEnabled) {
// Another recorder is active — reuse it or stop it first.
}
}
Index ¶
- Constants
- Variables
- type AlreadyEnabledError
- type ConfigError
- type LoggerHook
- type MetricsHook
- type Option
- func WithCompression(level int) Option
- func WithFile(path string) Option
- func WithLogger(hook LoggerHook) Option
- func WithMaxBytes(n uint64) Option
- func WithMaxSnapshots(n int) Option
- func WithMetrics(hook MetricsHook) Option
- func WithMinAge(d time.Duration) Option
- func WithSnapshotDir(dir string) Option
- func WithSnapshotPrefix(prefix string) Option
- func WithWriter(w io.Writer) Option
- type Recorder
- func (r *Recorder) Close() error
- func (r *Recorder) Enabled() bool
- func (r *Recorder) Reset()
- func (r *Recorder) Snapshot(ctx context.Context) error
- func (r *Recorder) SnapshotIf(ctx context.Context, tc TriggerContext, trigger TriggerFunc) bool
- func (r *Recorder) SnapshotIfAsync(ctx context.Context, tc TriggerContext, trigger TriggerFunc) bool
- func (r *Recorder) SnapshotToDir(ctx context.Context) (string, error)
- func (r *Recorder) SnapshotToFile(ctx context.Context, path string) error
- func (r *Recorder) SnapshotToWriter(ctx context.Context, dest io.Writer) (int64, error)
- func (r *Recorder) Start() error
- func (r *Recorder) Stop()
- type SnapshotError
- type SnapshotEvent
- type TriggerContext
- type TriggerFunc
Constants ¶
const ( // SnapshotSourceManual is set by [Recorder.Snapshot], [Recorder.SnapshotToFile], // and [Recorder.SnapshotToDir]. SnapshotSourceManual = "manual" // SnapshotSourceTrigger is set by [Recorder.SnapshotIf]. SnapshotSourceTrigger = "trigger" // SnapshotSourceAsync is set by [Recorder.SnapshotIfAsync]. SnapshotSourceAsync = "async" )
SnapshotSource labels the origin of a capture for observability consumers. Values are passed as SnapshotEvent.Source so a metrics hook can distinguish manual snapshots from triggered and asynchronous ones.
Variables ¶
var ErrAlreadyEnabled = errors.New(
"flightrecorder: another flight recorder is already active in this process",
)
ErrAlreadyEnabled is returned by Recorder.Start when another flight recorder is already active in this process. Go's runtime/trace allows only one active runtime/trace.FlightRecorder at a time.
Callers can check for this error using errors.Is:
if errors.Is(err, flightrecorder.ErrAlreadyEnabled) {
// Another recorder is active — reuse it or stop it first.
}
For richer error context, use errors.As with *AlreadyEnabledError.
Functions ¶
This section is empty.
Types ¶
type AlreadyEnabledError ¶ added in v0.1.1
type AlreadyEnabledError struct {
// Cause is the underlying runtime/trace error.
Cause error
}
AlreadyEnabledError indicates that Recorder.Start was called while another flight recorder or tracer is already active in this process.
The [Cause] field holds the underlying runtime error.
Both errors.Is with ErrAlreadyEnabled and errors.As with *AlreadyEnabledError match this error:
if errors.Is(err, flightrecorder.ErrAlreadyEnabled) {
// Sentinel check — backward compatible.
}
var ae *flightrecorder.AlreadyEnabledError
if errors.As(err, &ae) {
log.Printf("conflict: %v", ae.Cause)
}
func (*AlreadyEnabledError) Error ¶ added in v0.1.1
func (e *AlreadyEnabledError) Error() string
func (*AlreadyEnabledError) Is ¶ added in v0.1.1
func (e *AlreadyEnabledError) Is(target error) bool
Is reports whether this error matches the target. Returns true for ErrAlreadyEnabled so that errors.Is backward compatibility is preserved.
type ConfigError ¶ added in v0.1.1
type ConfigError struct {
// Field is the configuration option name: "MinAge" or "MaxBytes".
Field string
// Value is the invalid value that was provided.
Value any
// Constraint describes what the field must satisfy, e.g. "must be positive".
Constraint string
}
ConfigError describes invalid recorder configuration passed to New.
Callers can inspect the specific field and constraint:
var cfgErr *flightrecorder.ConfigError
if errors.As(err, &cfgErr) {
log.Printf("invalid %s: %s (got %v)", cfgErr.Field, cfgErr.Constraint, cfgErr.Value)
}
func (*ConfigError) Error ¶ added in v0.1.1
func (e *ConfigError) Error() string
type LoggerHook ¶ added in v0.2.0
LoggerHook receives diagnostic lifecycle messages (start, stop, cleanup, errors). The message is a printf-style format string and args are its arguments, mirroring log.Printf. It defaults to a no-op so the library never imports a logging package.
flightrecorder.WithLogger(func(format string, args ...any) {
slog.Info(fmt.Sprintf(format, args...))
})
type MetricsHook ¶ added in v0.2.0
type MetricsHook func(event SnapshotEvent, err error)
MetricsHook is invoked after every snapshot capture attempt that reaches the write stage. err is nil on success. Implementations must return quickly; if expensive processing is needed, do it asynchronously in consumer code.
The hook defaults to a no-op, keeping the library dependency-free. Consumers wire their own backend (Prometheus, OpenTelemetry, structured logs, etc.):
flightrecorder.WithMetrics(func(e flightrecorder.SnapshotEvent, err error) {
if err != nil {
snapshotErrors.Inc()
return
}
snapshotTotal.WithLabelValues(e.Source).Inc()
snapshotBytes.Observe(float64(e.Bytes))
})
type Option ¶
type Option func(*recorderConfig)
Option configures a Recorder.
func WithCompression ¶ added in v0.2.0
WithCompression enables gzip compression of snapshot output. The level maps directly to compress/gzip constants. A level of 0 (the default) disables compression entirely; pass gzip.DefaultCompression (-1), gzip.BestSpeed (1), gzip.BestCompression (9), or gzip.HuffmanOnly (-2) to enable it.
Compressed snapshot files use the ".trace.gz" extension and are loadable by `go tool trace` (supported since Go 1.19).
func WithFile ¶
WithFile sets the snapshot destination to a file at the given path. The file is opened (created or truncated) at Recorder.Snapshot time. For streaming to an existing io.Writer, use WithWriter instead.
func WithLogger ¶ added in v0.2.0
func WithLogger(hook LoggerHook) Option
WithLogger registers a LoggerHook for diagnostic lifecycle events (start, stop, cleanup, errors). The hook receives a printf-style format string and args. This is the dependency-free integration point for log/slog or any logging backend. See LoggerHook for details.
func WithMaxBytes ¶
WithMaxBytes sets the maximum size of the in-memory trace buffer. On average, expect a few MB of trace data per second of execution, or 10 MB/s for a busy service. Default: 10 MiB.
func WithMaxSnapshots ¶ added in v0.2.0
WithMaxSnapshots enables retention cleanup for directory-based snapshots. After each Recorder.SnapshotToDir call (and once during Recorder.Start), the oldest snapshot files in WithSnapshotDir beyond the count n are removed. A value of 0 (the default) disables retention.
Cleanup failures are reported via WithLogger and never fail a snapshot.
func WithMetrics ¶ added in v0.2.0
func WithMetrics(hook MetricsHook) Option
WithMetrics registers a MetricsHook invoked after every snapshot capture attempt that reaches the write stage. This is the dependency-free integration point for Prometheus, OpenTelemetry, or any metrics backend. The hook must return quickly. See MetricsHook for details.
func WithMinAge ¶
WithMinAge sets the minimum age of trace data that is reliably retained. The Go blog recommends setting this to ~2x the time window of the event you are debugging. For example, for a 5-second timeout, set 10 seconds. Default: 10s.
func WithSnapshotDir ¶ added in v0.2.0
WithSnapshotDir configures a directory for Recorder.SnapshotToDir, which writes each snapshot to an auto-generated, timestamped filename inside it. The directory is created (with os.MkdirAll, mode 0o750) on first use. There is no implicit default directory — snapshot-to-directory requires this option.
func WithSnapshotPrefix ¶ added in v0.2.0
WithSnapshotPrefix sets the filename prefix for Recorder.SnapshotToDir files (e.g. "snapshot-1700000000.trace"). It lets multiple services or instances share a directory without colliding. Default: "snapshot-".
func WithWriter ¶
WithWriter sets the destination for Recorder.Snapshot writes. If not set, snapshots are discarded (use Recorder.SnapshotToFile for file-based capture). For file output, use WithFile.
type Recorder ¶
type Recorder struct {
// contains filtered or unexported fields
}
Recorder wraps runtime/trace.FlightRecorder with safe lifecycle management, configurable snapshot sinks, and once-semantics to prevent snapshot races.
A Recorder is safe for concurrent use by multiple goroutines.
Only one flight recorder may be active per process. Calling Recorder.Start when another recorder is already running returns ErrAlreadyEnabled.
The lifecycle methods Recorder.Enabled, Recorder.Stop, and Recorder.Close are nil-safe: calling them on a nil *Recorder is a no-op (returns false / does nothing / returns nil). This supports the optional-recorder pattern where a struct embeds a *Recorder that may be nil when the feature is disabled.
func New ¶
New creates a Recorder from the given options. Returns an error if the configuration is invalid.
The Recorder is not started; call Recorder.Start to begin recording.
func (*Recorder) Close ¶
Close stops the recorder and closes any underlying resources (e.g., a file opened via WithFile). It is safe to call Close multiple times.
Recorder implements io.Closer so it can participate in shutdown ordering alongside other closable resources. Like Recorder.Stop, Close drains in-flight asynchronous captures first.
Close is nil-safe: calling it on a nil *Recorder returns nil.
func (*Recorder) Enabled ¶
Enabled reports whether the recorder is actively buffering traces.
Enabled is nil-safe: calling it on a nil *Recorder returns false.
func (*Recorder) Reset ¶
func (r *Recorder) Reset()
Reset clears the once-latch so that Recorder.Snapshot can fire again. Use this when you want to capture multiple snapshots over the recorder's lifetime (e.g., periodic slow-operation captures).
Reset does not restart a stopped recorder. Call Recorder.Start first if the recorder has been stopped.
func (*Recorder) Snapshot ¶
Snapshot writes the buffered trace to the configured writer. By default, only the first successful call has effect (once-semantics) to prevent snapshot races when multiple goroutines detect a problem simultaneously. Call Recorder.Reset to allow subsequent captures.
The context is checked for cancellation before the snapshot begins. If the context is already cancelled, Snapshot returns the context error immediately without writing. Note: runtime/trace.FlightRecorder.WriteTo does not accept a context, so cancellation cannot abort a write already in progress.
If the recorder is not enabled or has already been snapshotted, Snapshot is a no-op and returns nil.
func (*Recorder) SnapshotIf ¶
func (r *Recorder) SnapshotIf(ctx context.Context, tc TriggerContext, trigger TriggerFunc) bool
SnapshotIf evaluates the trigger against the given context and captures a snapshot if the trigger returns true. Returns true if a snapshot was initiated, false otherwise.
This is the primary method for middleware integration: the middleware constructs a TriggerContext from the operation result and delegates the decision to the trigger function.
func (*Recorder) SnapshotIfAsync ¶ added in v0.2.0
func (r *Recorder) SnapshotIfAsync(ctx context.Context, tc TriggerContext, trigger TriggerFunc) bool
SnapshotIfAsync is the non-blocking variant of Recorder.SnapshotIf: it evaluates the trigger and, if it fires, captures in a background goroutine, returning immediately. This is intended for hot paths (e.g., HTTP middleware) where trace file I/O must not block the response.
Returns true only when a capture was actually initiated. Returns false if the trigger did not fire OR if the recorder is shutting down (Recorder.Stop / Recorder.Close already began). In the shutdown case the capture is silently dropped — no goroutine is spawned.
The capture routes to the configured sink: WithSnapshotDir writes a new timestamped file; otherwise the writer sink (WithWriter/WithFile) is used with once-semantics.
The context is captured by the goroutine. If the context is cancelled before the write begins, the pre-write check skips the capture. Pass a context whose lifetime exceeds the write (e.g., detached from the request) if the snapshot must survive the caller returning.
Recorder.Stop and Recorder.Close drain all in-flight async captures before stopping, so the goroutine never outlives a clean shutdown.
func (*Recorder) SnapshotToDir ¶ added in v0.2.0
SnapshotToDir writes the trace to an auto-generated, timestamped file inside the directory configured with WithSnapshotDir. The directory is created on first use. The filename is "<prefix><unix-nano>.trace" (or ".trace.gz" when compression is enabled).
Unlike Recorder.Snapshot and Recorder.SnapshotToFile, SnapshotToDir is NOT once-latched: every call produces a new file. This supports the append-and-retain pattern for auto-triggered captures.
Calling SnapshotToDir without WithSnapshotDir returns a *ConfigError. When WithMaxSnapshots is set, retention cleanup runs after each write.
func (*Recorder) SnapshotToFile ¶
SnapshotToFile is a convenience that writes the trace to a file. It creates the file, writes the snapshot, and closes the file. Once-semantics apply as with Recorder.Snapshot.
The context is checked for cancellation before the snapshot begins. SnapshotToFile does NOT trigger retention cleanup; use Recorder.SnapshotToDir for the auto-named, retained directory pattern.
func (*Recorder) SnapshotToWriter ¶ added in v0.2.0
SnapshotToWriter writes the trace buffer directly to dest, bypassing the configured sink and the once-latch. This is the low-level escape hatch for callers that need a one-shot capture to an arbitrary destination (e.g., an HTTP response buffer for a /debug/trace endpoint).
Unlike Recorder.Snapshot, SnapshotToWriter:
- Does NOT use once-semantics (every call writes)
- Does NOT use the configured writer/file sink
- Does respect compression (if WithCompression is set)
Returns the number of bytes written (post-compression).
func (*Recorder) Start ¶
Start begins buffering execution trace in memory. Returns *AlreadyEnabledError (which satisfies errors.Is with ErrAlreadyEnabled) if another flight recorder or tracer is already active in this process.
If WithMaxSnapshots and WithSnapshotDir are configured, Start also prunes stale snapshot files left over from a previous process.
func (*Recorder) Stop ¶
func (r *Recorder) Stop()
Stop stops recording and releases the in-memory trace buffer. After Stop, Recorder.Enabled returns false and Recorder.Snapshot is a no-op. It is safe to call Stop multiple times.
Stop drains any in-flight asynchronous captures (Recorder.SnapshotIfAsync) before stopping, preventing a data race between runtime/trace.FlightRecorder.WriteTo and the runtime stop.
Stop is nil-safe: calling it on a nil *Recorder is a no-op.
type SnapshotError ¶ added in v0.1.1
type SnapshotError struct {
// Op is the operation that failed: "write", "create", or "close".
Op string
// Path is the file path involved, empty for writer-based snapshots.
Path string
// Err is the underlying error.
Err error
}
SnapshotError describes a failure during trace snapshot capture or snapshot file lifecycle.
Callers can inspect the operation and path for diagnostics:
var snapErr *flightrecorder.SnapshotError
if errors.As(err, &snapErr) {
log.Printf("%s %s failed: %v", snapErr.Op, snapErr.Path, snapErr.Err)
}
func (*SnapshotError) Error ¶ added in v0.1.1
func (e *SnapshotError) Error() string
type SnapshotEvent ¶ added in v0.2.0
type SnapshotEvent struct {
// Duration is the wall-clock time spent writing the trace to its sink.
Duration time.Duration
// Bytes is the number of bytes written to the final sink (after any
// compression). For failed writes this may be zero.
Bytes int64
// Path is the file path the snapshot was written to. It is empty for
// writer-based ([WithWriter]) snapshots.
Path string
// Compressed reports whether gzip compression was applied.
Compressed bool
// Source labels the capture origin: one of the SnapshotSource* constants
// (e.g. "manual", "trigger", "async").
Source string
// Kind is the operation category from [TriggerContext.Kind]
// (e.g. "command", "event", "query"). Empty for manual captures that
// have no associated trigger context.
Kind string
// Type is the specific operation type from [TriggerContext.Type]
// (e.g. "user.created", "order.processed"). Empty for manual captures.
Type string
}
SnapshotEvent describes a captured (or attempted) snapshot. It is passed to a MetricsHook after every capture attempt that reaches the write stage, whether the write succeeded or failed.
type TriggerContext ¶
type TriggerContext struct {
// Kind is the operation category: "command", "event", "query",
// "projection", or any custom string.
Kind string
// Type is the specific message or operation type
// (e.g., "user.created", "order.processed").
Type string
// Duration is how long the operation took.
Duration time.Duration
// Err is the error returned by the operation, or nil on success.
Err error
}
TriggerContext describes an operation that just completed. It is passed to TriggerFunc so the trigger can decide whether the operation warrants a flight recorder snapshot.
type TriggerFunc ¶
type TriggerFunc func(TriggerContext) bool
TriggerFunc decides whether a flight recorder snapshot should be captured based on the operation context.
Return true to capture a snapshot, false to skip.
func OnAll ¶
func OnAll(triggers ...TriggerFunc) TriggerFunc
OnAll returns a trigger that fires only if all given triggers fire. Useful for narrowing the capture window:
// Fire only on slow errors (not fast errors or slow successes):
trigger := flightrecorder.OnAll(
flightrecorder.OnError(),
flightrecorder.OnLatency(500*time.Millisecond),
)
func OnAlways ¶
func OnAlways() TriggerFunc
OnAlways returns a trigger that always fires. Useful for testing or for capturing a baseline trace on the first operation after startup.
func OnAny ¶
func OnAny(triggers ...TriggerFunc) TriggerFunc
OnAny returns a trigger that fires if any of the given triggers fire. This allows combining independent conditions:
// Fire on errors, or on commands that exceed 200ms:
trigger := flightrecorder.OnAny(
flightrecorder.OnError(),
flightrecorder.OnLatency(200*time.Millisecond),
)
func OnError ¶
func OnError() TriggerFunc
OnError returns a trigger that fires when an operation returns a non-nil error. Use this to capture traces of failures for root-cause analysis.
func OnErrorOrLatency ¶
func OnErrorOrLatency(threshold time.Duration) TriggerFunc
OnErrorOrLatency returns a trigger that fires when an operation either errors OR exceeds the given duration threshold. This is the most common trigger for production debugging — you want traces for both failures and latency spikes.
func OnLatency ¶
func OnLatency(threshold time.Duration) TriggerFunc
OnLatency returns a trigger that fires when an operation's duration exceeds the given threshold. Use this to capture traces of slow commands, queries, or event handlers.
Example: capture when any operation exceeds 100ms.
trigger := flightrecorder.OnLatency(100 * time.Millisecond)