traceloom

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 13, 2026 License: MIT Imports: 25 Imported by: 0

README

Traceloom for Go

A lightweight, dependency-free Go library for recording structured event timelines grouped by trace ID.

Traceloom is intended for small APIs, backend services, jobs, webhooks, and command-line programs where you want to reconstruct one logical process without running a full observability stack. Its JSONL wire format is compatible with the PHP and Node.js Traceloom packages.

It is not a replacement for a general-purpose logging library, OpenTelemetry, or a distributed tracing platform.

Requirements

  • Go 1.22 or newer
  • A Unix-like system or Windows; interprocess coordination uses native kernel file locks on both

The module has no external dependencies.

Installation

go get github.com/golovanov-dev/traceloom-go

Quick start

package main

import (
    "log"

    traceloom "github.com/golovanov-dev/traceloom-go"
)

func main() {
    tracer, err := traceloom.New("./logs")
    if err != nil {
        log.Fatal(err)
    }
    defer tracer.Close()

    trace, err := tracer.Start("")
    if err != nil {
        log.Fatal(err)
    }

    if err := trace.Event("request_start", traceloom.Data{
        "method": "POST",
        "path":   "/orders",
    }); err != nil {
        log.Fatal(err)
    }

    _ = trace.Event("auth_success", traceloom.Data{"user_id": 42})
    _ = trace.Event("request_end", traceloom.Data{"status": 201})
}

Every event written through the same Trace shares one trace_id. A trace is safe to call from multiple goroutines; sequence values advance in the order writes acquire the trace lock.

JSONL output

{"timestamp":"2026-07-10T10:41:20.112345Z","trace_id":"9f1a8e7c2d4b4a9f93e2b2b1454f0c0a","event":"request_start","sequence":1,"elapsed_ms":0.121,"data":{"method":"POST","path":"/orders"}}
{"timestamp":"2026-07-10T10:41:20.117381Z","trace_id":"9f1a8e7c2d4b4a9f93e2b2b1454f0c0a","event":"auth_success","sequence":2,"elapsed_ms":5.157,"data":{"user_id":42}}

Timestamps are UTC with six fractional digits. elapsed_ms uses Go's monotonic clock reading, retained separately from wall-clock time, so NTP and daylight-saving changes do not distort durations.

Continue an existing trace

trace, err := tracer.Start(incomingTraceID)

Valid incoming IDs contain 8–128 characters from A-Z, a-z, 0-9, ., _, -, and :. Invalid or empty IDs are replaced with a cryptographically random 32-character lowercase hex ID.

Configuration

Configuration uses functional options and becomes immutable after validation:

tracer, err := traceloom.New(
    "./logs",
    traceloom.WithMaxFileBytes(50*1024*1024),
    traceloom.WithMaxStringBytes(64*1024),
    traceloom.WithMaxKeyBytes(256),
    traceloom.WithMaxRecordBytes(256*1024),
    traceloom.WithMaxArrayItems(1000),
    traceloom.WithMaxPayloadNodes(10000),
    traceloom.WithMaxDepth(16),
    traceloom.WithSensitiveKeys("payment_token"),
    traceloom.WithStrictSensitiveKeys(false),
    traceloom.WithDirectoryMode(0o750),
    traceloom.WithFileMode(0o640),
    traceloom.WithRetentionDays(0),
    traceloom.WithTrustIncomingTraceID(false),
    traceloom.WithFailOnError(false),
    traceloom.WithOnError(func(err error) {
        log.Printf("tracing: %v", err)
    }),
)

For dependency injection:

configuration, err := traceloom.NewConfiguration("./logs")
tracer, err := traceloom.NewWithDependencies(configuration, writer, clock)

Writer, Clock, TraceEvent, Configuration, and the built-in JSONLFileWriter are public contracts.

maxRecordBytes is clamped to maxFileBytes, and maxStringBytes is clamped to maxRecordBytes.

Context and concurrency

Event is synchronous, as is conventional for a small Go I/O API. Applications already have goroutines when they need concurrent work:

go func() {
    _ = trace.Event("background_step", traceloom.Data{"step": 1})
}()

Use EventContext when lock acquisition should honor request cancellation or a deadline:

err := trace.EventContext(ctx, "database_complete", traceloom.Data{
    "rows": 10,
})

FlushContext is available on both Trace and Tracer. Without a caller deadline, filesystem lock waits are bounded internally.

Files, rotation, and interprocess locking

Files are selected by UTC date and rotated into numbered shards:

logs/
  2026-07-10.jsonl
  2026-07-10-1.jsonl
  2026-07-10-2.jsonl

Traceloom uses kernel file locks on every supported platform, with no dependencies: syscall.Flock on Linux, macOS, and the BSDs, and LockFileEx (resolved from kernel32) on Windows.

  • .traceloom.lock coordinates shard selection and rotation;
  • the active JSONL file is locked for one complete append;
  • the kernel releases a lock when the process holding it dies, so a crashed writer cannot strand one;
  • PHP, Node.js, and Go writers can append to the same directory.

Supported platforms are Unix-like systems and Windows. There is no lock-file fallback: a homemade lock has to guess when an owner has died, and guessing wrong puts two writers in the same file.

WithRetentionDays(n) expires shards older than n days. Retention runs on the first write of each UTC date, so a long-lived process keeps expiring old shards rather than sweeping once at startup.

Directories are created as 0750 and files as 0640 on POSIX systems. Keep trace files outside the web root: payloads can still contain sensitive information after masking.

Sensitive data

Masking is recursive and enabled by default. Keys are folded by case and punctuation, so api_key, apiKey, API-KEY, and X-Api-Key are recognized. Secret fragments are also matched, so spellings such as cookies, authorization_header, and session_token are covered:

password, passwd, secret, token, apikey, privatekey, credential, authorization, cookie, session, bearer, signature, jwt, accesskey.

A fragment is a substring that appears in no ordinary English word, so matching it cannot redact an innocent key. That rules out auth and key, which would swallow author, keyboard, and monkey; those spellings are covered by the exact key list instead, which includes auth, access_key, and secret_key.

Masked values become:

"[REDACTED]"

Set WithStrictSensitiveKeys(true) to disable fragment matching. Custom keys are merged with the defaults.

Masking uses names, including struct JSON field names, not value inspection. Two consequences are worth knowing:

  • A credential stored under an innocent key such as note is not detected, and neither is a secret inside a value: a JWT in a message, a ?token= in a URL, or a password in a DSN.
  • Key folding is ASCII. A key spelled with lookalike Unicode letters (pаssword with a Cyrillic а) does not match.

Payload support

The event root is traceloom.Data, an alias-friendly map[string]any. Nested values can include:

  • maps with string keys;
  • slices and arrays;
  • structs and exported fields with JSON tags;
  • pointers;
  • strings, integers, unsigned integers, floats, booleans, and nil;
  • json.Marshaler, encoding.TextMarshaler, and fmt.Stringer;
  • time.Time;
  • byte slices and byte arrays.

Reflection traversal is cycle-aware and bounded by two independent limits: maxArrayItems caps each array on its own, so one long array does not starve the fields beside it, while maxPayloadNodes bounds the payload as a whole and stops a wide or deeply nested value from costing unbounded work.

Keys are bounded too. A key longer than maxKeyBytes is truncated and given a digest of the original, so verylongkey…~1f0a2b3c4d5e6f70 stays unique: two long keys cannot collapse into one and silently overwrite each other's value. Without this, a single oversized key is enough to push a record past maxRecordBytes and degrade the whole event.

Marker Cause
{ "_truncated": true, ... } UTF-8 string exceeds maxStringBytes
{ "_binary": true, ... } Byte data or an invalid UTF-8 string
[CIRCULAR_REFERENCE] A map, slice, or pointer refers back to itself
[MAX_DEPTH_EXCEEDED] Nesting is deeper than maxDepth
[SERIALIZATION_FAILED: Type] A marshaler or stringer errors or panics
[UNSUPPORTED_TYPE: type] Function, channel, complex number, or other unsupported value
{ "_omitted_items": N } Entries exceed maxArrayItems or maxPayloadNodes

A payload key that spells one of these markers is escaped one underscore deeper: a payload key _truncated is recorded as __truncated. A record can therefore neither forge a marker nor overwrite one.

Non-finite floats are stored as strings. If a complete record cannot be encoded or exceeds maxRecordBytes, its data becomes { "_encoding_error": "..." }, preserving the event in the timeline.

A String() or MarshalJSON() that panics is contained and reported as [SERIALIZATION_FAILED: Type]. One that never returns — an infinite loop, or a String() that recurses into itself — cannot be contained, so payload types are expected to terminate.

Trusting incoming trace IDs

An incoming ID is not trusted by default. A caller cannot inject events into an existing timeline: Traceloom generates a fresh trace_id and keeps the accepted incoming value as parent_trace_id.

trace, err := tracer.Start(request.Header.Get("X-Trace-Id"))

Behind a trusted gateway or a service mesh, where the inbound ID is not client-controlled, continue the incoming trace instead:

tracer, err := traceloom.New(
    "./logs",
    traceloom.WithTrustIncomingTraceID(true),
)

HTTP integration

func handler(response http.ResponseWriter, request *http.Request) {
    trace, err := tracer.Start(request.Header.Get("X-Trace-Id"))
    if err != nil {
        http.Error(response, "trace unavailable", http.StatusInternalServerError)
        return
    }

    _ = trace.EventContext(request.Context(), "request_start", traceloom.Data{
        "method": request.Method,
        "path":   request.URL.Path,
    })

    response.Header().Set("X-Trace-Id", trace.ID())
    response.WriteHeader(http.StatusOK)

    _ = trace.EventContext(request.Context(), "request_end", traceloom.Data{
        "status": http.StatusOK,
    })
}

Framework-specific middleware is outside the core module.

Error handling

Runtime tracing failures are fail-safe by default. They increment DroppedEventCount() and optionally call OnError, but Event returns nil so tracing does not break the host application.

A dropped event still consumes its sequence number, so a failed write leaves a gap in the log rather than being renumbered away. A reader of a timeline that jumps from sequence 4 to 6 can see that an event is missing.

Configuration errors, trace-ID generation failures, and invalid event names always return errors. Set WithFailOnError(true) in tests or strict development environments to return I/O and payload-write failures as well.

if tracer.DroppedEventCount() > 0 {
    // The timeline has lost events.
}

OnError panics are recovered so an observability callback cannot crash the application.

Flush and close

Flush calls fsync on the active shard. Ordinary event writes use the operating system's file buffers without an fsync per line.

Close releases the active handle and is terminal: an event written after it is rejected and counted by DroppedEventCount() rather than reopening the shard. It is safe to call more than once.

CLI

Install the command:

go install github.com/golovanov-dev/traceloom-go/cmd/eventtrace@latest

Show a timeline:

eventtrace show 9f1a8e7c2d4b4a9f93e2b2b1454f0c0a --dir=logs

Example:

Trace: 9f1a8e7c2d4b4a9f93e2b2b1454f0c0a
10:41:20.112 request_start
10:41:20.117 auth_success +5.036 ms
Total duration: 5.157 ms

During development:

go run ./cmd/eventtrace show <trace-id> --dir=logs

The CLI streams arbitrarily long JSONL records, escapes terminal control bytes, and aggregates malformed-line warnings.

Development

go test ./...
go test -race ./...
go vet ./...
go build ./cmd/eventtrace
go run ./benchmarks/event 1000

Tests cover payload normalization, goroutine safety, multi-process append and rotation, CLI hardening, retention, strict/fail-safe behavior, and distribution-compatible examples.

When not to use Traceloom

Use a general-purpose logger when you need levels, handlers, and broad logging integrations. Use OpenTelemetry for standard distributed tracing. Use an observability platform for aggregation, dashboards, alerting, or cross-service querying.

Traceloom intentionally remains small: local structured timeline tracing without external infrastructure.

Documentation

Overview

Package traceloom records structured event timelines grouped by trace ID.

It is a small, dependency-free local tracing library rather than a general logger or a distributed tracing implementation.

Index

Constants

View Source 
const (
	// DefaultMaxFileBytes is the soft size limit for one JSONL shard.
	DefaultMaxFileBytes int64 = 50 * 1024 * 1024
	// DefaultMaxStringBytes is the UTF-8 byte limit for one payload string.
	DefaultMaxStringBytes = 64 * 1024
	// DefaultMaxKeyBytes is the UTF-8 byte limit for one payload key.
	DefaultMaxKeyBytes = 256

	// DefaultMaxRecordBytes is the byte limit for one JSONL record.
	DefaultMaxRecordBytes = 256 * 1024
	// DefaultMaxArrayItems is the item limit applied to each array on its own.
	DefaultMaxArrayItems = 1000
	// DefaultMaxPayloadNodes is the node budget shared by an event payload as a whole.
	DefaultMaxPayloadNodes = 10000
	// DefaultMaxDepth limits recursive payload traversal.
	DefaultMaxDepth = 16
	// DefaultDirectoryMode is used for newly created log directories on POSIX.
	DefaultDirectoryMode fs.FileMode = 0o750
	// DefaultFileMode is used for newly created log shards on POSIX.
	DefaultFileMode fs.FileMode = 0o640
	// Redacted replaces values stored under sensitive keys.
	Redacted = "[REDACTED]"
)
View Source 
const (
	MaxDepthExceeded  = "[MAX_DEPTH_EXCEEDED]"
	CircularReference = "[CIRCULAR_REFERENCE]"
)

Variables

This section is empty.

Functions

func CanonicalizeKey 

func CanonicalizeKey(key string) string

CanonicalizeKey folds case and punctuation for secret-key matching.

func GenerateTraceID 

func GenerateTraceID() (string, error)

GenerateTraceID returns a cryptographically random 32-character lowercase hex ID.

func SanitizeTraceID 

func SanitizeTraceID(traceID string) (string, bool)

SanitizeTraceID returns the accepted ID and true, or an empty string and false.

Types

type Clock 

type Clock interface {
	Now() time.Time
	Monotonic() time.Duration
}

Clock separates wall-clock timestamps from monotonic duration readings.

type Configuration 

type Configuration struct {
	// contains filtered or unexported fields
}

Configuration is validated and immutable after construction.

func NewConfiguration 

func NewConfiguration(logDirectory string, options ...Option) (Configuration, error)

NewConfiguration validates options and returns an immutable configuration.

func (Configuration) DirectoryMode 

func (c Configuration) DirectoryMode() fs.FileMode

func (Configuration) FailOnError 

func (c Configuration) FailOnError() bool

func (Configuration) FileMode 

func (c Configuration) FileMode() fs.FileMode

func (Configuration) LogDirectory 

func (c Configuration) LogDirectory() string

func (Configuration) MaxArrayItems 

func (c Configuration) MaxArrayItems() int

func (Configuration) MaxDepth 

func (c Configuration) MaxDepth() int

func (Configuration) MaxFileBytes 

func (c Configuration) MaxFileBytes() int64

func (Configuration) MaxKeyBytes 

func (c Configuration) MaxKeyBytes() int

func (Configuration) MaxPayloadNodes 

func (c Configuration) MaxPayloadNodes() int

func (Configuration) MaxRecordBytes 

func (c Configuration) MaxRecordBytes() int

func (Configuration) MaxStringBytes 

func (c Configuration) MaxStringBytes() int

func (Configuration) RetentionDays 

func (c Configuration) RetentionDays() int

func (Configuration) SensitiveKeys 

func (c Configuration) SensitiveKeys() []string

func (Configuration) StrictSensitiveKeys 

func (c Configuration) StrictSensitiveKeys() bool

func (Configuration) TrustIncomingTraceID 

func (c Configuration) TrustIncomingTraceID() bool

type ConfigurationError 

type ConfigurationError struct {
	Message string
}

ConfigurationError reports an invalid immutable configuration.

func (*ConfigurationError) Error 

func (e *ConfigurationError) Error() string

type Data 

type Data map[string]any

Data is one event's structured payload.

type JSONLFileWriter 

type JSONLFileWriter struct {
	// contains filtered or unexported fields
}

JSONLFileWriter appends events to UTC date shards with size rotation.

func NewJSONLFileWriter 

func NewJSONLFileWriter(configuration Configuration) *JSONLFileWriter

func (*JSONLFileWriter) Close 

func (writer *JSONLFileWriter) Close() error

Close releases the active shard. It is terminal and idempotent: a later write is rejected instead of reopening the file.

func (*JSONLFileWriter) Flush 

func (writer *JSONLFileWriter) Flush(ctx context.Context) error

func (*JSONLFileWriter) Write 

func (writer *JSONLFileWriter) Write(ctx context.Context, event TraceEvent) (err error)

type Option 

type Option func(*configValues)

Option customizes an immutable Configuration.

func WithDirectoryMode 

func WithDirectoryMode(value fs.FileMode) Option

WithDirectoryMode sets permissions for newly created log directories.

func WithFailOnError 

func WithFailOnError(value bool) Option

WithFailOnError enables strict runtime tracing errors when true.

func WithFileMode 

func WithFileMode(value fs.FileMode) Option

WithFileMode sets permissions for newly created JSONL shards.

func WithMaxArrayItems 

func WithMaxArrayItems(value int) Option

WithMaxArrayItems caps each array in a payload on its own, so one long array does not starve the fields beside it.

func WithMaxDepth 

func WithMaxDepth(value int) Option

WithMaxDepth sets the recursive payload depth limit.

func WithMaxFileBytes 

func WithMaxFileBytes(value int64) Option

WithMaxFileBytes sets the soft size limit for one shard.

func WithMaxKeyBytes 

func WithMaxKeyBytes(value int) Option

WithMaxKeyBytes sets the UTF-8 byte limit for one payload key. A longer key is truncated and given a digest suffix, so two long keys cannot collide into one.

func WithMaxPayloadNodes 

func WithMaxPayloadNodes(value int) Option

WithMaxPayloadNodes bounds a payload as a whole, which stops a wide or deeply nested value from costing an unbounded amount of work.

func WithMaxRecordBytes 

func WithMaxRecordBytes(value int) Option

WithMaxRecordBytes sets the byte limit for one encoded JSONL record.

func WithMaxStringBytes 

func WithMaxStringBytes(value int) Option

WithMaxStringBytes sets the UTF-8 byte limit for one payload string.

func WithOnError 

func WithOnError(handler func(error)) Option

WithOnError observes fail-safe runtime errors. Handler panics are recovered.

func WithRetentionDays 

func WithRetentionDays(value int) Option

WithRetentionDays removes older date shards during the writer's first rotation.

func WithSensitiveKeys 

func WithSensitiveKeys(keys ...string) Option

WithSensitiveKeys merges custom secret keys with the built-in defaults.

func WithStrictSensitiveKeys 

func WithStrictSensitiveKeys(value bool) Option

WithStrictSensitiveKeys disables secret-fragment matching when true.

func WithTrustIncomingTraceID 

func WithTrustIncomingTraceID(value bool) Option

WithTrustIncomingTraceID controls whether accepted inbound IDs are reused.

type Sanitizer 

type Sanitizer struct {
	// contains filtered or unexported fields
}

Sanitizer normalizes one event payload without retaining per-call mutable state. It is safe for concurrent use.

func NewSanitizer 

func NewSanitizer(configuration Configuration) *Sanitizer

func (*Sanitizer) Sanitize 

func (sanitizer *Sanitizer) Sanitize(data Data) (result Data)

type Trace 

type Trace struct {
	// contains filtered or unexported fields
}

Trace groups a thread-safe sequence of events under one trace ID.

func (*Trace) Event 

func (trace *Trace) Event(name string, data Data) error

func (*Trace) EventContext 

func (trace *Trace) EventContext(ctx context.Context, name string, data Data) error

func (*Trace) Flush 

func (trace *Trace) Flush() error

func (*Trace) FlushContext 

func (trace *Trace) FlushContext(ctx context.Context) error

func (*Trace) ID 

func (trace *Trace) ID() string

func (*Trace) ParentID 

func (trace *Trace) ParentID() string

type TraceEvent 

type TraceEvent struct {
	Timestamp     time.Time
	TraceID       string
	ParentTraceID string
	Name          string
	Sequence      uint64
	Elapsed       time.Duration
	Data          Data
}

TraceEvent is the public storage contract passed to Writer implementations.

type Tracer 

type Tracer struct {
	// contains filtered or unexported fields
}

Tracer owns shared configuration, writer, sanitizer, and dropped-event metrics.

func New 

func New(logDirectory string, options ...Option) (*Tracer, error)

New creates a tracer writing JSONL files into logDirectory.

func NewFromConfiguration 

func NewFromConfiguration(configuration Configuration) (*Tracer, error)

NewFromConfiguration creates a tracer with the built-in JSONL writer.

func NewWithDependencies 

func NewWithDependencies(configuration Configuration, writer Writer, clock Clock) (*Tracer, error)

NewWithDependencies creates a tracer with a custom writer and clock.

func (*Tracer) Close 

func (tracer *Tracer) Close() error

func (*Tracer) DroppedEventCount 

func (tracer *Tracer) DroppedEventCount() uint64

func (*Tracer) Flush 

func (tracer *Tracer) Flush() error

func (*Tracer) FlushContext 

func (tracer *Tracer) FlushContext(ctx context.Context) error

func (*Tracer) Start 

func (tracer *Tracer) Start(incomingTraceID string) (*Trace, error)

Start creates a trace, optionally continuing a validated incoming ID.

type TracingError 

type TracingError struct {
	Message string
	Err     error
}

TracingError reports a runtime tracing failure.

func (*TracingError) Error 

func (e *TracingError) Error() string

func (*TracingError) Unwrap 

func (e *TracingError) Unwrap() error

type Writer 

type Writer interface {
	Write(context.Context, TraceEvent) error
	Flush(context.Context) error
	Close() error
}

Writer persists normalized trace events.

Source Files

View all Source files

Directories

Path Synopsis
benchmarks
event command
cmd
eventtrace command
examples
basic command
http-api command
internal
eventtracecli

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.