loki

Documentation

Overview

Package loki provides a Grafana Loki output for the go-audit library.

The output pushes audit events to a Loki instance via the HTTP Push API (POST /loki/api/v1/push). Events are batched and delivered as JSON-encoded push requests with configurable stream labels, gzip compression, multi-tenant support, and TLS.

Stream Labels

Users control which fields become Loki stream labels via the Labels configuration. Static labels (e.g., job, environment) are constant across all events. Dynamic labels (event_type, severity, event_category, app_name, host, pid) are derived per-event from audit.EventMetadata via the audit.MetadataWriter interface.

Custom user-defined fields are never labels — they stay in the log line and are queryable via LogQL JSON parsing:

{event_type="auth_failure"} | json | actor_id="alice"

Batching

Events are buffered internally and flushed when the batch reaches [Config.BatchSize] events, [Config.MaxBatchBytes] total payload bytes, or [Config.FlushInterval] elapses — whichever comes first. The Output.Close method flushes any remaining events.

Import

Import this package for its side effect of registering the "loki" output type with the audit output registry:

import _ "github.com/axonops/go-audit/loki"

Index

• Constants
• func NewFactory(lokiMetrics Metrics) audit.OutputFactory
• type BasicAuth
  • func (ba BasicAuth) GoString() string
  • func (ba BasicAuth) String() string
• type Config
  • func (c Config) Format(f fmt.State, _ rune)
  • func (c Config) String() string
• type DynamicLabels
• type LabelConfig
• type Metrics
• type Output
  • func New(cfg *Config, metrics audit.Metrics, lokiMetrics Metrics) (*Output, error)
  • func (o *Output) Close() error
  • func (o *Output) DestinationKey() string
  • func (o *Output) Name() string
  • func (o *Output) ReportsDelivery() bool
  • func (o *Output) SetFrameworkFields(appName, host, timezone string, pid int)
  • func (o *Output) Write(data []byte) error
  • func (o *Output) WriteWithMetadata(data []byte, meta audit.EventMetadata) error

Examples

• New
• NewFactory

Constants

const (
	// DefaultBatchSize is the default maximum events per push request.
	DefaultBatchSize = 100

	// DefaultMaxBatchBytes is the default maximum uncompressed push
	// payload size in bytes (1 MiB).
	DefaultMaxBatchBytes = 1 << 20

	// DefaultFlushInterval is the default maximum time between push
	// requests when the batch has not yet reached [DefaultBatchSize].
	DefaultFlushInterval = 5 * time.Second

	// DefaultTimeout is the default HTTP request timeout.
	DefaultTimeout = 10 * time.Second

	// DefaultMaxRetries is the default retry count for 429/5xx.
	DefaultMaxRetries = 3

	// DefaultBufferSize is the default internal event buffer capacity.
	DefaultBufferSize = 10_000
)

Default values for Config fields.

const (
	// MaxBatchSize is the upper bound for [Config.BatchSize].
	MaxBatchSize = 10_000

	// MaxMaxBatchBytes is the upper bound for [Config.MaxBatchBytes] (10 MiB).
	MaxMaxBatchBytes = 10 << 20

	// MaxFlushInterval is the upper bound for [Config.FlushInterval].
	MaxFlushInterval = 5 * time.Minute

	// MaxTimeout is the upper bound for [Config.Timeout].
	MaxTimeout = 5 * time.Minute

	// MaxMaxRetries is the upper bound for [Config.MaxRetries].
	MaxMaxRetries = 20

	// MaxBufferSize is the upper bound for [Config.BufferSize].
	MaxBufferSize = 1_000_000
)

Upper bounds for Config fields.

const (
	// MinMaxBatchBytes is the lower bound for [Config.MaxBatchBytes] (1 KiB).
	MinMaxBatchBytes = 1024

	// MinFlushInterval is the lower bound for [Config.FlushInterval].
	MinFlushInterval = 100 * time.Millisecond

	// MinTimeout is the lower bound for [Config.Timeout].
	MinTimeout = 1 * time.Second

	// MinBufferSize is the lower bound for [Config.BufferSize].
	MinBufferSize = 100
)

Lower bounds for Config fields.

Functions

func NewFactory

func NewFactory(lokiMetrics Metrics) audit.OutputFactory

NewFactory returns an audit.OutputFactory that creates Loki outputs from YAML configuration with the provided Loki-specific metrics captured in the closure. Pass nil to disable Loki metrics.

Example

ExampleNewFactory demonstrates registering a Loki output factory with custom Loki-specific metrics.

package main

import (
	"fmt"

	audit "github.com/axonops/go-audit"
	"github.com/axonops/go-audit/loki"
)

func main() {
	// Create a factory with custom metrics (pass nil to disable).
	factory := loki.NewFactory(nil)

	// The factory can be registered with the audit output registry:
	audit.RegisterOutputFactory("loki-custom", factory)

	fmt.Println("factory registered")
}

Output:
factory registered

Types

type BasicAuth

type BasicAuth struct {
	// Username is the HTTP basic auth username. REQUIRED when BasicAuth
	// is non-nil; an empty value causes [New] to return an error.
	Username string

	// Password is the HTTP basic auth password. MAY be empty when the
	// Loki endpoint accepts username-only authentication.
	Password string
}

BasicAuth holds HTTP basic authentication credentials for Loki push requests. Mutually exclusive with [Config.BearerToken]. The library redacts credentials in all log output and string representations.

func (BasicAuth) GoString

func (ba BasicAuth) GoString() string

GoString implements fmt.GoStringer to prevent credential leakage via %#v.

func (BasicAuth) String

func (ba BasicAuth) String() string

String returns a redacted representation to prevent credential leakage.

type Config

type Config struct {
	// URL is the full Loki push API endpoint, including path.
	// Example: "https://loki:3100/loki/api/v1/push"
	// REQUIRED; must be https unless AllowInsecureHTTP is true.
	URL string

	// BasicAuth configures HTTP basic authentication. Mutually
	// exclusive with BearerToken.
	BasicAuth *BasicAuth

	// BearerToken sets the Authorization: Bearer header. Mutually
	// exclusive with BasicAuth.
	BearerToken string

	// TenantID sets the X-Scope-OrgID header for Loki multi-tenancy.
	TenantID string

	// Headers are additional HTTP headers sent with every push request.
	Headers map[string]string

	// Labels controls stream label configuration.
	Labels LabelConfig

	// TLSCA is the path to a PEM-encoded CA certificate used to verify
	// the Loki server certificate. When empty, the system root CA pool
	// is used. Use this field when Loki is configured with a private or
	// self-signed CA.
	TLSCA string

	// TLSCert is the path to a PEM-encoded client certificate for mTLS
	// authentication. MUST be set together with [Config.TLSKey]; setting
	// one without the other causes [New] to return an error.
	TLSCert string

	// TLSKey is the path to the PEM-encoded private key for the client
	// certificate. MUST be set together with [Config.TLSCert]; setting
	// one without the other causes [New] to return an error.
	TLSKey string

	// TLSPolicy controls the TLS version and cipher suite policy applied
	// to all Loki connections. When nil, the default policy (TLS 1.3
	// only) is used. See [audit.TLSPolicy] for details on enabling TLS
	// 1.2 fallback.
	TLSPolicy *audit.TLSPolicy

	// BatchSize is the maximum events per push request.
	// Zero defaults to [DefaultBatchSize] (100).
	// Values above [MaxBatchSize] (10,000) are rejected.
	BatchSize int

	// MaxBatchBytes is the maximum uncompressed payload size per push
	// request. Zero defaults to [DefaultMaxBatchBytes] (1 MiB).
	// Valid range: [MinMaxBatchBytes] (1 KiB) to [MaxMaxBatchBytes] (10 MiB).
	MaxBatchBytes int

	// FlushInterval is the maximum time between push requests when the
	// batch has not yet reached BatchSize. The timer resets after
	// every flush. Zero defaults to [DefaultFlushInterval] (5s).
	FlushInterval time.Duration

	// BufferSize is the internal async event buffer capacity. When full,
	// new events are dropped. Zero defaults to [DefaultBufferSize]
	// (10,000).
	BufferSize int

	// Timeout is the HTTP request timeout covering the full push
	// request/response lifecycle. Zero defaults to [DefaultTimeout]
	// (10s).
	Timeout time.Duration

	// MaxRetries is the retry count for 429 and 5xx responses.
	// Zero defaults to [DefaultMaxRetries] (3).
	MaxRetries int

	// Compress enables gzip compression of push requests. The YAML
	// factory defaults to true when the gzip key is omitted; the Go
	// zero value is false for programmatic construction.
	Compress bool

	// AllowInsecureHTTP permits http:// URLs. MUST NOT be true in
	// production.
	AllowInsecureHTTP bool

	// AllowPrivateRanges permits connections to RFC 1918 private
	// addresses. Intended for testing and private deployments.
	AllowPrivateRanges bool
}

Config holds configuration for the Loki Output.

func (Config) Format

func (c Config) Format(f fmt.State, _ rune)

Format implements fmt.Formatter to prevent credential leakage via all format verbs including %+v and %#v. Value receiver ensures both Config and *Config are protected.

func (Config) String

func (c Config) String() string

String returns a safe representation of the config without credentials. Value receiver ensures both Config and *Config are protected.

type DynamicLabels

type DynamicLabels struct {
	ExcludeAppName       bool
	ExcludeHost          bool
	ExcludeTimezone      bool
	ExcludePID           bool
	ExcludeEventType     bool
	ExcludeEventCategory bool
	ExcludeSeverity      bool
}

DynamicLabels toggles which per-event fields become Loki stream labels. All fields default to included (zero value = include). Set Exclude* to true to remove a field from labels.

type LabelConfig

type LabelConfig struct {
	// Static labels are constant across all events from this output.
	// Keys must match [a-zA-Z_][a-zA-Z0-9_]* (Loki requirement).
	Static map[string]string

	// Dynamic controls which per-event fields become labels.
	// Zero value means all dynamic labels are included.
	Dynamic DynamicLabels
}

LabelConfig controls which fields become Loki stream labels.

type Metrics

type Metrics interface {
	// RecordLokiDrop is called when an event is dropped (buffer full
	// or retries exhausted).
	RecordLokiDrop()

	// RecordLokiFlush is called after each successful push to Loki.
	RecordLokiFlush(batchSize int, dur time.Duration)

	// RecordLokiRetry is called when a push is retried after a 429
	// or 5xx response.
	RecordLokiRetry(statusCode int, attempt int)

	// RecordLokiError is called when a push fails with a non-retryable
	// error (4xx except 429) or after all retries are exhausted.
	RecordLokiError(statusCode int)
}

Metrics is an optional interface for recording Loki output operational metrics. Pass nil to disable metrics collection.

type Output

type Output struct {
	// contains filtered or unexported fields
}

Output pushes audit events to a Grafana Loki instance via the HTTP Push API. It implements audit.Output, audit.MetadataWriter, audit.DeliveryReporter, audit.DestinationKeyer, and audit.FrameworkFieldReceiver.

Events are buffered and flushed in batches based on count, byte size, or time interval — whichever threshold is reached first.

func New

func New(cfg *Config, metrics audit.Metrics, lokiMetrics Metrics) (*Output, error)

New creates a new Loki Output from the given config. It validates the config, builds an SSRF-safe HTTP client, and starts the background batch goroutine. Both metrics parameters are optional (may be nil).

Example

ExampleNew demonstrates creating a Loki output with stream labels and gzip compression.

package main

import (
	"fmt"
	"time"

	"github.com/axonops/go-audit/loki"
)

func main() {
	cfg := &loki.Config{
		URL:                "http://localhost:3100/loki/api/v1/push",
		AllowInsecureHTTP:  true, // local dev only
		AllowPrivateRanges: true, // local dev only
		BatchSize:          100,
		FlushInterval:      5 * time.Second,
		Timeout:            10 * time.Second,
		MaxRetries:         3,
		BufferSize:         10000,
		Compress:           true,
		Labels: loki.LabelConfig{
			Static: map[string]string{
				"job":         "audit",
				"environment": "development",
			},
		},
	}

	out, err := loki.New(cfg, nil, nil)
	if err != nil {
		fmt.Printf("error: %v\n", err)
		return
	}
	defer func() { _ = out.Close() }()

	fmt.Println(out.Name())
	fmt.Println(out.ReportsDelivery())
}

Output:
loki:localhost:3100
true

func (*Output) Close

func (o *Output) Close() error

Close signals the batch goroutine to drain and flush, then waits for completion. In-flight HTTP retries are cancelled via context. Close is idempotent.

func (*Output) DestinationKey

func (o *Output) DestinationKey() string

DestinationKey returns the Loki URL with query parameters and fragment stripped, enabling duplicate destination detection via audit.DestinationKeyer.

func (*Output) Name

func (o *Output) Name() string

Name returns the human-readable identifier for this output.

func (*Output) ReportsDelivery

func (o *Output) ReportsDelivery() bool

ReportsDelivery returns true, indicating that Output reports its own delivery metrics from the batch goroutine after actual HTTP delivery, not from the Write enqueue path.

func (*Output) SetFrameworkFields

func (o *Output) SetFrameworkFields(appName, host, timezone string, pid int)

SetFrameworkFields receives logger-wide framework metadata for use as Loki stream labels. Called once by the core library at logger construction time. The data is stored atomically, safe for concurrent access from the batchLoop goroutine.

func (*Output) Write

func (o *Output) Write(data []byte) error

Write enqueues a serialised audit event without metadata. When called directly (outside the core library's MetadataWriter dispatch), events are delivered with framework-only stream labels and no per-event dynamic labels. Prefer WriteWithMetadata.

func (*Output) WriteWithMetadata

func (o *Output) WriteWithMetadata(data []byte, meta audit.EventMetadata) error

WriteWithMetadata enqueues a serialised audit event with per-event metadata for batched delivery. The data is copied before enqueuing. If the internal buffer is full, the event is dropped and [Metrics.RecordLokiDrop] is called. WriteWithMetadata never blocks.