resolog

README

resolog

A resource-aware CloudWatch log tailer. resolog = resolve + log.

日本語 README

Most tools answer "how do I tail a log group?" (aws logs tail, lucagrulla/cw, StartLiveTail). resolog answers the question that comes first: given a resource, what should I even tail? — then interleaves every stream it finds, docker-compose style.

Main use: hand it a Step Functions execution ARN and it tails the state machine plus every Lambda, Batch, and ECS task that execution ran, all together.

Install

go install github.com/tawAsh1/resolog/cmd/resolog@latest

Or download a binary from Releases. They carry build-provenance attestations:

gh attestation verify resolog_*.tar.gz --repo tawAsh1/resolog

Usage

The real backends use the standard AWS credential chain. The default backend is live (StartLiveTail).

resolog log-group:/aws/lambda/my-fn                 # real-time tail
resolog --backend poll -f log-group:/my/group       # historical, then follow
resolog --backend poll --since 1h --sort -t sfn-execution:<execution-arn>   # a finished run, in time order
resolog --backend poll --since "2026-07-05 09:00" --until "2026-07-05 10:00" --filter '?ERROR ?WARN' log-group:/my/group   # a window, server-side filtered
resolog --output json log-group:/my/group | jq .message               # machine-readable, one JSON object per line
resolog --fields level,msg log-group:/my/group                        # structured logs: show just these fields

resolog arn:aws:ecs:us-east-1:123:task/prod/abc123  # paste a raw ARN, no scheme needed

resolog ls sfn-execution <state-machine-arn>        # list executions, pick one
resolog ls batch-job <queue>
resolog ls ecs-task <cluster>
resolog ls log-group /aws/lambda/

resolog pick --backend poll --sort sfn-execution <state-machine-arn>   # list, choose, tail in one step

pick shows the same listing as ls, lets you choose one entry interactively (through fzf when it is installed, else a numbered prompt), and tails it — tail flags come first, then <scheme> [filter].

References are a raw resource ARN, <scheme>:<rest>, or a bare log group name. A raw ARN dispatches by its service, so you can paste one as-is. Schemes: log-group, sfn-execution, sfn-state-machine (tails its most recent execution), batch-job, batch-queue (every active job in a queue), lambda, ecs-task (one stream per container), ecs-service (every task in a service), codebuild (a build id/ARN or a project name), cfn-stack (everything a CloudFormation stack logs).

Flag	Description
--backend live|poll	live (StartLiveTail, default) or poll (FilterLogEvents, historical)
-f	keep polling for new events (poll backend)
--since	only fetch events at/after this time: a duration ago (10m, 1h30m) or an absolute time (RFC3339, 2026-07-05 09:00, 15:04)
--until	only fetch events at/before this time; pair with --since for a window. Same forms as --since
--filter	CloudWatch Logs filter pattern, applied server-side (both backends)
--grep	client-side regexp; only events whose message matches are shown
--output text|json	json writes one compact JSON object per line ({"time","group","stream","label","message"}); -t/--no-color are ignored there
--fields	comma-separated JSON fields (dot paths like log.level) to show instead of the full message; text output only. Non-JSON lines pass through unchanged
--sort	poll only; buffer and print in time order across streams (see Ordering)
-t	show timestamps
--no-color	disable colored output

--filter and --grep compose: --filter narrows what CloudWatch sends over the wire, --grep further narrows what gets printed. With --fields, --grep matches the extracted view — what you see is what it greps.

Ordering

By default lines print in arrival order — interleaved across sources, like docker compose logs. Across different streams this is not time order.

--sort (poll, finished resources only) buffers everything and prints in time order by each resource's own reported clock. Honest caveats:

• Clocks differ across resources; resolog never claims causal order between them.
• CloudWatch ingestion lags, so a finished task's last lines (e.g. a failure stack trace) can land late and fall outside the window.
• A whole-log-group source (e.g. a Lambda tailed by group) can include lines from other invocations within the window.
• --sort waits for the fetch to finish before printing; on Ctrl-C it flushes the ordered prefix it has so far.
• --sort holds everything in memory; it errors past --sort-max events (default 1,000,000) rather than risk running out of memory. Narrow with --since/--until.

Live output — the newest lines as they arrive — is intentionally never reordered. Without --sort, streaming uses a fixed amount of memory (a page at a time) no matter how much you tail.

Library

resolog is also a Go library; the CLI is just one consumer. See the package docs.

res, _ := sfn.New(sfnClient, sfn.WithBatchResolver(batch.New(batchClient))).
	Resolve(ctx, executionARN)
sink := resolog.NewRenderer(os.Stdout, true, false)
resolog.Tail(ctx, res, livetail.New(logsClient), sink)

Integration tests

Unit tests use fakes; integration/ (build tag integration) exercises the loggroup + poll backend and the sfn resolver against a real-ish AWS API via LocalStack (community edition). Run them with:

make integration-test   # spins up LocalStack in Docker, runs the tests, tears down

They're skipped by plain go test ./... and require Docker; see integration/*_test.go for what's covered.

Status

v0, still young, but now exercised against real production AWS workloads, not just unit tests. Extracted from batchkoi's log tailer.

License

MIT

Documentation

Overview

Package resolog is a resource-aware CloudWatch log tailer.

Most tools answer "how do I tail a log group?" (lucagrulla/cw, aws logs tail, StartLiveTail). resolog answers the question that comes before that: "given a resource, what should I even tail?" — and then interleaves every stream it discovers, docker-compose style.

The design has three orthogonal seams. A consumer can plug in at any of them; nothing forces data through the whole stack.

Resolver  resource reference -> log sources (+ a terminal signal)
Backend   a log source       -> a stream of events
Sink      a stream of events -> output (the default Sink is a TUI renderer)

The primary Resolver is sfn-execution: hand it a Step Functions execution ARN and it resolves the state machine plus every Lambda / Batch / ECS task it ran, and tails them all together.

This package (the module root) is the core: the interface contracts and the Tail orchestrator that wires a Resolution through a Backend into a Sink. Resolvers and Backends live in subpackages and are composed explicitly by the consumer — there is no global registry and no plugin system. See cmd/resolog for the reference wiring, including the scheme dispatch table.

Index

• Constants
• func Tail(ctx context.Context, res Resolution, backend Backend, sink Sink, ...) error
• type Backend
• type Event
• type FilterSink
  • func (f FilterSink) Consume(ctx context.Context, events <-chan Event) error
• type JSONSink
  • func (s JSONSink) Consume(ctx context.Context, events <-chan Event) error
• type Lister
• type MapSink
  • func (m MapSink) Consume(ctx context.Context, events <-chan Event) error
• type Option
  • func WithErrorHandler(f func(Source, error)) Option
  • func WithGracePeriod(d time.Duration) Option
• type Renderer
  • func NewRenderer(out io.Writer, color, showTime bool) *Renderer
  • func (r *Renderer) Consume(ctx context.Context, events <-chan Event) error
• type Resolution
• type Resolver
• type ResourceRef
• type Sink
• type SortingSink
  • func (s SortingSink) Consume(ctx context.Context, events <-chan Event) error
• type Source

Constants

const DefaultGracePeriod = 5 * time.Second

DefaultGracePeriod is how long Tail keeps streaming after a Resolution reports Done, to catch log lines CloudWatch delivers late. See Resolution.Done.

Functions

func Tail

func Tail(ctx context.Context, res Resolution, backend Backend, sink Sink, opts ...Option) error

Tail is the core orchestrator. It reads sources out of res as they are discovered, opens a Backend stream for each, merges everything into one interleaved channel, and feeds that to sink. It returns when:

• all sources have ended (historical backends drain naturally), or
• res.Done fires and the grace period elapses (terminal resource), or
• ctx is cancelled (Ctrl-C), or
• sink.Consume returns.

Completion is driven by res.Done (resource status), never by inferring that "the logs stopped" — that inference is wrong often enough to be useless.

Types

type Backend

type Backend interface {
	// Stream returns a channel of Events for src. The channel is closed when
	// the stream ends (historical exhausted, or ctx cancelled). A live backend
	// only ends on ctx cancellation.
	Stream(ctx context.Context, src Source) (<-chan Event, error)
}

Backend turns a Source into a stream of Events. Implementations: livetail (StartLiveTail, real-time) and poll (FilterLogEvents, historical).

type Event

type Event struct {
	Source    Source
	Timestamp time.Time // the event time reported by CloudWatch
	Message   string
	Ingestion time.Time // when CloudWatch ingested it (may lag Timestamp)
}

Event is one log line, tagged with the Source it came from.

type FilterSink

type FilterSink struct {
	Inner Sink
	Keep  func(Event) bool
}

FilterSink wraps another Sink and forwards only events Keep accepts. It is the client-side counterpart to a Backend's server-side filter pattern (e.g. --grep vs. --filter in the CLI): every event still flows through the pipeline, but only the ones Keep returns true for reach Inner.

Composition order matters when FilterSink wraps a SortingSink: put FilterSink on the outside (Tail's sink = FilterSink{Inner: sortingSink, ...}) so dropped events never occupy space in the sort buffer, rather than the other way around.

func (FilterSink) Consume

func (f FilterSink) Consume(ctx context.Context, events <-chan Event) error

Consume implements Sink. It mirrors SortingSink.Consume's discipline: relay the kept subset of events to Inner via an intermediate channel that this method owns and closes, then wait for Inner.Consume to return.

On ctx cancellation, Consume still closes out and waits for Inner to unwind (as SortingSink waits for its flush) before returning, but reports ctx.Err() rather than whatever Inner returned — cancellation is the authoritative reason Consume is ending.

type JSONSink

type JSONSink struct {
	// W is where lines are written. Must be non-nil.
	W io.Writer
}

JSONSink is a Sink that writes one compact JSON object per event, one per line (JSON Lines) — the machine-readable counterpart to the Renderer, for piping into jq and friends. It is the canonical "supply your own Sink" example the Sink contract mentions.

The field order and naming are stable and part of the output contract:

{"time":"...","group":"...","stream":"...","label":"...","message":"..."}

time is the event's reported timestamp in RFC3339Nano, falling back to its ingestion time when the former is absent — the same rule the Renderer and SortingSink apply. stream is omitted when the source is group-wide (no single stream).

func (JSONSink) Consume

func (s JSONSink) Consume(ctx context.Context, events <-chan Event) error

Consume implements Sink.

type Lister

type Lister interface {
	// List returns the refs matching filter (a prefix, status, etc. — the exact
	// meaning is resolver-defined).
	List(ctx context.Context, filter string) ([]ResourceRef, error)
}

Lister is the inverse of a Resolver: where Resolve maps ref -> sources, List maps a kind -> the refs that exist. It is an optional, separate interface so a Resolver can implement it (or not); CLI code type-asserts for it to offer "list -> pick -> tail" in one step.

Cleanly enumerable kinds have an enumerate API (log groups, SFn executions, Batch jobs, Lambdas). EC2 does not ("does this instance even have logs?"), so it is deliberately not a List target.

type MapSink

type MapSink struct {
	Inner Sink
	Map   func(Event) Event
}

MapSink wraps another Sink and passes every event through Map before forwarding it to Inner. It is the transformation counterpart to FilterSink: where FilterSink decides whether an event proceeds, MapSink decides what it looks like when it does (the CLI's --fields JSON-field extraction is the motivating consumer). A nil Map forwards events unchanged.

func (MapSink) Consume

func (m MapSink) Consume(ctx context.Context, events <-chan Event) error

Consume implements Sink. It mirrors FilterSink.Consume's discipline: relay the (mapped) events to Inner via an intermediate channel that this method owns and closes, then wait for Inner.Consume to return.

On ctx cancellation, Consume still closes out and waits for Inner to unwind before returning, but reports ctx.Err() rather than whatever Inner returned — cancellation is the authoritative reason Consume is ending.

type Option

type Option func(*options)

Option configures Tail.

func WithErrorHandler

func WithErrorHandler(f func(Source, error)) Option

WithErrorHandler registers a callback invoked when a Backend fails to open a stream for a source. Without it such errors are silent — which, with a live backend, looks indistinguishable from "no logs yet", so the CLI sets one.

func WithGracePeriod

func WithGracePeriod(d time.Duration) Option

WithGracePeriod overrides how long Tail keeps streaming after Done fires.

type Renderer

type Renderer struct {
	// Out is where lines are written. Defaults to os.Stdout when nil is passed
	// to NewRenderer.
	Out io.Writer

	// Color enables ANSI color. Callers typically set this from isatty.
	Color bool

	// ShowTime prepends each line with the event timestamp.
	ShowTime bool

	// MaxGutter caps the width (in runes) of the label gutter. Labels longer
	// than this are middle-ellipsized so a long resource name (e.g. a folded log
	// stream id) can't push every message off the right edge. 0 means no cap —
	// appropriate when output is piped, so downstream tools see full labels. The
	// CLI sets it from the terminal width.
	MaxGutter int
	// contains filtered or unexported fields
}

Renderer is the default Sink: a docker-compose-style interleaved printer that assigns each Source a stable color and prefixes every line with its label.

It is intentionally simple (one line per event, no progress bar / paging). The richer TUI ("k9s for CloudWatch") is a later, separate Sink — keeping this one minimal preserves the small public surface the design calls for.

func NewRenderer

func NewRenderer(out io.Writer, color, showTime bool) *Renderer

NewRenderer builds a Renderer writing to out (os.Stdout if nil).

func (*Renderer) Consume

func (r *Renderer) Consume(ctx context.Context, events <-chan Event) error

Consume implements Sink.

type Resolution

type Resolution struct {
	// Sources delivers log sources as they are discovered. It is closed once
	// discovery is complete (no more sources will ever appear).
	Sources <-chan Source

	// Done is closed when the resource reaches a terminal state (job SUCCEEDED,
	// execution FAILED…). It drives --follow shutdown. A nil Done means the
	// resource is never terminal (e.g. a bare log group), so following runs
	// until the caller cancels the context.
	//
	// Completion MUST be driven by resource status, never by "logs went quiet":
	// CloudWatch lags, and the last lines often arrive after the resource ends.
	// Tail applies a grace period after Done before stopping. Keeping this as a
	// signal (not a return value) prevents Batch/SFn status logic from leaking
	// into the tailer.
	Done <-chan struct{}
}

Resolution is the live result of resolving a reference. Sources may grow over time (a running Step Functions execution spawns tasks); Done reports that the underlying resource has reached a terminal state.

type Resolver

type Resolver interface {
	// Scheme is the reference prefix this Resolver handles, e.g.
	// "sfn-execution" or "batch-job". Used by CLI-level scheme dispatch.
	Scheme() string

	// Resolve turns a reference into a Resolution. ref is the scheme-stripped
	// remainder (an ARN, a name, a job id…). Resolve should return quickly;
	// ongoing discovery happens on the returned channels.
	Resolve(ctx context.Context, ref string) (Resolution, error)
}

Resolver maps a resource reference to the log sources it produces. Resolvers are the primary extension point: log-group, batch-job, sfn-execution, lambda…

A Resolver is just a package that returns a New() value; consumers wire them explicitly. There is deliberately no global registry.

type ResourceRef

type ResourceRef struct {
	Scheme    string // the Resolver scheme this ref belongs to
	Ref       string // the scheme-stripped reference, ready to pass to Resolve
	Label     string
	Status    string // e.g. "RUNNING", "SUCCEEDED" (resolver-defined)
	StartedAt time.Time
}

ResourceRef is one entry from a Lister: a reference that could be tailed, plus enough metadata to show it in a picker.

type Sink

type Sink interface {
	// Consume reads events until the channel is closed or ctx is cancelled,
	// then returns. The error is propagated out of Tail.
	Consume(ctx context.Context, events <-chan Event) error
}

Sink consumes the merged, interleaved event stream. The default Sink is the renderer in this package; a consumer can supply its own (JSON, custom TUI…).

type SortingSink

type SortingSink struct {
	Inner Sink
	Limit int
}

SortingSink buffers every event and, once the stream ends, emits them to Inner ordered by event time. Tail interleaves sources by arrival, not by timestamp, so reviewing a finished resource (e.g. a completed Step Functions execution tailed with poll + --since) otherwise reads out of order across streams.

It holds all events in memory and prints nothing until the stream closes, so it suits only bounded, historical tailing — not follow mode or the live backend, where the stream never ends. The CLI's --sort flag wires it and requires `--backend poll` without `-f`.

Limit caps how many events it will buffer; past it Consume returns an error rather than growing without bound (OOM). 0 means unlimited. Bound the input with --since/--until instead of relying on a huge limit.

If the context is cancelled (Ctrl-C) mid-fetch, it still flushes the ordered prefix it has buffered rather than discarding everything. Ordering is content-stable (see lessEvent), so the same events always print in the same order — golden-test friendly and free of spurious diff churn.

func (SortingSink) Consume

func (s SortingSink) Consume(ctx context.Context, events <-chan Event) error

Consume implements Sink.

type Source

type Source struct {
	// Key is a stable identifier used for color assignment and de-duplication.
	// It must be unique per logical stream within a run.
	Key string

	// Label is the short human-facing name shown in the output gutter,
	// e.g. "lambda/ingest" or "batch[3]".
	Label string

	// LogGroup is the CloudWatch Logs log group name.
	LogGroup string

	// LogStream optionally narrows to a single stream within the group.
	// Empty means "all streams in the group".
	LogStream string
}

Source is a single resolved log stream to tail. A Resolver emits these as it discovers them; a Backend turns each one into a stream of Events.