logrefract

README

logrefract

The Universal, High-Speed JSON Log Formatter, Analyzer & Observer.

logrefract is a "bare-metal" CLI utility designed to format, analyze, and observe structured JSON logs with extreme efficiency. While written in Go, it is a universal tool that provides deep visibility into logs from any language or framework (Java, Python, Rust, Node.js, C#, etc.).

Performance

logrefract is built for extreme speed, processing over 1 million lines per second on standard hardware. By using a high-efficiency byte-scanner and a manual state machine, it processes log streams with ultra-low latency and a stable, near-zero memory growth footprint.

It is designed to handle massive, high-throughput production log streams without slowing down your data pipeline or exhausting system resources. Whether you are processing 1MB or 100GB, the resource profile remains constant and predictable.

(For detailed measurements, see the Full Performance Benchmarks)

Installation

via Go (Recommended)

go install github.com/kishan-thanki/logrefract/cmd/logrefract@latest

Manual

logrefract is a single, self-contained binary with zero external dependencies. Simply download the binary for your OS and move it to your /usr/local/bin.

Shell Completion (Optional)

Generate and install completion scripts for a better CLI experience:

# Bash (add to ~/.bashrc)
source <(logrefract -completion bash)

# Zsh (add to ~/.zshrc)
source <(logrefract -completion zsh)

Usage

The Standard Pipe

Pipe any JSON log stream directly into logrefract:

# Kubernetes Logs
kubectl logs -f my-pod | logrefract

# Docker Logs
docker logs -f my-container | logrefract

# Local application output
./my-app-binary | logrefract

Real-World Scenarios

High-performance examples for production analysis:

1. Debugging Outages (The "Story")

Find an error and see exactly what happened 10 lines before and after it:

tail -f app.log | logrefract -filter level=ERROR -C 10

2. Monitoring API Health

Instantly isolate server-side failures (500 errors):

tail -f access.log | logrefract -filter "status>=500"

3. High-Traffic Observability

Monitor a high-velocity stream without crashing your terminal by sampling only 5% of traffic:

tail -f traffic.log | logrefract -sample 5

4. Multi-Service Investigation

Filter logs for a specific microservice and highlight trace IDs to follow a request:

cat cloud-logs.json | logrefract -filter service=auth-api -highlight "req_882f"

Direct File Input

logrefract -input production.log -output readable.log

Features

Analysis Engine

Perform complex logic on your logs directly in the terminal without complex queries.

• Numeric Comparisons: Filter by status codes or metrics (e.g., logrefract -filter "status>400").
• Time-Based Filtering: Isolate logs within a time window (e.g., -since 15m or -until 2026-06-24T12:00:00).
• Regex Matching: Perform complex fuzzy searches (e.g., -regex "failed.*payment").
• Raw JSON Piping: Output the filtered results as raw JSON instead of pretty-printing (e.g., -json > filtered.json).
• Match Counting: Output only the total number of matches with the -count flag.
• Exclusion Logic: Hide noise instantly (e.g., logrefract -exclude level=DEBUG).
• Logical OR: Filter for multiple states (e.g., logrefract -filter level=ERROR,WARN).
• Universal Key Mapping: Adapt to any proprietary logging format with -map level=severity.

Stream Observation

Monitor high-traffic production environments with precision.

• Contextual Awareness: See the "story" around your errors with grep-style context (-C 5).
• High-Volume Sampling: Watch traffic without flooding your terminal using high-speed sampling (-sample 10).
• Visual Highlighting: Make specific trace IDs or patterns "pop" with -highlight <str>.

Universal Formatter

Transform raw JSON into a clean, actionable data view.

• Smart Colorization: Automatically color-coded log levels for instant visual scanning.
• Deep Pretty Printing: Expand complex nested JSON objects with the -pretty flag.
• Stable Field Sorting: Non-standard fields are sorted alphabetically for consistent, diff-friendly output.

License

Licensed under the Apache License 2.0. See LICENSE for details.