triage

README

triage

Lightweight, dependency-free security analysis of byte strings for Go: Shannon entropy scoring, content classification, and secret/credential detection. Point it at any string and it tells you whether it looks like a URL, an IP, a file path, an opaque high-entropy blob, or a leaked credential.

Extracted from txtr, where it powers the --triage/--secrets modes for firmware and malware analysis.

Install

go get github.com/richardwooding/triage

Usage

package main

import (
	"fmt"

	"github.com/richardwooding/triage"
)

func main() {
	res := triage.Classify([]byte("AKIAIOSFODNN7EXAMPLE"), 4.5)

	fmt.Printf("entropy: %.2f\n", res.Entropy)        // entropy: 3.68
	fmt.Printf("categories: %v\n", res.Categories)    // categories: []
	for _, s := range res.Secrets {
		fmt.Printf("[%s] %s: %s\n", s.Severity, s.Rule, s.Match)
		// [HIGH] AWS Access Key: AKIAIOSFODNN7EXAMPLE
	}
	fmt.Println(res.Interesting())                    // true
}

Classify takes the bytes to analyze and a minEntropy threshold (bits/byte) above which an opaque token is flagged as high-entropy. Pass 0 to disable high-entropy flagging.

Each SecretFinding carries the byte offsets of the match (Start/End, where str[Start:End] == Match), so you can highlight or redact the exact span.

The raw entropy function is exported too:

triage.Entropy([]byte("hello"))   // 1.92

Redaction

Use Redact to scrub secrets out of text (logs, crash dumps, anything you're about to store or forward) before it leaves your process:

clean := triage.Redact([]byte(`db: "AKIAIOSFODNN7EXAMPLE"`))
// db: "[REDACTED]"

Redact never modifies its input and returns a new slice. RedactWith lets you supply a custom mask. Overlapping matches are merged so each region is masked exactly once, and the fixed mask avoids leaking the original secret's length.

What it detects

Content categories (first match wins, structural classification):

Category	Example
URL	https://example.com/path
Email	user@example.com
IPv4 / IPv6	10.0.0.5, 2001:db8::1 (validated via net.ParseIP)
Domain	api.github.com
WinPath / UnixPath	C:\Windows\System32, /usr/bin/txtr
Hex / Base64	deadbeefcafebabe, TWFuIGlz...
UUID	550e8400-e29b-41d4-a716-446655440000

Secret rules (precompiled, with optional entropy gating):

AWS access keys, PEM private keys, GitHub tokens, Slack tokens, Stripe keys, Google API keys, JSON Web Tokens, and generic key=value secret assignments.

High-entropy blobs: long opaque tokens above the entropy threshold that don't classify as benign structural content (so file paths and cipher lists don't get flagged, but base64/hex key material does).

Design

• Zero dependencies — standard library only.
• ReDoS-safe — all patterns use Go's regexp (RE2), which guarantees linear-time matching; patterns are compiled once at package init, never per call.
• Low false positives — IPs are validated with net.ParseIP; the generic secret rule requires an actual :/= assignment (not prose); domains require a lowercase TLD; high entropy is suppressed for recognized structural content.
• Fast — strings shorter than the shortest possible secret skip the secret scan; entropy is a single O(n) pass.

License

MIT — see LICENSE.

Documentation

Overview

Package triage provides lightweight security analysis of byte strings: Shannon entropy scoring, content classification (URL, email, IPv4/IPv6, domain, file path, hex, base64, UUID), and secret/credential detection (AWS keys, PEM private keys, GitHub/Slack/Stripe tokens, Google API keys, JWTs, and generic key=value assignments).

It depends only on the standard library, making it easy to embed in scanners, CI secret gates, log scrubbers, and forensics tooling. The entry point is Classify; Entropy is exposed separately for reuse.

Index

• Constants
• func Entropy(data []byte) float64
• func Redact(str []byte) []byte
• func RedactWith(str, mask []byte) []byte
• type Category
• type Result
  • func Classify(str []byte, minEntropy float64) Result
  • func (r Result) Interesting() bool
• type SecretFinding
• type Severity

Constants

const DefaultRedactMask = "[REDACTED]"

DefaultRedactMask is the replacement written over each detected secret by Redact. A fixed token (rather than length-preserving asterisks) avoids leaking the original secret's length.

Functions

func Entropy

func Entropy(data []byte) float64

Entropy returns the Shannon entropy of data in bits per byte, a value in the range [0, 8]. Higher values indicate more randomness: natural-language text typically scores ~4.0-4.5, base64-encoded data ~6, and cryptographic key material approaches 8.

func Redact

func Redact(str []byte) []byte

Redact returns a copy of str with every detected secret replaced by DefaultRedactMask. It is intended for scrubbing logs, crash dumps, and other text before it is stored or forwarded. The input is never modified.

Detection uses the same rules as Classify; bytes that are not part of a secret are preserved verbatim. Overlapping matches are merged so each region is masked exactly once.

func RedactWith

func RedactWith(str, mask []byte) []byte

RedactWith behaves like Redact but replaces each detected secret with the supplied mask instead of DefaultRedactMask.

Types

type Category

type Category string

Category is a human-readable classification of a string's content.

const (
	CategoryURL         Category = "URL"
	CategoryEmail       Category = "Email"
	CategoryIPv4        Category = "IPv4"
	CategoryIPv6        Category = "IPv6"
	CategoryUUID        Category = "UUID"
	CategoryPathWindows Category = "WinPath"
	CategoryPathUnix    Category = "UnixPath"
	CategoryDomain      Category = "Domain"
	CategoryHex         Category = "Hex"
	CategoryBase64      Category = "Base64"
)

Classification categories.

type Result

type Result struct {
	// Entropy is the Shannon entropy (bits/byte) of the trimmed string.
	Entropy float64
	// HighEntropy is true when Entropy meets the configured threshold and the
	// string looks like an opaque token rather than natural-language text.
	HighEntropy bool
	// Categories holds the content classification (at most one, first match
	// wins by priority); empty when nothing matched.
	Categories []Category
	// Secrets holds any detected secrets/credentials.
	Secrets []SecretFinding
}

Result holds the triage analysis of a single string.

func Classify

func Classify(str []byte, minEntropy float64) Result

Classify analyzes a string and returns its entropy, content category, and any detected secrets. minEntropy is the threshold (bits/byte) above which an opaque token is flagged as high-entropy; pass 0 to disable that flag.

func (Result) Interesting

func (r Result) Interesting() bool

Interesting reports whether the result is worth surfacing in --secrets mode: it contains a detected secret or qualifies as a high-entropy blob.

type SecretFinding

type SecretFinding struct {
	Rule     string   `json:"rule"`
	Severity Severity `json:"severity"`
	Match    string   `json:"match"`
	// Start and End are the byte offsets of the match within the scanned
	// string: str[Start:End] == Match. End is exclusive. They make it possible
	// to highlight or redact the exact span (see [Redact]).
	Start int `json:"start"`
	End   int `json:"end"`
}

SecretFinding describes a single secret detected inside a string.

type Severity

type Severity string

Severity classifies how serious a secret finding is.

const (
	SeverityHigh   Severity = "HIGH"
	SeverityMedium Severity = "MEDIUM"
	SeverityLow    Severity = "LOW"
)

Severity levels for secret findings.