ndjson

Documentation

Overview

Package ndjson classifies log files as NDJSON and extracts the text lines fed to pattern mining from structured entries, per ADR 0006.

Index

• func Extract(line string) string
• type Format
  • func DetectFormat(lines []string) Format

Functions

func Extract

func Extract(line string) string

Extract converts one NDJSON line into the text line fed to pattern mining. Envelope entries {"ts": ..., "severity": ..., "payload": {...}} extract from the payload; any other object is the payload itself. The extracted line is "<severity> <message>", just "<message>" when no severity-like string field exists, or the compact payload JSON when no message-like string field exists. A line that does not parse as a JSON object is returned unchanged.

Types

type Format

type Format string

Format classifies the on-disk format of a log file.

const (
	// FormatText marks a file for the plain text pipeline.
	FormatText Format = "text"
	// FormatNDJSON marks a file whose lines are JSON objects, one per line.
	FormatNDJSON Format = "ndjson"
)

func DetectFormat

func DetectFormat(lines []string) Format

DetectFormat classifies a file's lines. A file is NDJSON only when it has at least one non-empty line and every non-empty line parses as a JSON object. Files mixing text with JSON lines stay on the text path, so the extraction never has to guess on a half-structured file.