logfmt

README

logfmt

A fast, allocation-free reader for the logfmt line format in Go:

level=info msg="user login" user=john id=42 success=true

The package operates on []byte and reports keys and values as sub-slices of the input, so iterating a line performs zero allocations. It has no dependencies outside the standard library, and parses a ~1.4 KB line at ~3 GB/s — roughly 6× go-logfmt, with key extraction 12× faster still.

Install

go get github.com/JohanLindvall/logfmt

Requires Go 1.21 or newer. (CI tests that floor on every push on amd64, and the current stable release on both amd64 and arm64.) Ranging over All needs Go 1.23 in your module — the library itself stays at 1.21, so it never forces a toolchain upgrade on you.

Usage

Iterate over every key/value pair

Iterate calls your callback once per pair. The key and val slices alias the input buffer, so copy them if you need to keep them past the call. Return false from the callback to stop early.

line := []byte(`level=info msg="user login" user=john id=42`)

err := logfmt.Iterate(line, func(key, val []byte) bool {
    fmt.Printf("%s = %s\n", key, val)
    return true // return false to stop early
})
if err != nil {
    log.Fatal(err)
}

On Go 1.23 or newer, All is the same walk as a range loop:

for key, val := range logfmt.All(line) {
    fmt.Printf("%s = %s\n", key, val)
}

Notes:

• A bare key with no = (e.g. debug) is reported with val equal to the literal true. IsBareKey(val) tells that apart from an explicit debug=true, which is otherwise byte-identical.
• Quoted values are returned without the surrounding quotes but are not unescaped — backslash escapes are left intact. val doesn't say whether it was quoted, and escapes only mean anything inside quotes, so decode with AppendValue or GetQuoted rather than calling AppendUnescape on whatever the callback hands you (see Escapes belong to quoted values only).
• Returned slices alias the input (or, for bare keys, a shared constant) — treat them as read-only and copy anything that must outlive the input.
• The parser is deliberately lenient and diverges from go-logfmt in a few documented ways (e.g. a stray " in an unquoted value is a literal byte, not an error). See the package documentation.
• All lookups (Get, GetQuoted, AppendValue, GetMany) resolve duplicate keys the same way: the first non-empty occurrence wins; an empty value is used only when the key never appears with a non-empty one.

One record per call

Newlines are ordinary whitespace to this parser — there is no record framing. Hand it a multi-line buffer and you get every line's pairs as one flat stream, with no boundary marker, and lookups can match a key from a later line. SplitRecord peels off one record at a time without allocating, and handles CRLF:

for len(data) > 0 {
    var rec []byte
    rec, data = logfmt.SplitRecord(data)
    level, _ := logfmt.Get(rec, "level")
    fmt.Printf("%s\n", level)
}

Look up a single key, unescaped

AppendValue finds a key and appends its unescaped value to your buffer, returning the extended slice and whether the key was present. It always appends, so the result never aliases the input and is yours to keep.

var buf []byte
val, ok := logfmt.AppendValue(buf[:0], line, "msg")
if !ok {
    // key absent
}
fmt.Printf("msg = %s\n", val)

Look up a single key, raw

Get returns the raw value (surrounding quotes removed, escape sequences left intact) and whether the key was found. The result aliases the input — no copy, no allocation — and is valid until the input is modified. Use AppendValue instead when you want the value unescaped into your own buffer.

val, ok := logfmt.Get(line, "msg")
if !ok {
    // key absent
}
fmt.Printf("msg = %s\n", val) // raw value, aliasing line

A key present with an empty value (msg=) returns ok == true and a non-nil empty slice, so it stays distinct from an absent key.

Look up several keys in one pass

GetMany extracts multiple keys in a single scan, stopping early once all are found. Each returned value is raw and aliases the input; a missing key is reported as nil. (A present but empty value, such as from key=, is a non-nil zero-length slice, so it stays distinct from an absent key.) Pass a [][]byte to reuse as the result slice across calls and avoid allocating it each time.

keys := []string{"timestamp", "level"}
var buf [][]byte // reuse across calls

vals := logfmt.GetMany(line, keys, buf)
for i, v := range vals {
    if v == nil {
        continue // keys[i] not present
    }
    fmt.Printf("%s = %s\n", keys[i], v)
}

Keys are matched linearly against each parsed field, which is fastest for the handful of keys these lookups target. Measured on a 24-field line, GetMany stays ahead up to roughly ten keys; past that, use Iterate with a map keyed by string(k) (20 keys: ~505 ns vs ~385 ns) — the compiler optimizes that conversion away in a map index.

Unescape a raw value

AppendUnescape decodes the escapes in a raw value (as returned by Iterate, All, Get or GetMany), appending to a destination buffer. It recognises \n, \r, \t and JSON-style \uXXXX unicode escapes including surrogate pairs — so values encoded by go-logfmt (which writes control characters as \u00XX) round-trip correctly. Any other escaped byte (such as \" or \\) is emitted as-is; malformed \u sequences and a trailing lone backslash are kept verbatim.

It always appends, so the result never aliases the input.

dst := logfmt.AppendUnescape(nil, []byte(`hello\tworld`)) // -> hello<TAB>world

Most values contain no escapes at all, so guard with NeedsUnescape when you want to skip the copy entirely. Use GetQuoted rather than Get here — it reports whether the value was quoted, which is what makes the decode safe (see below):

if v, quoted, ok := logfmt.GetQuoted(line, "msg"); ok {
    if quoted && logfmt.NeedsUnescape(v) {
        v = logfmt.AppendUnescape(buf[:0], v) // decoded into buf
    }
    // otherwise v still aliases line, with no copy made
}

Escapes belong to quoted values only

A backslash means something only inside quotes. msg="a\nb" holds an escape; path=C:\Users\bob holds three literal backslashes — and go-logfmt's encoder writes it exactly that way, because a backslash is not one of the bytes that force quoting. Decoding without knowing which of the two you have is silent corruption, since both are perfectly valid logfmt:

line := []byte(`path=C:\Users\bob\new`)

logfmt.Get(line, "path")            // C:\Users\bob\new   — raw, correct
logfmt.AppendValue(nil, line, "path") // C:\Users\bob\new — knows it was unquoted
// AppendUnescape(nil, raw) would give "C:Usersbob<NL>ew": \U→U, \b→b, \n→newline

Iterate, All, Get and GetMany hand out quoted and unquoted values alike without distinguishing them. Two entry points carry the missing bit:

• AppendValue decodes for you, and only when the value was quoted.
• GetQuoted returns the bit, for the zero-copy path above.

Reach for AppendUnescape directly only on a value you already know was quoted.

Parse a timestamp value

ParseTime parses a logfmt timestamp value and reports whether it succeeded. It accepts an RFC3339Nano string, a 2006-01-02 15:04:05.999 -0700 MST string, or a unix epoch (exactly 10 integer digits with an optional fractional part). Trailing delimiters left over from a slightly malformed line (e.g. a stray }) are trimmed first, and on success the returned time is normalized to UTC.

t, ok := logfmt.ParseTime([]byte("1748239806.3691056"))
if ok {
    fmt.Println(t) // 2025-05-26 06:10:06.3691056 +0000 UTC
}

Millisecond/microsecond epochs (13 or 16 digits) and date-only strings are rejected rather than guessed at — see the package docs for the full accepted set. If you know your emitter's layout, time.Parse with that layout is both faster and stricter.

Read-only really means read-only

Returned slices are windows onto the input, so treat every one of them as read-only: writing through a value overwrites your log line, and a bare key's true is a package-level constant shared by every caller in the process.

Appending is the subtler case, and the two APIs differ deliberately:

• Get and GetMany cap their results (cap == len), so append(v, …) copies instead of overwriting whatever follows the value in the input. They set a slot once per lookup, so the capping is free. AppendValue and AppendUnescape go further and never alias the input at all.
• Iterate does not cap what it passes the callback. Doing so costs ~4.5% on field-dense input (measured) because it lands once per field. Inside a callback, copy before appending — append(dst[:0], v...), string(v) — or re-slice with v[:len(v):len(v)] yourself.

Errors

Only Iterate and Validate report syntax errors, and both return a *SyntaxError carrying the byte offset of the fault:

if err := logfmt.Validate(line); err != nil {
    var se *logfmt.SyntaxError
    if errors.As(err, &se) {
        fmt.Printf("bad record at byte %d: %s\n", se.Offset, se.Reason)
    }
}

errors.Is(err, logfmt.ErrBadFormat) matches any of them, so sentinel checks work too. There are exactly two faults: an unterminated quoted value, and a closing quote followed by a non-space byte.

Two consequences of streaming:

• When Iterate returns an error, every pair before the fault has already been delivered to your callback. That prefix is valid.
• The lookups report no errors at all. They stop as soon as their keys are settled, so a malformed tail beyond that point is never examined — an error return would promise a validation they do not perform. They give you what the reachable prefix holds; call Validate when a record's validity matters.

Absence is uniform: Get and AppendValue return false, GetMany leaves the slot nil.

Scope

A reader, deliberately and only. Not included: an encoder, an io.Reader streaming decoder, typed accessors (int/bool/duration), and map building. Values come back as []byte for you to convert with strconv. That is what keeps the package dependency-free, allocation-free, and small enough to fuzz the whole parser against a byte-by-byte reference implementation on every change. To write logfmt, use go-logfmt or your logging library's encoder.

Benchmarks

go test -bench=. -benchmem      # this package's microbenchmarks
make bench-md                   # regenerate the committed tables in bench/

Iterate, All, Get, GetQuoted, GetMany and SplitRecord allocate nothing on a well-formed record (and AppendValue/AppendUnescape nothing beyond growing your buffer). The one exception is a malformed record, which costs a single 24-byte *SyntaxError — and the lookups pay it only when they have to walk past the fault to settle their keys.

Cost splits into a fixed per-field overhead of ~5 ns plus scanning: ~11.7 GB/s through unquoted values (word-at-a-time SWAR) and ~27 GB/s through quoted ones (bytes.IndexByte, SIMD in the stdlib). A short unquoted value that ends within a few bytes of its = is cheaper still: the key scan has already seen those bytes and settles the value without a second scan. Short fields are therefore overhead-bound, long values scan-bound. Lookups are linear in how deep the key sits: ~7 ns per field skipped.

That 27 GB/s is for quoted values with no escaped quotes in them. Escaped quotes cost extra, but boundedly: the first \" in a value restarts bytes.IndexByte, and from then on — provided the escapes are close enough together to be worth it — the parser walks a word at a time looking for the next " or \, consuming each escape as it goes, and falls back to bytes.IndexByte as soon as they thin out again. So a value dense with escapes — embedded JSON, where every quote is one — costs a few nanoseconds per escape rather than a fresh IndexByte call each, while a value with two escapes 200 bytes apart never leaves the fast path it was already on. A 1 KB value with 500 escapes parses roughly 59× slower than a clean 1 KB; Benchmark_IterateEscaped sweeps that axis, and Benchmark_UnescapeEscaped sweeps it for AppendUnescape, which uses the same trick while decoding and is the slower half at high density.

On amd64, building with GOAMD64=v3 (Haswell+, 2013 onwards) makes the parser 1–2% faster (BMI's TZCNT for the word-at-a-time scanning). It is a consumer build flag, not something the module can set.

vs other Go logfmt parsers

The bench/ module is a separate module, so the root package stays dependency-free; it compares against go-logfmt, kr/logfmt and Grafana Loki's in-tree decoder. (The Loki entry is a stand-in adapted from go-logfmt under MIT rather than a vendored copy — Loki's own tree is AGPL-licensed — verified equivalent to Loki's decoder on these inputs; see bench/lokifmt.)

The numbers live in the generated tables, not here. They are produced by make bench-md (and by the bench CI workflow, which commits them), each stamped with the CPU and Go version that produced it — no figure is copied into this file, because a copied one goes stale silently and this one did:

• bench/results_amd64.md — cross-library comparison
• bench/results_arm64.md — same, on arm64
• bench/pkg_results_amd64.md — this package's own microbenchmarks

For orientation only, on the ~1.4 KB sample line this package parses every pair roughly 7× faster than go-logfmt with zero allocations, and extracts two keys roughly 14× faster by stopping as soon as both are found. Consult the tables for the actual figures on actual hardware; treat any ratio quoted in prose as approximate and possibly a release behind.

License

MIT — see LICENSE.

Documentation

Overview

Package logfmt provides a fast, allocation-free reader for the logfmt key/value line format:

level=info msg="user login" user=john id=42 success=true

API

Iterate is the core primitive: it walks a line and hands each key/value pair to a callback as sub-slices of the input, allocating nothing on a well-formed record. Values are raw — reported exactly as they appear in the input, with surrounding quotes stripped but escape sequences left intact. On top of it:

• All is the same walk as a range-over-func iterator: for key, val := range logfmt.All(line). Ranging needs Go 1.23 in the calling module; the package itself still builds at Go 1.21.
• Get returns the raw value for one key (zero-copy) and whether it was present.
• GetMany returns the raw values for several keys in a single pass, stopping early once all are found; a missing key yields nil, while a present-but-empty value is a non-nil empty slice.
• GetQuoted is Get plus whether the value was written quoted — the bit that decides whether unescaping it is correct.
• AppendValue appends one key's unescaped value to a caller-provided buffer, decoding only values that were quoted.
• AppendUnescape decodes escape sequences (\n, \r, \t, and JSON-style \uXXXX) in a quoted value; NeedsUnescape reports whether raw holds a backslash at all, so the decode can be skipped when it cannot.
• SplitRecord peels one record off a multi-line buffer (see Records and framing below).
• Validate parses a record to completion and reports the first syntax error, which the early-stopping lookups cannot.
• IsBareKey distinguishes a bare key's implicit "true" from a real one.
• ParseTime parses the timestamp formats that commonly appear in logfmt.

The lookups — Get, GetQuoted, GetMany and AppendValue — resolve duplicate keys the same way: the first non-empty occurrence wins, and an empty value is used only when the key never appears with a non-empty one. Data and values are []byte throughout — nothing asks the caller to convert input or results to string; only the lookup keys, in practice compile-time constants, are strings.

Escapes belong to quoted values only

A backslash means something only inside quotes. msg="a\nb" holds an escape; path=C:\Users\bob holds three literal backslashes, and go-logfmt's encoder writes it exactly that way, because a backslash is not one of the bytes that force quoting. Unescaping without knowing which of the two you have turns C:\Users\bob into C:Usersbob with an embedded newline — silently, since every byte of it is valid logfmt.

Iterate, All, Get and GetMany hand out quoted and unquoted values alike and do not distinguish them, so the raw value alone cannot tell you which it was. Two entry points carry the missing bit:

• AppendValue decodes for you, and only when the value was quoted.
• GetQuoted returns the bit, for callers who want to skip the copy when nothing needs decoding.

Reach for AppendUnescape directly only on a value you already know was quoted.

Records and framing

The parser has no notion of a record boundary: '\n' and '\r' are ordinary whitespace, exactly like ' '. Passing a multi-line buffer to Iterate therefore yields the pairs of every line as one flat sequence, with no indication of where one line ended and the next began — and a lookup will happily match a key from a later line. Callers that need per-record semantics must split the input themselves. SplitRecord does it without allocating, and handles CRLF:

for len(data) > 0 {
	var rec []byte
	rec, data = logfmt.SplitRecord(data)
	level, _ := logfmt.Get(rec, "level")
	// ...
}

bufio.Scanner works too when the input arrives as a stream. Feeding whole buffers to Iterate is supported and fast, but only when the flat view is what you want.

Aliasing and concurrency

Returned slices alias the input, a caller-provided buffer, or (for bare keys) a shared package-level constant — treat them as read-only, and copy any that must outlive the input. The bare-key sentinel is the one result that does not alias the input at all: it outlives and ignores any change to it, and IsBareKey reports true for it whether it arrived through a callback or through Get, GetQuoted or GetMany.

Read-only means what it says: these slices are windows onto the input, and a bare key's "true" is a constant shared by every caller in the process, so writing through any of them corrupts something you do not own.

Appending is handled differently by the entry points, deliberately. Get, GetQuoted and GetMany return values capped to their length, so appending to one copies rather than overwriting the bytes that follow it in the input; they cap once per lookup, which costs nothing measurable. AppendValue and AppendUnescape always copy into the caller's buffer, so their results never alias the input at all. Iterate and All do not cap what they hand the callback to the value's own length, because that lands once per field and measures ~4.5% on field-dense input; they do bound it at the end of the record, so an append cannot run past the data you handed in, but it can still overwrite later fields of the same record. Inside a callback, copy first (append(dst[:0], v...), string(v)) or re-slice to v[:len(v):len(v)] before appending.

The package holds no state, so it is safe for concurrent use as long as callers honour that rule.

Errors

The only malformed inputs are an unterminated quoted value and a closing quote followed by a non-space byte. Iterate and Validate report both as a *SyntaxError carrying the byte offset; errors.Is(err, ErrBadFormat) matches any of them, and a *SyntaxError type assertion or errors.As gets the offset. Note that the returned error is never ErrBadFormat itself, so compare with errors.Is rather than ==. Because parsing is streaming, Iterate has already delivered every pair preceding the fault before it returns — treat the callback's output as a valid prefix, not as something to discard.

That *SyntaxError is the one allocation this package makes on its own behalf. Every entry point is allocation-free on a well-formed record; a malformed one costs a single 24-byte error value, and the lookups pay it only when they have to walk past the fault to settle their keys (they discard it, since they report no errors). "Allocation-free" throughout this documentation means exactly that.

The lookups report no syntax errors at all. Get, GetQuoted, AppendValue and GetMany stop as soon as their keys are settled, so a malformed tail beyond that point is never examined; reporting an error they cannot reliably detect would promise a validation they do not perform. They return what the reachable prefix yields. Call Validate when a record's validity matters.

Absence is uniform: Get, GetQuoted and AppendValue return false, GetMany leaves the slot nil. A key present with an empty value stays distinct from an absent one — Get returns true with a non-nil empty slice, GetMany a non-nil empty slot.

Leniency

The parser is deliberately lenient — built for reading real-world logs, it never rejects input it can read as key/value pairs. This differs from go-logfmt in a few ways:

• A '"' inside an unquoted value is a literal byte, not a syntax error (a=x" b=c yields a="x"" and b="c").
• A '\' inside an unquoted value is a literal byte too, never an escape introducer (a=C:\n yields the four bytes C:\n, not C: followed by a newline). go-logfmt agrees; see "Escapes belong to quoted values only".
• Unknown escapes decode leniently (the escaped byte itself) instead of being rejected, and a malformed \uXXXX is kept verbatim.
• Control bytes other than whitespace (0x00–0x08, 0x0E–0x1F) are ordinary key/value bytes.
• Keys are never unquoted: "a b"=c parses as the bare key `"a` and the pair `b"`=c. Quoting is meaningful only in value position, immediately after '='.
• 'key=' followed by whitespace is an empty value, and the following token starts a new pair.
• A '=' with no key before it yields a pair with an EMPTY key: =v parses as the pair ""="v", where go-logfmt reports "unexpected '='". A lookup for the key "" can therefore genuinely match.
• A '=' inside an unquoted value is a literal byte: a==b parses as the pair "a"="=b", which go-logfmt also rejects. Only the first '=' of a field separates key from value.

A bare key with no '=' is reported with the value "true", matching logfmt convention for boolean flags. That value is a shared sentinel, so IsBareKey can tell "debug" from "debug=true" — by content the two are identical.

Scope

This is a reader only, by design. There is no encoder, no io.Reader-based streaming decoder, no typed accessors (integers, booleans, durations) and no map-building convenience: values come back as []byte for the caller to convert. ParseTime is the one concession, because timestamp formats in real logs vary enough to be worth centralising. That keeps the package dependency-free, allocation-free and its semantics small enough to fuzz against a reference implementation. Write logfmt with go-logfmt or your logging library's encoder; convert values with strconv.

Index

• Variables
• func All(data []byte) func(yield func(key, val []byte) bool)
• func AppendUnescape(dst []byte, raw []byte) []byte
• func AppendValue(dst, data []byte, key string) ([]byte, bool)
• func Get(data []byte, key string) ([]byte, bool)
• func GetMany(data []byte, keys []string, buf [][]byte) [][]byte
• func GetQuoted(data []byte, key string) (val []byte, quoted, found bool)
• func IsBareKey(val []byte) bool
• func Iterate(data []byte, fn func(key, val []byte) bool) error
• func NeedsUnescape(raw []byte) bool
• func ParseTime(ts []byte) (time.Time, bool)
• func SplitRecord(data []byte) (record, rest []byte)
• func Validate(data []byte) error
• type SyntaxError
  • func (e *SyntaxError) Error() string
  • func (e *SyntaxError) Is(target error) bool

Variables

var ErrBadFormat = errors.New("bad logfmt format")

ErrBadFormat is the sentinel every syntax error matches: the input is not valid logfmt, for example a quoted value that is never closed or that is followed by a non-space byte. Errors returned by Iterate and Validate are *SyntaxError values carrying the offset; test them with errors.Is(err, ErrBadFormat).

Functions

func All

func All(data []byte) func(yield func(key, val []byte) bool)

All returns an iterator over data's key/value pairs, for use with range:

for key, val := range logfmt.All(line) {
	fmt.Printf("%s=%s\n", key, val)
}

The pairs, their aliasing and the bare-key sentinel are exactly as described on Iterate. Ranging over a function requires Go 1.23 in the calling module; this package itself still builds on Go 1.21, where All is callable directly.

A range loop yields two values, so All cannot report whether a value was quoted any more than it can report an error. Values that need unescaping are GetQuoted's and AppendValue's job.

A range loop has nowhere to deliver an error, so All simply stops at a malformed field, having yielded the valid prefix. Call Validate if you need to know; use Iterate to get the error and the pairs in one pass.

func AppendUnescape

func AppendUnescape(dst []byte, raw []byte) []byte

AppendUnescape decodes the backslash escapes in a raw logfmt value, appends the result to dst and returns the extended slice. It recognises \n, \r, \t and JSON-style \uXXXX unicode escapes (including surrogate pairs, as emitted by go-logfmt for control characters); any other escaped byte (such as \" or \\) is emitted as the byte itself. A lone surrogate half decodes to U+FFFD, matching encoding/json. A malformed \u (bad or truncated hex) and a trailing lone backslash are kept verbatim rather than rejected.

Pass it ONLY a value that was quoted in the input. Escapes are meaningful only inside quotes: an emitter writes path=C:\Users\bob unquoted and means every byte literally, so decoding that yields C:Usersbob with an embedded newline. Iterate, All, Get and GetMany all hand out quoted and unquoted values alike without distinguishing them; GetQuoted reports which it was, and AppendValue applies this function only when it should. Feeding raw values through here unconditionally is the one way to corrupt data with this package.

It always appends — the result never aliases raw — so it composes like the other Append functions in the standard library. Pass dst[:0] to reuse a buffer without allocating. To skip the copy entirely for values that need no decoding, guard with NeedsUnescape; that pattern is also faster than decoding unconditionally, since most values contain no escapes at all.

func AppendValue

func AppendValue(dst, data []byte, key string) ([]byte, bool)

AppendValue looks up key in data, appends its unescaped value to dst and returns the extended slice along with whether the key was present. When the key is absent it returns dst unchanged and false.

Only a quoted value carries escapes, and AppendValue knows which values were quoted, so an unquoted one is copied through byte for byte — path=C:\Users\bob comes back intact rather than "decoded" into nonsense.

It always appends, so the result never aliases data and is safe to keep. Callers who would rather not copy values that need no decoding should use GetQuoted with NeedsUnescape instead:

if v, quoted, ok := logfmt.GetQuoted(line, "msg"); ok {
	if quoted && logfmt.NeedsUnescape(v) {
		v = logfmt.AppendUnescape(buf[:0], v)
	}
	// v now aliases line (no copy) or buf (decoded)
}

Duplicate keys resolve exactly as in Get and GetMany: the first non-empty occurrence wins, and an empty value is used only when no non-empty one exists. A malformed record yields whatever could be parsed before the fault; use Validate when you need to know.

func Get

func Get(data []byte, key string) ([]byte, bool)

Get returns the raw value for key in data — the value as it appears in the input, with any surrounding quotes removed but escape sequences left intact — and whether the key was present. A bare key (one written with no '=' at all) yields the shared "true" sentinel that IsBareKey recognises, which is the one result that does not alias data.

Raw means raw: do not pass the result to AppendUnescape without first establishing that the value was quoted, since escapes are meaningful only inside quotes. GetQuoted reports that, and AppendValue handles it for you. An absent key yields (nil, false); a key present with an empty value yields a non-nil empty slice and true, so the two stay distinguishable. Decode escapes with AppendUnescape, or use AppendValue for a one-call unescaped lookup.

The result aliases data (treat it as read-only) and is valid only until data is modified. It has capacity equal to its length, so appending to it copies rather than overwriting the bytes that follow the value in data. (Iterate, which calls back once per field rather than once per lookup, does not cap what it hands the callback — capping there costs measurably.)

Duplicate keys resolve as in AppendValue and GetMany: the first non-empty occurrence wins (iteration stops there); an empty value is returned only when the key never appears with a non-empty one.

Get reports no syntax errors. It stops as soon as the key is settled, so a malformed tail beyond that point is never examined; what it can reach, it returns. Call Validate when you need the record checked.

func GetMany

func GetMany(data []byte, keys []string, buf [][]byte) [][]byte

GetMany looks up several keys in a single pass over data. It returns a slice the same length as keys, where the i-th element is the raw value for keys[i] (any surrounding quotes removed, escape sequences left intact), or nil if that key is not present. A present but empty value (for example from "key=") aliases data and is a non-nil zero-length slice, so it is distinct from a missing key's nil.

A key matched by both an empty and a non-empty value resolves to the first non-empty one: an empty value is recorded only provisionally and is overridden by any later non-empty value for the same key.

Entries in keys are expected to be distinct. A key listed twice is a degenerate case: each parsed field fills the first slot for that key not yet settled with a non-empty value, so duplicate slots are filled by successive occurrences in data, and when the key occurs fewer times than it is listed the extra slots stay nil — reading as absent even though the key is present.

The returned values alias data (treat them as read-only) and are valid only until data is modified; each has capacity equal to its length, so appending to one copies rather than overwriting the bytes that follow it in data. A bare key yields the shared "true" sentinel, which does not alias data.

GetMany does not report which values were quoted, so it cannot tell you which ones AppendUnescape may safely decode; use GetQuoted or AppendValue for keys whose values you intend to unescape. buf is reused as the result slice when it is large enough, avoiding a [][]byte allocation; pass back a previous result. If a key appears more than once with a non-empty value, the first such occurrence wins; iteration stops once every key has a non-empty value.

Like Get, GetMany reports no syntax errors — it early-stops, so a malformed tail past the settled keys is never reached. Call Validate when you need the record checked.

Each parsed field is matched against keys linearly, which is the fastest arrangement for the handful of keys these lookups are meant for. Measured on a 24-field line, GetMany stays ahead up to roughly ten keys; past that, Iterate with a map keyed by string(k) wins (20 keys: ~505 ns versus ~385 ns).

func GetQuoted

func GetQuoted(data []byte, key string) (val []byte, quoted, found bool)

GetQuoted is Get plus the one fact Get throws away: whether the value was written as a double-quoted token.

That matters because logfmt escape sequences are meaningful only inside quotes. An emitter writes path=C:\Users\bob unquoted and means every byte of it literally, so unescaping a value without knowing how it was written turns \U into U and \n into a newline. GetQuoted is the zero-copy way to decode correctly:

if v, quoted, ok := logfmt.GetQuoted(line, "msg"); ok {
	if quoted && logfmt.NeedsUnescape(v) {
		v = logfmt.AppendUnescape(buf[:0], v)
	}
	// v now aliases line (no copy) or buf (decoded)
}

AppendValue does the same job in one call, at the cost of always copying.

quoted is false for an absent key and for a bare key's implicit "true". Everything else — aliasing, capping, duplicate resolution, the absence of syntax errors — is exactly as described on Get.

func IsBareKey

func IsBareKey(val []byte) bool

IsBareKey reports whether val is the sentinel that Iterate and All substitute for a bare key — one written with no '=' at all, such as the "debug" in "level=info debug" — as opposed to a real value that happens to read "true". The two are otherwise indistinguishable, since both arrive as the bytes "true".

It compares identity, not contents. Values delivered by Iterate and All and values returned by Get, GetQuoted and GetMany all report true for a bare key; AppendValue's result never does, since it copies. A []byte of the caller's own reports false however it reads.

func Iterate

func Iterate(data []byte, fn func(key, val []byte) bool) error

Iterate parses data as a logfmt record and calls fn once for each key/value pair, in order. key and val are sub-slices that alias data — except for a bare key with no '=' (for example "debug", or a trailing token), whose val is a shared constant "true". Treat both as read-only, and copy them if they must outlive the call.

`key=` followed by whitespace is an EMPTY value, and that whitespace still separates the next token: "key= value" yields ("key", "") and then the bare key ("value", "true"). go-logfmt reads it the same way, bar the bare-key sentinel. A quoted value is returned without its surrounding double quotes but is NOT unescaped — backslash escapes are left intact.

val does not record whether it was quoted, and that distinction is the one that decides whether decoding it is correct: escapes mean something only inside quotes, so an unquoted path=C:\Users\bob holds three literal backslashes. Running val through AppendUnescape from inside this callback therefore corrupts every unquoted value that contains one. Use AppendValue, which decodes only what was quoted, or GetQuoted, which hands back the bit.

fn may return false to stop iteration early, in which case Iterate returns nil. Iterate returns a *SyntaxError (which errors.Is matches against ErrBadFormat) if data contains a malformed quoted value, and otherwise nil. Every pair before the fault has already been delivered. It allocates nothing on well-formed input; a returned SyntaxError is the only allocation it can make.

func NeedsUnescape

func NeedsUnescape(raw []byte) bool

NeedsUnescape reports whether raw contains a backslash at all. Values returned by Iterate, All, Get, GetQuoted and GetMany are raw; use this to skip the decode (and its copy) when it is unnecessary.

It is conservative in one direction: a false result guarantees AppendUnescape would not change raw, but a true one does not guarantee it would. The sequences AppendUnescape deliberately keeps verbatim — a malformed \u such as `\uZZZZ`, and a trailing lone backslash — contain a backslash and so report true while decoding to themselves. That costs a needless copy, never a wrong answer.

A true result also does not mean decoding is CORRECT: escapes are meaningful only inside quotes, so check the value was quoted first (GetQuoted), or let AppendValue handle both questions.

func ParseTime

func ParseTime(ts []byte) (time.Time, bool)

ParseTime parses a logfmt timestamp value and reports whether it succeeded. It accepts an RFC3339Nano string, a "2006-01-02 15:04:05.999 -0700 MST" string, or a unix epoch (10 integer digits with an optional fractional part). Trailing delimiters left over from a slightly malformed line (e.g. a stray '}') are trimmed first. On success the returned time is normalized to UTC.

The accepted set is deliberately narrow — these are the shapes real logfmt emitters produce — so several plausible-looking timestamps are rejected rather than guessed at:

• Epochs must have exactly 10 integer digits, which bounds them to 1970-01-01 .. 2286-11-20 and excludes negative (pre-1970) values. Ten digits is a digit COUNT, not a magnitude: a zero-padded "0000000000" is accepted and is the epoch itself, so the lower bound is 1970 rather than the 2001-09-09 that unpadded ten-digit values start at.
• Millisecond and microsecond epochs (13 and 16 digits, as written by JavaScript's Date.now or Java's System.currentTimeMillis) are rejected; the digit count alone cannot distinguish them from a far-future second epoch. Divide them yourself, or parse with strconv and time.UnixMilli.
• Date-only ("2006-01-02"), time-only and other layouts are rejected.

Callers with a known emitter should prefer time.Parse with that emitter's exact layout; ParseTime is for mixed-source logs where the layout is not known up front.

It copies nothing, at any input length. The only allocations it can incur are time.Parse's own, on two shapes this package cannot avoid without hand-rolling the layouts:

• A zone abbreviation the runtime cannot resolve — "+0200 CEST" on a host whose local zone is not CEST — costs 4 allocations for the fabricated Location. This one is an ACCEPTED form: it returns the right instant and ok == true, so "every accepted form is allocation-free" would be wrong. Whether a given abbreviation resolves depends on the host's local zone and on the parsed date, so the same layout can cost 0 or 4.
• A value matching no layout at all costs 5, for the discarded *time.ParseError.

Everything else — epochs, RFC3339Nano, and a numeric-offset-plus-UTC value — is allocation-free, which is what Test_Unit_ParseTime_Allocs pins.

func SplitRecord

func SplitRecord(data []byte) (record, rest []byte)

SplitRecord splits off the first logfmt record from data, returning it and the remainder. A record ends at the first '\n' (a trailing '\r' is trimmed, so CRLF input works); if there is no newline, the whole of data is the record and rest is nil.

This package treats '\n' as ordinary whitespace, so a multi-line buffer handed straight to Iterate parses as one flat run of pairs with no record boundaries — and a lookup can match a key from a later line. Split first:

for len(data) > 0 {
	var rec []byte
	rec, data = logfmt.SplitRecord(data)
	level, _ := logfmt.Get(rec, "level")
	// ...
}

The returned record aliases data and is capped to its length. It may be empty (for a blank line); Iterate and the lookups handle that as a record with no pairs.

func Validate

func Validate(data []byte) error

Validate parses data to completion and reports the first syntax error, or nil if the whole record is well-formed. The lookups deliberately do not do this: they early-stop, so they cannot see a fault past the keys they settled. Use Validate when a record's validity matters, and errors.Is(err, ErrBadFormat) or a *SyntaxError type assertion to inspect the result.

Types

type SyntaxError

type SyntaxError struct {
	// Offset is the byte index in the input at which the fault was detected:
	// the opening quote of an unterminated value, or the offending byte after
	// a closing quote.
	Offset int
	// Reason is a short description of the fault, without position.
	Reason string
}

SyntaxError describes a malformed logfmt record and where the parser gave up. Iterate and Validate return it; errors.Is(err, ErrBadFormat) reports true for any of them, so sentinel comparisons keep working.

func (*SyntaxError) Error

func (e *SyntaxError) Error() string

func (*SyntaxError) Is

func (e *SyntaxError) Is(target error) bool

Is makes every SyntaxError match ErrBadFormat under errors.Is.