percept

README

percept

Captures network traffic and writes what it understands as newline-delimited JSON.

$ percept -r testdata/sample.pcap | wc -l
88

$ percept -r testdata/sample.pcap | jq -r .parser | sort | uniq -c | sort -rn | head -6
   9 smtp
   8 ftp
   6 dns
   6 ssh
   6 dicom
   5 smb

$ percept -r testdata/sample.pcap | jq -c 'select(.parser=="ssh" and .kind=="host_key")'
{"ts":"2026-08-13T17:30:00.087000000Z","parser":"ssh","kind":"host_key",
 "flow_id":"tcp 192.168.1.9:22 192.168.1.20:49314",
 "flow":{"transport":"tcp","src_ip":"192.168.1.9","src_port":22,
         "dst_ip":"192.168.1.20","dst_port":49314},
 "fields":{"type":"ssh-ed25519","bits":256,
           "fingerprint":"SHA256:eTadjyR8ZVEKR7EThJo69ISvyLbv/FZ1oHqdM4wS2zY"}}

Built on gopacket for capture and decoding. Protocol knowledge lives in independent parsers; the engine handles everything else.

Building

libpcap headers are required, and cgo must be enabled.

go install github.com/TOMFoolery-Labs/percept/cmd/percept@latest

Or from a checkout:

make build          # or: go build ./cmd/percept

• macOS: libpcap ships with the Xcode command line tools.
• Debian/Ubuntu: apt install libpcap-dev
• Fedora/RHEL: dnf install libpcap-devel

Running

percept -r capture.pcap                  # replay a capture file
sudo percept -i en0                      # capture live
sudo percept -i en0 -parsers dns,tls     # only the parsers you want
percept -r capture.pcap -o events.ndjson
percept -list-parsers
percept -list-interfaces

Protocols that are not on their conventional port can be rebound without recompiling. -port replaces a parser's default ports and is repeatable:

percept -i en0 -port ssh=tcp/22,tcp/2200 -port http=tcp/80,tcp/8080
percept -list-parsers -port ssh=tcp/2200     # check what took effect

Live capture needs raw packet access: root on macOS, or CAP_NET_RAW on Linux.

If the kernel drops packets because percept cannot keep up, it says so on stderr while the capture runs, and -stats reports libpcap's counters at the end. A capture that quietly loses packets produces output that looks complete and is not, so the warning is not gated behind -v. Capture files keep no such counters, so the capture field is absent for them.

By default the kernel-side BPF filter is derived from the ports the enabled parsers claim, so traffic nothing can parse is dropped before it reaches userspace. -filter overrides it; -filter "" disables it.

Output

One JSON object per line. The envelope is the same for every parser, and fields holds the protocol-specific part.

{
  "ts": "2026-08-13T17:30:00.048000000Z",
  "parser": "http",
  "kind": "request",
  "flow_id": "tcp 192.168.1.20:49312 93.184.216.34:80",
  "flow": {
    "transport": "tcp",
    "src_ip": "192.168.1.20", "src_port": 49312,
    "dst_ip": "93.184.216.34", "dst_port": 80
  },
  "fields": { "txn": 0, "method": "GET", "uri": "/index.html", "...": "..." }
}

flow is oriented the way the message travelled, so a response reads server to client. flow_id is the same for both directions, so requests and responses can be joined.

Encapsulated traffic carries an extra tunnel object. flow always names the hosts actually talking — the inner addresses and ports — and tunnel names the path their traffic took:

"tunnel": {
  "src_ip": "203.0.113.1", "dst_ip": "203.0.113.2",
  "encapsulation": ["UDP", "VXLAN", "Ethernet"]
}

Built-in parsers

Parser	Ports	Emits
dhcp	udp 67, 68	discover, offer, request, ack, nak, decline, release, inform, bootp
dhcpv6	udp 546, 547	solicit, advertise, request, reply, renew, release, …
dicom	tcp 104, 11112	associate-request, associate-accept, c-store-rq, c-find-rq, …
dns	udp 53, 5353, 5355; tcp 53	query, response
ftp	tcp 21	command, reply
hl7	tcp 2575	HL7 v2 message code: adt, oru, orm, ack, …
http	tcp 80, 8000, 8080, 8888	request, response
ike	udp 500, 4500	ike-sa-init, ike-auth, create-child-sa, informational, isakmp
kerberos	udp 88; tcp 88	as-req, as-rep, tgs-req, tgs-rep, ap-req, error
ldap	udp 389; tcp 389, 3268	bind-request, search-request, search-result-entry, bind-response, …
nbns	udp 137	query, registration, release, and their -response forms
ntp	udp 123	request, response, message
radius	udp 1812, 1813, 1645, 1646	access-request, access-accept, access-reject, accounting-request, …
rdp	tcp 3389	connection-request, connection-confirm
smb	tcp 445, 139	negotiate, session-setup, tree-connect, create, ioctl, …
smtp	tcp 25, 587, 2525	command, reply, message
snmp	udp 161, 162	get-request, response, trap, get-bulk-request, …
ssh	tcp 22, 2222	version, kexinit, host_key, negotiated, disconnect
stun	udp 3478, 3479	binding-request, binding-success, allocate-request, …
syslog	udp 514; tcp 514, 601	message (RFC 5424 and RFC 3164)
telnet	tcp 23	negotiate, subnegotiate
tftp	udp 69	read-request, write-request, error, option-ack
tls	tcp 443, 8443, 993, 995, 465, 636, 2762, 3269	client_hello, server_hello

hl7 reports metadata only — who sent what kind of message to whom, and whether it was accepted. Patient data lives in segments the parser never reads: it retains the routing header and the acknowledgement, and for every other segment keeps only the three-character identifier before discarding the rest unread. That is a property of how the message is read, not a list of fields someone remembered to omit. HL7 ports are chosen per site, so expect -port hl7=tcp/2100.

kerberos reports every domain authentication in the clear: which account asked for which service, from where, with what encryption, and why it was refused. Ticket contents and authenticators are ciphertext and are described only by their encryption type.

ldap reports who asked for what, never the secret. A simple bind sends the password in cleartext; percept records that the bind happened and the name it was for, and never reads the credential. Search result entries are reported by distinguished name and attribute names — not values, which is where password hashes live. Ports 636 and 3269 are LDAP inside TLS and belong to the tls parser, which binds them.

snmp does not report the community string, which in v1 and v2c is the only credential and is sent in cleartext. It names one only when it is a well-known vendor default — a device still answering to public is the finding, and that value is not a secret. A v3 username is reported; it identifies who is polling rather than authenticating them.

rdp reads the connection negotiation, which is the only part not encrypted. It reports the account the client names in its routing cookie and which security protocols each end will accept — a server that settles for bare RDP has neither TLS nor network level authentication.

NTLM is decoded wherever it appears rather than being a parser of its own: http reports it from Authorization and WWW-Authenticate headers, naming the domain account and the workstation it came from. The challenge responses are never read, being crackable offline and relayable as they stand.

smb reports the commands that describe a session — who authenticated, which shares they attached to, which files they opened — and not the per-block reads and writes, which on a busy connection would bury everything else by orders of magnitude. A failed response is reported whatever its command. SMB 3 encryption and 3.1.1 compression are named rather than skipped silently, since the move from cleartext to encrypted is itself worth knowing, and an SMB1 peer is recorded because its presence is the finding.

dicom reports the association negotiation — which modality talked to which archive, what operations were agreed, what software each end runs — and the DIMSE commands that follow. It never reads a data set. A DICOM message splits across presentation data values, and a bit in each header says whether it carries a command or a data set; commands say what is happening and data sets are where patient identifiers and pixels live. The parser steps over the second by its length without examining it. Port 2762 is DICOM inside TLS and belongs to the tls parser.

smtp, ftp, and telnet are three cleartext protocols still in use, and each carries a credential on the wire. All three report that authentication happened and who, never the secret: FTP's PASS is reported by length only, SMTP's AUTH PLAIN yields the account and not the password, and AUTH LOGIN yields neither — its two base64 lines are distinguished only by position, and mistaking one for the other would log a password as a username. Message bodies and terminal sessions are stepped over unread for the same reason.

ftp records the data endpoint from PORT, EPRT, PASV, and EPSV, though percept cannot yet follow the data channel — that needs dynamic port binding. EPSV names only a port, so the address comes from the control connection; it is the form curl, lftp, and FileZilla send by default.

radius is the fourth protocol here that carries a credential, and it is handled the same way. User-Password is obfuscated with the shared secret rather than encrypted, so anyone holding the secret reads it; the parser records that a password was presented and never its value, and treats CHAP and the EAP exchange identically. What it does report is the login record: the account, the realm, the calling station — a MAC address on wireless — and the equipment that asked, whose called-station identifier carries the SSID.

dhcpv6 pairs with dhcp for dual-stack networks. The client identifier is the piece worth having: DHCPv6 replaces the hardware address field with a DUID, and the two common forms still contain the MAC, so a v6-only network is no less inventoried than a v4 one. Addresses and delegated prefixes are nested inside the identity association rather than being options of their own, which is where a parser that only walks the top level finds nothing.

nbns is the oldest name resolution a Windows network has and the one still running when nothing else works. Its queries are broadcast, so a single listener learns every machine's NetBIOS name and which of them claim to be domain controllers. It is also where two well-known intrusions are visible: a name conflict is what a spoofed registration produces, and a broadcast query for a name that does not exist is what a responder answers to harvest credentials.

ike reports the opening of an IPsec negotiation, which is the last thing readable before everything after it is encapsulated — a site-to-site tunnel appearing where none was before is visible here and nowhere else. Vendor identifiers name the implementation at each end. Once the exchange is sealed the parser says so and reads no further; key exchange data, nonces, and authentication payloads are stepped over even in the cleartext phase.

stun is how anything behind NAT learns its own public address, so a binding response ties an internal host to the endpoint the outside world sees. The software attribute identifies the application making the call. Message integrity is a keyed digest and is never read.

tftp has no authentication of any kind, so every transfer is one anybody on the segment could have made — it is mostly seen where something boots over the network. Requests and errors are reported; the data and acknowledgement packets are not, since a transfer is one of each per block and the request already says what is moving.

Parsers still to build — MSRPC — are in ROADMAP.md.

Architecture

capture  ->  engine  ->  parser  ->  output
(libpcap)    decode      protocol    NDJSON
             dispatch    fields
             reassemble

internal/engine reads packets, decodes the link and network layers, and routes each one by port. What happens next depends on the transport, and that split is the central design decision:

• UDP goes straight to a PacketParser, which sees one datagram's payload. Correct for protocols where a message never spans packets — NTP, DNS, DHCP.
• TCP goes through gopacket/reassembly and reaches a StreamParser as an ordered byte stream per direction. HTTP messages have no relationship to segment boundaries, so anything less would fail on ordinary traffic.

A parser may be both. dns and syslog implement both interfaces and bind the same port on each transport, sharing one decoder, so a message looks the same whether it arrived as a datagram or inside a stream — the transport shows up in the event's flow rather than in its fields.

Each direction of a TCP connection gets its own goroutine reading from an io.Pipe. The pipe is unbuffered, which means a slow parser applies backpressure to capture rather than growing a queue, and delivered bytes are consumed before the assembler reuses its buffer.

Parsers never write output. They fill in a struct and call Emit; the engine stamps it with timing and flow metadata and forwards it to a sink.

Adding a parser

Implement Name, Bindings, and one of the two parse methods, then add one line to internal/parsers/parsers.go.

A datagram protocol:

type Parser struct{}

func (*Parser) Name() string               { return "syslog" }
func (*Parser) Bindings() []parser.Binding { return parser.UDP(514) }

func (*Parser) ParsePacket(pc *parser.PacketContext, payload []byte) error {
	msg, err := decode(payload)
	if err != nil {
		return err // counted as a parse error; capture continues
	}
	pc.Emit("message", msg)
	return nil
}

A stream protocol:

func (*Parser) Bindings() []parser.Binding { return parser.TCP(25) }

func (*Parser) NewStream(sc *parser.StreamContext) parser.StreamHandler {
	return &conn{sc: sc}
}

// Handle is called once per direction, each on its own goroutine, with a
// reader over that direction's reassembled bytes. Read until io.EOF.
func (c *conn) Handle(dir parser.Direction, r io.Reader) {
	s := bufio.NewScanner(r)
	for s.Scan() {
		c.sc.Emit(dir, "command", parse(s.Text()))
	}
}

func (c *conn) Close() {}

Notes that matter in practice:

• Emitted structs are marshalled to JSON, so tag the fields. Use omitzero on optional ones, so a missing field means the sender did not supply it rather than supplying zero.
• The registry rejects two parsers claiming the same port, and rejects a PacketParser bound to TCP or a StreamParser bound to UDP. Wiring mistakes fail at startup, not silently at runtime.
• Emit may be called from either direction's goroutine. Guard any state the two share.
• A panic inside a parser is recovered and costs that one connection.
• Sanitize text that came from the peer. Hostnames and version strings are attacker-controlled and end up in logs and dashboards.

Do not block inside Handle waiting for the other direction. The assembler is single-threaded, so the bytes you are waiting for arrive through the same assembler that is currently blocked feeding you; waiting indefinitely deadlocks capture. When a result needs both directions, compute it in Close, which runs after both have finished — that is how the ssh parser reports negotiated algorithms. When a result is needed mid-stream, bound the wait, as the http parser does to pair a response with its request.

Development

make build    # go build ./cmd/percept
make test     # go test ./...
make race     # go test -race -count=2 ./...
make vet
make fmt
make sample   # regenerate testdata/sample.pcap

The engine tests build synthetic frames in memory and run them through the real pipeline, including reassembly, so no capture privileges or fixtures are needed.

testdata/sample.pcap is a small generated capture for exercising the command line. It is committed, and make sample regenerates it from internal/gensample when a new parser needs traffic to work with.

Limitations

• HTTP/2 and QUIC are not parsed. HTTP/2 shares ports with HTTP/1.x but uses binary framing; QUIC is UDP and encrypted.
• Parsers are selected by port. -port moves a parser to whatever port a site actually uses, but nothing identifies a protocol by looking at its bytes, so traffic on an unexpected port is invisible.
• Stream events carry the capture time of the most recent segment on the connection, so a message assembled from several segments is stamped with the last one.
• The two directions of a TCP connection are parsed concurrently, so events from one connection may be written out of order relative to each other. Sort by ts if order matters.
• Fragmented IP datagrams are not reassembled, so a fragmented message is missed.
• Tunnelled traffic is attributed to the hosts inside the tunnel, but the kernel filter cannot see inside one. The BPF filter derived from the enabled parsers matches the outer headers, so encapsulated traffic is discarded before percept sees it — pass -filter "" (or a filter of your own, such as proto gre or udp port 4789) to capture it.

ROADMAP.md tracks what is planned about all of this.

Releases

CHANGELOG.md records what changed in each version, including the output changes that matter to anyone parsing the JSON.

Contributing

ROADMAP.md lists the parsers that are wanted and why. CONTRIBUTING.md covers setup and the two mistakes that are easy to make in a parser.

License

Apache License 2.0. Copyright 2026 TOMfoolery Labs LLC. See LICENSE.