pii

README

pii

Finds personal data in text, in pure Go with no dependencies (standard library only, CGO_ENABLED=0). It looks for:

• e-mail addresses
• phone numbers
• IBANs
• French social security numbers (NIR)
• payment card numbers
• IPv4 and IPv6 addresses

It was written for the go-llmflow gateway's PII guardrail, which masks prompts before they reach the model and masks completions before they reach the client. The package itself only reports what it found and where; the caller decides what to do with it.

s := pii.New()                       // every kind; pii.New(pii.IBAN, pii.Card) for some
ms := s.Find(text)                   // []pii.Match{Kind, Start, End}, ordered, no overlap
clean := pii.Mask(text, ms, pii.Placeholder) // "IBAN [IBAN], mail [EMAIL]"

What a match means

A recogniser that has a check digit uses it. A match is therefore a value that could exist, not just a string with the right number of digits.

kind	recognised by	checked by
iban	2 letters, 2 digits, then alphanumerics; single spaces allowed (the printed groups of four)	ISO 13616: the exact length for the country, and mod 97 = 1 after moving the first four characters to the end with A=10 … Z=35
nir	sex 1/2/3/4/7/8, year, month, department (2A/2B for Corsica), commune, order number, 2-digit key; spaces allowed	key = 97 − (the 13 digits mod 97), with 2A read as 19 and 2B as 18 (INSEE); month 01–13, 20–42 or 50–99
card	13–19 digits, single spaces or dashes allowed	Luhn (ISO/IEC 7812-1 annex B), plus a known network's prefix and length: Visa, Mastercard, American Express, Discover, JCB, Diners
email	a practical subset of RFC 5322 addr-spec: a dot-atom local part, LDH domain labels, an alphabetic top-level label	local part ≤ 64 octets and address ≤ 254 (RFC 5321 §4.5.3.1). A git@host:path remote is not treated as a mailbox
phone	E.164 (+, at most 15 digits) and the French plan (0 + 9 digits, +33 / 0033, the optional (0)), with space, dot or dash separators	format only, since phone numbers have no checksum
ipv4	dotted decimal, not inside a longer dotted run (version numbers, OIDs)	net/netip (which rejects octets with leading zeros)
ipv6	the three text forms of RFC 4291 §2.2, including a trailing IPv4	net/netip; zones are not accepted

When two candidates overlap, the one that starts first wins, then the longer one, then the checksum-validated kind.

MaxLen (254 bytes) bounds the length of any match. A caller masking a stream can hold back the last MaxLen bytes, and will then never release the start of a match whose end it has not seen yet.

False positives, measured

testdata/benign.txt contains no personal data by construction. It is 45 lines of text a model reads and writes every day, chosen to look like the real thing:

• versions, OIDs, dates, timestamps, UUIDs, hashes;
• ISBN, EAN, SIREN and SIRET numbers;
• prices, coordinates, MAC addresses;
• C++ and Rust ::, an SSH git remote;
• a card number that fails Luhn, a NIR and an IBAN with wrong keys;
• invalid IP addresses.

TestFalsePositives fails if any count rises above what was measured when the recognisers were written:

kind	false positives
iban, nir, card, email, ipv4, ipv6	0
phone	1: ref 0123456789, a valid French number in every respect except its meaning

Phone numbers are the one kind that cannot be checked. A guardrail that masks them will also mask 10-digit references starting with 0.

Security

The input is untrusted (prompts and model output). Every regular expression runs on Go's RE2 engine, in linear time. FuzzFind checks on arbitrary input that:

• matches are ordered and within bounds;
• every checksum-validated match re-validates on its own;
• masking twice gives the same result as masking once.

The tests also run with GOARCH=386.

Performance

BenchmarkFind scans 64 KiB of the benign corpus. The comparison against Microsoft Presidio, the reference in this domain, has not been made yet: the machine was too loaded to measure (load average 81).

References

• ISO 13616 (IBAN) and the SWIFT registry lengths: https://en.wikipedia.org/wiki/International_Bank_Account_Number
• NIR structure and key, Corsica: https://fr.wikipedia.org/wiki/Num%C3%A9ro_de_s%C3%A9curit%C3%A9_sociale_en_France
• ISO/IEC 7812-1 annex B (Luhn) and network prefixes: https://en.wikipedia.org/wiki/Luhn_algorithm, https://en.wikipedia.org/wiki/Payment_card_number
• ITU-T E.164: https://en.wikipedia.org/wiki/E.164; the French numbering plan: https://en.wikipedia.org/wiki/Telephone_numbers_in_France
• RFC 5322 §3.2.3, §3.4.1; RFC 5321 §4.5.3.1: https://www.rfc-editor.org/rfc/rfc5322, https://www.rfc-editor.org/rfc/rfc5321
• RFC 4291 §2.2: https://www.rfc-editor.org/rfc/rfc4291

Licence

See LICENSE.

Documentation

Overview

Package pii finds personal data in text: e-mail addresses, phone numbers, IBANs, French social security numbers (NIR), payment card numbers and IP addresses.

Every recogniser that has a check digit uses it — mod-97 for IBAN (ISO 13616) and NIR (INSEE), Luhn for cards (ISO/IEC 7812-1 annex B) — and IP addresses are confirmed by net/netip, so a match is a value that could be real, not merely something with the right number of digits. Phone numbers and e-mail addresses have no checksum; they are matched on their format (E.164 and the French national plan; a practical subset of RFC 5322).

The package only finds; what to do with a match is the caller's choice. Mask replaces matches, and MaxLen bounds the length of any match so that a caller masking a stream knows how much text to hold back.

Index

• Constants
• Variables
• func Luhn(s string) bool
• func Mask(text string, matches []Match, repl func(Match) string) string
• func Placeholder(m Match) string
• func ValidCard(s string) bool
• func ValidIBAN(s string) bool
• func ValidNIR(s string) bool
• type Kind
  • func ParseKind(s string) (Kind, bool)
• type Match
• type Scanner
  • func New(kinds ...Kind) *Scanner
  • func (s *Scanner) Find(text string) []Match

Constants

const MaxLen = 254

MaxLen is the longest match, in bytes, of any kind: an e-mail address of at most 254 bytes (RFC 5321 §4.5.3.1.3 path limit). A streaming caller that holds back the last MaxLen bytes never releases the beginning of a match whose end it has not seen yet.

Variables

var Kinds = []Kind{IBAN, NIR, Card, Email, IPv6, IPv4, Phone}

Kinds lists every kind, in the order overlapping matches are preferred when they start at the same place and have the same length: checksum- validated kinds first.

Functions

func Luhn

func Luhn(s string) bool

Luhn reports whether the digit string s passes the Luhn check.

func Mask

func Mask(text string, matches []Match, repl func(Match) string) string

Mask returns text with each match replaced by repl(m). The matches must be ordered and must not overlap, as Find returns them.

func Placeholder

func Placeholder(m Match) string

Placeholder is the default replacement: the kind in capitals between brackets, "IBAN".

func ValidCard

func ValidCard(s string) bool

ValidCard reports whether s (digits only) has a known network's prefix and length and a valid Luhn check digit.

func ValidIBAN

func ValidIBAN(s string) bool

ValidIBAN reports whether s, without spaces, is an IBAN of its country's length whose ISO 13616 check (mod 97 = 1) holds.

func ValidNIR

func ValidNIR(s string) bool

ValidNIR reports whether s (15 characters, no spaces) is a NIR whose key matches.

Types

type Kind

type Kind string

Kind is a type of personal data.

const (
	Email Kind = "email"
	Phone Kind = "phone"
	IBAN  Kind = "iban"
	NIR   Kind = "nir"
	Card  Kind = "card"
	IPv4  Kind = "ipv4"
	IPv6  Kind = "ipv6"
)

func ParseKind

func ParseKind(s string) (Kind, bool)

ParseKind returns the Kind named s.

type Match

type Match struct {
	Kind       Kind
	Start, End int
}

Match is one finding: text[Start:End] is a value of the kind.

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner finds the kinds it was made for.

func New

func New(kinds ...Kind) *Scanner

New returns a Scanner for the given kinds, or every kind when none is given. An unknown kind is ignored by Find; use ParseKind to validate names.

func (*Scanner) Find

func (s *Scanner) Find(text string) []Match

Find returns the matches in text, ordered by position and without overlap. Where two candidates overlap, the one that starts first wins, then the longer one, then the one earlier in Kinds.