cidr

README

cidr

cidr is a fast, immutable IPv4/IPv6 lookup library for Go: build a set of CIDR prefixes once, then answer two questions with zero per-query allocation —

• membership — is this address in the set? (Set.Contains → yes/no)
• value lookup — what is attached to the most-specific prefix covering this address? (Table[V].Lookup → yes + data, e.g. the owning AS number and organisation)

It is built for static, build-once-query-many workloads: blocklists and allowlists, IP-to-ASN and geo tables, parked/sinkhole ranges, firewall feeds. For those, a sorted range array is both faster and an order of magnitude smaller than a prefix trie (see Architecture).

No dependency beyond the Go standard library.

Data flow

  SOURCES                     BUILD (once)                QUERY (many, concurrent)
  ───────                     ────────────                ────────────────────────
  spec text ─┐
  refs JSON ─┤  Load*/Parse*   Builder            ┌─ Set.Contains(addr)  → yes/no
  <cidr> …  ─┼──────────────►  TableBuilder[V] ───┤
  lo–hi     ─┘  Add / AddPrefix  │ .Freeze()      └─ Table[V].Lookup(addr) → value + ok
                / AddRange       ▼
                          immutable, sorted net/netip range arrays
                          (one binary search · 0 alloc · lock-free reads)

Documentation

• Start here — Introduction (what the name means and the one idea it rests on) and the Executive summary (what this is and why it exists)
• Deep dive — Architecture: the range-array design, longest-prefix match, benchmarks, and the trie trade-off
• Operations — User guide: the API, the spec and refs formats, data sources, the CLI, and day-2 refresh
• Examples — example/README: library, HTTP, Unix-socket, and MCP integrations

Features

• Two structures, one job each: Set (membership) merges overlaps for the smallest form; Table[V] (value lookup) resolves nesting into longest-prefix match at build time.
• Zero-allocation queries — a single binary search over a contiguous, cache-friendly array. No pointer-chasing, no interface dispatch.
• IPv4 and IPv6 through one net/netip code path.
• Generic values — Table[V] carries any V: a struct, an Info, or a compact uint64 via EncodeASN/DecodeASN.
• Immutable and concurrent-safe once built (Builder/TableBuilder → Freeze), so many goroutines can query without a lock.
• Loaders for any feed — LoadASN/LoadSet read a <cidr> <ASN> <org> stream; LoadFunc takes any custom <cidr> data... format; AddRange ingests start/end address ranges (iptoasn, RIR delegated files) directly.

Quick start

package main

import (
	"fmt"
	"net/netip"

	"github.com/netstar-labs/cidr"
)

func main() {
	// Membership — yes/no.
	b := cidr.NewBuilder()
	b.AddPrefix("192.0.2.0/24")
	b.AddPrefix("2001:db8::/32")
	set := b.Freeze()
	fmt.Println(set.Contains(netip.MustParseAddr("192.0.2.10"))) // true

	// Value lookup — yes + data, most-specific prefix wins.
	tb := cidr.NewTableBuilder[string]()
	tb.AddPrefix("1.1.1.0/24", "AS13335 Cloudflare")
	tb.AddPrefix("1.1.1.128/25", "AS13335 customer sub-block") // nested, more specific
	table := tb.Freeze()

	who, ok := table.Lookup(netip.MustParseAddr("1.1.1.200"))
	fmt.Println(who, ok) // "AS13335 customer sub-block" true
}

The spec format

ParseSpec, LoadSet, and LoadASN read a whitespace-delimited text stream, one prefix per line — the shape of a typical IP-to-ASN table:

# comments and blank lines are ignored
10.0.0.0/8
1.1.1.0/24     13335 Cloudflare, Inc.
8.8.8.0/24     AS15169 Google LLC
203.0.113.7    64500 Example Org        # a bare address is a /32 host route
2001:db8::/32  64502 Documentation v6

The ASN (optionally AS-prefixed) and organisation name are optional; a plain list of CIDRs is a valid degenerate case.

set, table, err := cidr.LoadASN(specReader) // membership Set + value Table[Info]

The same lines also come wrapped in the common refs JSON envelope ({"name","version","list":[...]}, e.g. refs.netstar.dev) — cidr.LoadRefsASN / LoadRefsSet / ParseRefs read it, and the cidr CLI's -spec accepts either form.

For non-ASN feeds, LoadFunc builds a Table[V] of any value type from a per-line parse callback, and AddRange ingests start/end ranges. Where to pull real ASN/geo data (MaxMind, iptoasn, CAIDA, …) and how to convert it is in the user guide.

Command-line tool

cmd/cidr is a standalone lookup tool over a spec file. Addresses come from the arguments, or from stdin (one per line) when none are given.

go install github.com/netstar-labs/cidr/cmd/cidr@latest

cidr -spec asn.txt 1.1.1.200 8.8.8.8              # NDJSON, one object per address
printf '1.1.1.1\n9.9.9.9\n' | cidr -spec asn.txt -brief
cidr -spec block.txt -match < ips.txt             # filter: print only listed addresses

Flags: -spec FILE (required), -brief (terse aligned lines), -match (grep-like filter; exit 1 if nothing matched), -quiet (no stderr tally), -version. A run tally lands on stderr.

Cross-compile a version-stamped static binary with build/cidr (linux/amd64, with an optional scp install to a host).

More tools

cmd/ ships six more programs — see the cmd README:

• ipfold — fold an unorganized IP list into the minimal CIDR set (10.0.0.12, .13, .14, .15 → 10.0.0.12/30); built for 100M+ addresses (ipfold < ips.txt).
• iptoasn, mm-geolite2-asn, mm-dbip — fetch a provider's IP-to-ASN/geo table and write the cidr spec, with an optional systemd generator (see data sources).
• mmdb-write — compile a cidr spec into a MaxMind DB (.mmdb) file, supporting both ASN (GeoIP2-ASN-compatible) and Country (GeoLite2-Country-compatible) output schemas. Pipe any data generator directly: iptoasn | mmdb-write -o asn.mmdb.
• mmdb-build-countries — build the country nomenclature dataset (from Wikidata) that mmdb-write uses for Country DBs.

Examples

Each is a self-contained main.go with no dependency beyond the standard library and this package — see example/.

Example	What it shows	Run
library/	the package used directly: Set, Table[Info], the encoded-uint64 path, nesting, IPv6	go run ./example/library
http/	an HTTP REST API — /contains, /lookup, and an NDJSON batch stream	go run ./example/http -addr :8080
unix/	a Unix-domain-socket line service: address in, JSON line out	go run ./example/unix -socket /tmp/cidr.sock
mcp/	an MCP (Model Context Protocol) stdio server exposing the set as agent tools	go run ./example/mcp

Performance

Both queries are a single binary search with no allocation (Apple M2 Pro, Go 1.25):

operation	time	allocations
Set.Contains	~88 ns/op	0
Table.Lookup	~88 ns/op	0

Against a path-compressed prefix trie on the same data, the range array is roughly 2× faster on IPv4, allocation-free where the trie allocates per query, and ~8–60× smaller in memory; the gap widens with set size. The one thing the array gives up — cheap incremental insert/remove of single prefixes under live queries — is not what a static, wholesale-rebuilt set needs. The full analysis, including when a trie is the right choice, is in docs/architecture.md.

Layout

The package is five root .go files (plus the cmd/, example/, and build/ trees):

File	Purpose
cidr.go	Package doc; ParsePrefix; the Set membership type, its Builder, and the range-merge that fuses overlaps at Freeze
table.go	Table[V] longest-prefix-match value table and TableBuilder[V]; the build-time line sweep that resolves nesting; EncodeASN/DecodeASN
input.go	Text/JSON loaders — ParseSpec, LoadSet, LoadASN, LoadFunc, the Refs envelope (ParseRefs/LoadRefs*), and the Info/SpecEntry types
range.go	Range ingest — AddRange, RangePrefixes/AppendRangePrefixes, and the u128 range-to-CIDR arithmetic behind them
mmdb.go	Stdlib-only MaxMind DB (.mmdb) reader — OpenMMDB/OpenMMDBBytes, (*MMDB).Lookup/Metadata/Close; safe for concurrent lookups

Install

go get github.com/netstar-labs/cidr

Requires Go 1.25 or newer.

License

See LICENSE.

Documentation

Overview

Package cidr provides fast, immutable IPv4/IPv6 lookups over a set of CIDR prefixes, built once and queried concurrently with zero per-query allocation.

Two structures cover the two questions callers ask:

• Set answers membership — "is this address in any added prefix?" (yes/no). Overlapping prefixes are merged, so it is the smallest, fastest form.

• Table[V] answers value lookup — "what value is attached to the most specific prefix covering this address?" (yes + data). It resolves nested prefixes into disjoint segments at build time (longest-prefix match), so a query is a single binary search that returns the value directly.

Both are backed by sorted, contiguous arrays of net/netip addresses: no pointer-chasing, no interface dispatch, and a flat memory layout that serialises trivially. For the static, build-once-query-many workloads this package targets — blocklists, allowlists, IP-to-ASN and geo tables — a sorted range array is both faster and an order of magnitude smaller than a prefix trie; see docs/architecture.md for the measurements and the trade-off.

The package has no dependency beyond the Go standard library.

Build with a Builder / TableBuilder, then Freeze to an immutable value:

b := cidr.NewBuilder()
b.AddPrefix("192.0.2.0/24")
set := b.Freeze()
set.Contains(netip.MustParseAddr("192.0.2.10")) // true

Index

• func AppendRangePrefixes(dst []netip.Prefix, lo, hi netip.Addr) []netip.Prefix
• func BuildASN(entries []SpecEntry) (*Set, *Table[Info])
• func DecodeASN(v uint64) (asn uint32, orgID uint32, flags uint8)
• func EncodeASN(asn uint32, orgID uint32, flags uint8) uint64
• func LoadASN(r io.Reader) (*Set, *Table[Info], error)
• func LoadRefsASN(r io.Reader) (*Set, *Table[Info], error)
• func ParsePrefix(s string) (netip.Prefix, error)
• func RangePrefixes(lo, hi netip.Addr) []netip.Prefix
• type Builder
  • func NewBuilder() *Builder
  • func (b *Builder) Add(p netip.Prefix)
  • func (b *Builder) AddPrefix(s string) error
  • func (b *Builder) AddRange(lo, hi netip.Addr)
  • func (b *Builder) Freeze() *Set
• type Info
• type MMDB
  • func OpenMMDB(path string) (*MMDB, error)
  • func OpenMMDBBytes(data []byte) (*MMDB, error)
  • func (m *MMDB) Close() error
  • func (m *MMDB) Lookup(ip net.IP) (record map[string]any, ok bool, err error)
  • func (m *MMDB) Metadata() map[string]any
• type Refs
  • func ParseRefs(r io.Reader) (*Refs, error)
  • func (rf *Refs) Entries() []SpecEntry
• type Set
  • func LoadRefsSet(r io.Reader) (*Set, error)
  • func LoadSet(r io.Reader) (*Set, error)
  • func (s *Set) Contains(a netip.Addr) bool
  • func (s *Set) Len() int
• type SpecEntry
  • func ParseSpec(r io.Reader) ([]SpecEntry, error)
• type Table
  • func LoadFunc[V any](r io.Reader, parse func(fields []string) (netip.Prefix, V, bool)) (*Table[V], error)
  • func (t *Table[V]) Lookup(a netip.Addr) (V, bool)
• type TableBuilder
  • func NewTableBuilder[V any]() *TableBuilder[V]
  • func (b *TableBuilder[V]) Add(p netip.Prefix, v V)
  • func (b *TableBuilder[V]) AddPrefix(s string, v V) error
  • func (b *TableBuilder[V]) AddRange(lo, hi netip.Addr, v V)
  • func (b *TableBuilder[V]) Freeze() *Table[V]

Functions

func AppendRangePrefixes

func AppendRangePrefixes(dst []netip.Prefix, lo, hi netip.Addr) []netip.Prefix

AppendRangePrefixes appends the minimal CIDR cover of [lo, hi] to dst and returns the extended slice, reusing dst's capacity. Use it in hot loops (folding millions of ranges) to avoid a fresh allocation per call; RangePrefixes is the dst==nil convenience wrapper.

func BuildASN

func BuildASN(entries []SpecEntry) (*Set, *Table[Info])

BuildASN compiles already-parsed entries into a membership Set plus an LPM value Table[Info] in one pass — the "entries → Set + Table" step LoadASN wraps, exported so a caller that parsed the entries itself (to count or filter them first) can build both without re-parsing.

func DecodeASN

func DecodeASN(v uint64) (asn uint32, orgID uint32, flags uint8)

DecodeASN unpacks a value produced by EncodeASN.

func EncodeASN

func EncodeASN(asn uint32, orgID uint32, flags uint8) uint64

EncodeASN packs an autonomous-system answer into a uint64:

bits [31:0]  ASN            full 4-byte ASN space
bits [55:32] org id (24)    index into a caller-side []string (16.7M orgs)
bits [63:56] flags (8)      caller-defined (e.g. bogon / anycast / rir)

It is the compact "encoded uint64" value for a Table[uint64]; DecodeASN recovers the fields, and the org id can index a side table for the full name.

func LoadASN

func LoadASN(r io.Reader) (*Set, *Table[Info], error)

LoadASN reads a spec from r and compiles both a membership Set (yes/no) and an LPM value Table[Info] (yes+data). One pass builds both, so a caller can answer "is this address listed?" and "which AS/org owns it?" from the same input.

func LoadRefsASN

func LoadRefsASN(r io.Reader) (*Set, *Table[Info], error)

LoadRefsASN reads a refs JSON envelope and compiles a membership Set plus an LPM value Table[Info] — the JSON analogue of LoadASN.

func ParsePrefix

func ParsePrefix(s string) (netip.Prefix, error)

ParsePrefix parses "10.0.0.0/8" or "2001:db8::/32", or a bare address ("1.2.3.4", "::1") promoted to a host route (/32 or /128). The returned prefix is masked to its network address.

func RangePrefixes

func RangePrefixes(lo, hi netip.Addr) []netip.Prefix

RangePrefixes decomposes the inclusive interval [lo, hi] into the minimal, ordered set of CIDR prefixes that exactly cover it (the standard range-to-CIDR split). It returns nil for an invalid range: mismatched families or hi < lo.

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder accumulates prefixes for a Set. Not safe for concurrent use.

func NewBuilder

func NewBuilder() *Builder

NewBuilder returns an empty membership-set builder.

func (*Builder) Add

func (b *Builder) Add(p netip.Prefix)

Add inserts a prefix (masked to its network) into the set under construction.

func (*Builder) AddPrefix

func (b *Builder) AddPrefix(s string) error

AddPrefix parses s (a CIDR or bare address, see ParsePrefix) and adds it, returning any parse error.

func (*Builder) AddRange

func (b *Builder) AddRange(lo, hi netip.Addr)

AddRange adds the inclusive address interval [lo, hi] to the membership set. Both ends must be the same family and lo <= hi; an invalid range is ignored. This is the natural entry point for range-based feeds (start/end columns) such as iptoasn.com or the RIR delegated files.

func (*Builder) Freeze

func (b *Builder) Freeze() *Set

Freeze compiles the accumulated prefixes into an immutable Set.

type Info

type Info struct {
	Prefix string `json:"prefix"`
	ASN    uint32 `json:"asn,omitempty"`
	Org    string `json:"org,omitempty"`
}

Info is the value attached to a prefix loaded from a spec: the origin AS number and organisation name, alongside the matched prefix in CIDR form. It is the value type of the Table that LoadASN builds, and encodes to compact JSON for the service examples.

type MMDB

type MMDB struct {
	// contains filtered or unexported fields
}

MMDB is an opened MaxMind DB held in memory, safe for concurrent lookups.

func OpenMMDB

func OpenMMDB(path string) (*MMDB, error)

OpenMMDB reads and parses the MaxMind DB at path.

func OpenMMDBBytes

func OpenMMDBBytes(data []byte) (*MMDB, error)

OpenMMDBBytes parses an in-memory MaxMind DB. The slice is retained (not copied) and must not be mutated while the MMDB is in use.

func (*MMDB) Close

func (m *MMDB) Close() error

Close is a no-op: the database is a plain in-memory buffer, reclaimed by GC once the MMDB is unreferenced. It exists for API symmetry (and a future mmap backend) and does NOT release the buffer, so it is safe to call concurrently with in-flight Lookups.

func (*MMDB) Lookup

func (m *MMDB) Lookup(ip net.IP) (record map[string]any, ok bool, err error)

Lookup returns the record for ip as a decoded map (the shape depends on the database's schema: ASN, country, or any custom fields the producer packed). ok is false when ip has no record.

func (*MMDB) Metadata

func (m *MMDB) Metadata() map[string]any

Metadata returns the decoded metadata map (node_count, record_size, ip_version, database_type, description, build_epoch, …).

type Refs

type Refs struct {
	Name    string   `json:"name"`
	Version int      `json:"version"` // yyyymmdd
	List    []string `json:"list"`
}

Refs is the common refs resource envelope — a named, versioned list of spec lines — as published at refs.netstar.dev and similar feeds:

{"name": "parked", "version": 20260705, "list": ["1.2.3.0/24", ...]}

Each list entry uses the same grammar as a ParseSpec line ("<cidr> [ASN] [org...]"), so a bare CIDR list and an IP-to-ASN list are both valid bodies.

func ParseRefs

func ParseRefs(r io.Reader) (*Refs, error)

ParseRefs decodes a refs JSON envelope from r.

func (*Refs) Entries

func (rf *Refs) Entries() []SpecEntry

Entries parses the refs list into spec entries, skipping blank/comment lines and any entry whose first field is not a valid prefix.

type Set

type Set struct {
	// contains filtered or unexported fields
}

Set is an immutable IPv4/IPv6 membership set. The zero Set is a valid empty set. Safe for concurrent use once built.

func LoadRefsSet

func LoadRefsSet(r io.Reader) (*Set, error)

LoadRefsSet reads a refs JSON envelope and compiles a membership Set — the JSON analogue of LoadSet.

func LoadSet

func LoadSet(r io.Reader) (*Set, error)

LoadSet reads a spec (or a plain CIDR list) from r and compiles a membership Set, ignoring any ASN/org fields.

func (*Set) Contains

func (s *Set) Contains(a netip.Addr) bool

Contains reports whether a is covered by any prefix in the set.

func (*Set) Len

func (s *Set) Len() int

Len returns the number of disjoint intervals in the set (after merging), for both families. It is not the number of prefixes added.

type SpecEntry

type SpecEntry struct {
	Prefix netip.Prefix
	ASN    uint32
	Org    string
}

SpecEntry is one parsed spec line.

func ParseSpec

func ParseSpec(r io.Reader) ([]SpecEntry, error)

ParseSpec reads a spec (see the package example) from r into entries. Lines whose first field is not a valid CIDR or address are skipped, so an occasional junk line in a large feed does not fail the load; only an underlying read error is returned.

type Table

type Table[V any] struct {
	// contains filtered or unexported fields
}

Table is an immutable IPv4/IPv6 longest-prefix-match table returning a value V (e.g. an ASN, an org name, or an encoded uint64). Nesting is resolved at build time into disjoint segments, so Lookup is one binary search with no allocation. The zero Table is a valid empty table. Safe for concurrent use once built.

func LoadFunc

func LoadFunc[V any](r io.Reader, parse func(fields []string) (netip.Prefix, V, bool)) (*Table[V], error)

LoadFunc reads a line-oriented stream and builds a Table[V], delegating the format to the caller: parse maps a line's whitespace-separated fields to a prefix and value, returning ok=false to skip the line. Blank lines and '#'-comments are dropped before parse is called. This is the generic loader for any "<cidr-or-prefix> <data...>" feed whose value is not an ASN — see the user guide for a worked example.

func (*Table[V]) Lookup

func (t *Table[V]) Lookup(a netip.Addr) (V, bool)

Lookup returns the value of the most-specific prefix covering a, or the zero V and false when no prefix covers it.

type TableBuilder

type TableBuilder[V any] struct {
	// contains filtered or unexported fields
}

TableBuilder accumulates (prefix, value) pairs for a Table. Not safe for concurrent use.

func NewTableBuilder

func NewTableBuilder[V any]() *TableBuilder[V]

NewTableBuilder returns an empty value-table builder.

func (*TableBuilder[V]) Add

func (b *TableBuilder[V]) Add(p netip.Prefix, v V)

Add attaches value v to prefix p (masked to its network). When prefixes nest, the longest (most specific) one wins at lookup time. Adding the same prefix more than once keeps the most recently added value.

func (*TableBuilder[V]) AddPrefix

func (b *TableBuilder[V]) AddPrefix(s string, v V) error

AddPrefix parses s (a CIDR or bare address, see ParsePrefix) and adds it with value v, returning any parse error.

func (*TableBuilder[V]) AddRange

func (b *TableBuilder[V]) AddRange(lo, hi netip.Addr, v V)

AddRange attaches value v to the inclusive interval [lo, hi]. The range is decomposed into its minimal set of CIDR prefixes, each carrying v, so longest-prefix match stays well defined: a more-specific range's pieces have longer masks and win over a covering one. An invalid range is ignored.

func (*TableBuilder[V]) Freeze

func (b *TableBuilder[V]) Freeze() *Table[V]

Freeze compiles the accumulated prefixes into an immutable Table.