bq

README

bq

bq reads BSON documents from standard input and writes JSON to standard output. It is a small, dependency-free utility — the BSON wire format is parsed inline, so there is no mongo-driver (or any other third-party) dependency.

Install

Requires Go 1.26 or newer.

go install git.navicore.tech/navicore/bq@latest

This builds and places the bq binary in $GOBIN (or $GOPATH/bin; ensure that directory is on your $PATH).

To build from a source checkout instead:

git clone https://git.navicore.tech/navicore/bq.git
cd bq
just build        # produces ./bq

Usage

bq [options] < bson-stream

Output formats (-f, --format)

Format	Description	Streaming?
ndjson	One compact JSON object per line (default).	yes
json	A single compact JSON array of all documents.	no
pretty	A pretty-printed (indented) JSON array of documents.	no

Options

-f, --format     ndjson | json | pretty   (default: ndjson)
-i, --indent     indent string            (default: "  ", used by pretty)
    --datetime   iso | epoch              (default: iso)
-v, --version    print version and exit
-h, --help       print help and exit

Examples

cat dump.bson | bq
cat dump.bson | bq -f pretty
cat dump.bson | bq -f json --datetime epoch

Assumptions

These are the working assumptions bq is currently built on. If your input differs, please adjust or flag it so we can refine.

1. Input is a concatenation of length-prefixed BSON documents. This is the format produced by mongodump, bsondump, and raw MongoDB result streams. Each document begins with a little-endian int32 total length, followed by its body, followed by a 0x00 terminator. There is no other framing.

2. Output is JSON. Default is NDJSON (newline-delimited JSON, one document per line) for easy piping into jq, grep, etc.

3. Extended-JSON-style markers. Types with no native JSON representation are rendered as objects with a leading $ marker, e.g. {"$oid": "..."}, {"$date": "..."}, {"$binary": "...", "$binarySubT": N}, {"$regex": "...", "$options": "..."}, {"$timestamp": {"t":..,"i":..}}, {"$code": "..."}, {"$minKey": 1}, {"$maxKey": 1}.

4. Datetimes are UTC. In iso mode they are rendered as RFC 3339 strings ($date string); in epoch mode as the raw signed millisecond count ($date number).

5. Integers keep their BSON width. int32 and int64 decode to Go's int32/int64 and are marshalled accordingly. Consumers that parse with JavaScript-style "number" semantics may lose precision on large int64 values; switch to a JSON parser that preserves integer range if that matters.

6. Duplicate keys within a single document collapse. BSON permits duplicate field names; JSON objects do not. The last occurrence wins, matching Go's map[string]any behaviour.

Supported BSON element types

Type	Code	Notes
double	0x01
string	0x02
document	0x03	nested
array	0x04	decoded to a JSON array
binary	0x05	base64 + subtype
undefined	0x06	deprecated; rendered as null
ObjectId	0x07	24-char hex
bool	0x08
datetime	0x09	ms since Unix epoch; iso or epoch
null	0x0A
regex	0x0B	pattern + options
DBPointer	0x0C	deprecated; rendered as null
JS code	0x0D
symbol	0x0E	rendered as string
JS w/scope	0x0F
int32	0x10
timestamp	0x11	internal MongoDB timestamp type
int64	0x12
decimal128	0x13	see Limitations
min key	0xFF
max key	0x7F

Limitations

• Decimal128 (0x13) is not fully decoded. Correct rendering requires an IEEE 754-2008 decode including the densely-packed-decimal (DPD) step. Rather than ship an unverified implementation, decimal128 values are currently emitted as their raw 128 bits, e.g. {"$numberDecimal_raw": "00110400000000000008030000000000"}. No data is lost; the value can be reconstructed from the hex. Full decoding is tracked as a follow-up — we will revisit it once there is real decimal128 data to validate against.

License

MIT

Documentation

Overview

Command bq reads BSON documents from stdin and writes JSON to stdout.

See README.md for assumptions and supported formats.