crid

README

crid

crid is a Snowflake-variant distributed ID generator for Go. It produces unique 63-bit integer IDs by combining a timestamp with a sequence number drawn from a block reserved from a shared registry. Nodes need no per-node identity -- uniqueness comes from the registry handing out non-overlapping blocks per timestamp.

Highlights: 63-bit IDs that fit in int64; high-throughput block reservation with async pre-allocation; pluggable registry (in-memory, Postgres, or custom); configurable bit layout, epoch, block size, and threshold.

Install

go get -u github.com/from-cero/crid

Requires Go 1.26 or newer.

Quick start

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/from-cero/crid"
	"github.com/from-cero/crid/registry/memory"
)

func main() {
	ctx := context.Background()

	reg := memory.New()
	node, err := crid.New(reg)
	if err != nil {
		log.Fatal(err)
	}

	parser, err := crid.NewParser()
	if err != nil {
		log.Fatal(err)
	}

	id, err := node.Generate(ctx)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s -> %s\n", id, parser.Parse(id))
}

How it works

1. Node calls Registry.Allocate(ctx, timestamp, blockSize) to reserve a block of sequence numbers for the current second.
2. Generate serves IDs from that block in memory: id = (timestamp << sequenceBits) | sequence.
3. When the remaining count drops below threshold, the next block is reserved in the background. If the current block is exhausted before pre-allocation completes, Generate falls back to a synchronous call.

Two IDs can only collide if they share both timestamp and sequence number, which the registry prevents.

ID layout

 63                    sequenceBits             0
+--+-------------------+------------------------+
|0 |     timestamp     |        sequence        |
+--+-------------------+------------------------+

Default: 31 timestamp bits (~68 years from epoch) and 32 sequence bits (~4.3 billion IDs/second). timestampBits + sequenceBits must equal 63.

Working with IDs

id.Int64()  // raw int64
id.String() // decimal string

ID marshals to/from JSON as a quoted decimal string to avoid precision loss in JavaScript:

{ "id": "123456789012345" }

Parsing

parser, err := crid.NewParser() // must use same epoch and format as the generating Node
parsed := parser.Parse(id)
parsed.Timestamp // time.Time
parsed.Sequence  // int64

[!IMPORTANT] A Parser must be created with the same epoch and format as the Node that generated the IDs. A mismatch produces silently wrong results, not an error.

Configuration

Pass options to New and NewParser:

Option	Default	Description
WithFormat(WithTimestampBits, WithSequenceBits)	31 / 32	Bit split; must sum to 63.
WithEpoch(time.Time)	2026-01-01 00:00:00 UTC	Reference time; must not be in the future.
WithBlockSize(int64)	10,000	Sequence numbers reserved per registry call. Must be in [1, 2^sequenceBits].
WithThreshold(int64)	5,000	Remaining count that triggers background pre-allocation. Must not exceed block size; below 1 disables pre-allocation.

node, err := crid.New(reg,
	crid.WithEpoch(time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)),
	crid.WithFormat(crid.WithTimestampBits(41), crid.WithSequenceBits(22)),
	crid.WithBlockSize(1_000),
	crid.WithThreshold(200),
)

Tuning: Keep threshold >= peak_rate * registry_latency so background pre-allocation stays ahead of demand. Larger blocks mean fewer round-trips but larger wasted gaps if a node restarts mid-block.

Registry

type Registry interface {
	// Allocate reserves blockSize sequence numbers for timestamp and returns
	// the starting value of the block. Successive calls for the same timestamp
	// must return non-overlapping blocks.
	Allocate(ctx context.Context, timestamp, blockSize int64) (start int64, err error)
}

Memory

import "github.com/from-cero/crid/registry/memory"

reg := memory.New()

Suitable for tests and single-process deployments. Does not persist allocations across restarts.

Postgres

For durable, multi-process deployments. A separate module so the core stays dependency-free.

go get -u github.com/from-cero/crid/registry/postgres

Requires Go 1.26+ and github.com/jackc/pgx/v5 v5.10.0+.

import (
	"github.com/from-cero/crid/registry/postgres"
	"github.com/jackc/pgx/v5/pgxpool"
)

pool, err := pgxpool.New(ctx, dsn)
if err != nil {
	log.Fatal(err)
}
defer pool.Close()

reg, err := postgres.New(pool, postgres.WithTable("crid_example_allocations"))
if err != nil {
	log.Fatal(err)
}
// Create the table once at startup, or manage it with migrations instead.
if err := reg.EnsureSchema(ctx); err != nil {
	log.Fatal(err)
}

node, err := crid.New(reg)

New accepts *pgxpool.Pool, *pgx.Conn, or pgx.Tx. The table name defaults to crid_allocations; override with postgres.WithTable("schema.table"). Each Allocate is a single atomic UPSERT, so concurrent callers across all processes receive non-overlapping blocks:

CREATE TABLE IF NOT EXISTS crid_allocations (
    ts       BIGINT PRIMARY KEY,
    next_seq BIGINT NOT NULL
);

Guarantees and caveats

• Uniqueness is guaranteed for all nodes sharing one registry, as long as the registry honors non-overlapping blocks per timestamp.
• IDs are not time-ordered. The timestamp reflects when the block was reserved, not when the ID was generated. Async pre-allocation means an ID can carry a slightly earlier timestamp. Do not rely on monotonicity or k-sortability.
• Error sentinel. Generate returns ID(-1) on failure. Always check the returned error; do not treat 0 as "no ID" (zero is a valid ID).
• Clock. A backward clock jump below the epoch returns ErrClockBeforeEpoch. Uniqueness is never affected by clock movement.

Errors

New/NewParser validation errors wrap ErrInvalidConfig: ErrInvalidBitFormat, ErrEpochInFuture, ErrInvalidBlockSize, ErrInvalidThreshold. New also returns ErrNilRegistry.

Generate may return ErrClockBeforeEpoch, ErrTimestampOverflow, ErrSequenceOverflow, or ErrInvalidSequence, plus any registry error. Use errors.Is to test them.

Concurrency

Node is safe for concurrent use by multiple goroutines. Background pre-allocation runs in its own goroutine and synchronizes on the node's internal lock.

Documentation

Index

• Variables
• type FormatOption
  • func WithSequenceBits(bits uint8) FormatOption
  • func WithTimestampBits(bits uint8) FormatOption
• type ID
  • func (id ID) Int64() int64
  • func (id ID) MarshalJSON() ([]byte, error)
  • func (id ID) String() string
  • func (id *ID) UnmarshalJSON(data []byte) error
• type Node
  • func New(reg registry.Registry, opts ...Option) (*Node, error)
  • func (n *Node) Generate(ctx context.Context) (ID, error)
• type Option
  • func WithBlockSize(b int64) Option
  • func WithEpoch(e time.Time) Option
  • func WithFormat(opts ...FormatOption) Option
  • func WithThreshold(t int64) Option
• type ParsedID
  • func (p ParsedID) String() string
• type Parser
  • func NewParser(opts ...Option) (*Parser, error)
  • func (p *Parser) Parse(id ID) ParsedID

Variables

var (
	// ErrInvalidConfig is returned when New is called with a config that fails validation.
	ErrInvalidConfig = errors.New("invalid config")

	// ErrInvalidBitFormat is returned when format.timestampBits + format.sequenceBits != 63.
	ErrInvalidBitFormat = errors.New("bit format must sum to 63")

	// ErrEpochInFuture is returned when cfg.epoch is the time
	// after the current system time at the time of Node creation.
	ErrEpochInFuture = errors.New("configured epoch is in the future")

	// ErrInvalidBlockSize is returned when cfg.blockSize is not a positive integer
	// or exceeds the maximum sequence number for the given format (2^format.sequenceBits).
	ErrInvalidBlockSize = errors.New("block size must be positive and <= max sequence number for the given format")

	// ErrInvalidThreshold is returned when cfg.threshold exceeds cfg.blockSize.
	ErrInvalidThreshold = errors.New("threshold must not exceed block size")

	// ErrNilRegistry is returned when New is called with a nil Registry.
	ErrNilRegistry = errors.New("registry cannot be nil")

	// ErrClockBeforeEpoch is returned when the system clock is behind the configured epoch.
	ErrClockBeforeEpoch = errors.New("system clock is before the configured epoch")

	// ErrTimestampOverflow is returned when the current time
	// exceeds the maximum representable timestamp for the given format.
	ErrTimestampOverflow = errors.New("timestamp exceeds maximum for the given format")

	// ErrSequenceOverflow is returned when too many IDs are generated in the same second
	// and the sequence number exceeds the maximum for the given format.
	ErrSequenceOverflow = errors.New("too many IDs generated in the same second")

	// ErrInvalidSequence is returned when the sequence number acquired from the registry
	// is out of range for the given format.sequenceBits.
	ErrInvalidSequence = errors.New("invalid sequence number for given format")
)

Types

type FormatOption

type FormatOption func(*format)

FormatOption configures the bit layout of generated IDs.

func WithSequenceBits

func WithSequenceBits(bits uint8) FormatOption

WithSequenceBits sets the number of bits used for the sequence component.

func WithTimestampBits

func WithTimestampBits(bits uint8) FormatOption

WithTimestampBits sets the number of bits used for the timestamp component.

type ID

type ID int64

ID is a 63-bit Snowflake-style distributed identifier.

func (ID) Int64

func (id ID) Int64() int64

Int64 returns the ID as a plain int64.

func (ID) MarshalJSON

func (id ID) MarshalJSON() ([]byte, error)

MarshalJSON encodes the ID as a quoted decimal string to avoid precision loss in JavaScript, which cannot represent 63-bit integers exactly.

func (ID) String

func (id ID) String() string

String returns the ID as a decimal string.

func (*ID) UnmarshalJSON

func (id *ID) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes the ID from a quoted decimal string.

type Node

type Node struct {
	// contains filtered or unexported fields
}

Node generates unique IDs from sequence blocks reserved in a Registry. The embedded timestamp reflects block-allocation time, not generation time, so IDs are not strictly time-ordered. It is safe for concurrent use by multiple goroutines.

func New

func New(reg registry.Registry, opts ...Option) (*Node, error)

New creates a Node backed by reg, applying any options on top of the defaults.

func (*Node) Generate

func (n *Node) Generate(ctx context.Context) (ID, error)

Generate returns the next unique ID, reserving a new block of sequence numbers from the registry when the current allocation is exhausted.

type Option

type Option func(*config)

Option configures a Node or Parser at creation time.

func WithBlockSize

func WithBlockSize(b int64) Option

WithBlockSize sets the number of sequence numbers a Node reserves from the registry per allocation. It must be positive. The default is 10,000.

func WithEpoch

func WithEpoch(e time.Time) Option

WithEpoch sets the reference time from which timestamps are measured. The epoch must not be in the future. The default is 2026-01-01 00:00:00 UTC.

func WithFormat

func WithFormat(opts ...FormatOption) Option

WithFormat sets the bit layout of generated IDs using the given format options.

func WithThreshold

func WithThreshold(t int64) Option

WithThreshold sets how many sequence numbers must remain in the current block before the Node asynchronously pre-allocates the next one. It must not exceed the block size. A value below 1 disables pre-allocation. The default is 5,000.

type ParsedID

type ParsedID struct {
	Timestamp time.Time
	Sequence  int64
}

ParsedID holds the decoded components of an ID.

func (ParsedID) String

func (p ParsedID) String() string

String returns a human-readable representation of the timestamp and sequence.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser decodes IDs without requiring a running Node.

func NewParser

func NewParser(opts ...Option) (*Parser, error)

NewParser creates a Parser configured with the given options.

func (*Parser) Parse

func (p *Parser) Parse(id ID) ParsedID

Parse decodes an ID into its timestamp and sequence components.