smolid

README

smolid

smolid is a 64-bit (8-byte) ID scheme for Go that is URL-friendly, temporally sortable, and optimized for database locality. It is designed for use cases where 128-bit UUIDs are unnecessarily large, but a standard auto-incrementing integer is insufficient.

• URL-Friendly: Encoded as short, unpadded base32 strings (e.g., acpje64aeyez6).
• Temporally Sortable: Most significant bits contain a millisecond-precision timestamp.
• Compact: Fits into a standard Go uint64 and a PostgreSQL bigint.
• Type-Aware: Supports embedding an optional 7-bit type identifier directly into the ID.

ID Structure

A smolid consists of 64 bits partitioned as follows:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          time_high                            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    time_low     |ver|t| rand  | type or rand|       rand      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

• Timestamp (41 bits): Millisecond-precision timestamp with a custom epoch (2025-01-01). Valid until 2094.
• Version (2 bits): Reserved for versioning (v1 is 01).
• Type Flag (1 bit): Boolean flag indicating if the Type field is used.
• Random/Type (20 bits):
  • If Type Flag is unset: 20 bits of pseudo-random data.
  • If Type Flag is set: 4 bits of random data, a 7-bit Type ID, and 9 bits of random data.

Usage

The godoc is available as a reference.

Basic Example

import "github.com/dotvezz/smolid"

// Generate a new ID
id := smolid.New()
fmt.Println(id.String()) // e.g., "acpje64aeyez6"

// Parse from string
parsed, _ := smolid.FromString("acpje64aeyez6")

Using Embedded Types

Embedded types allow you to identify the resource type (e.g., User, Post, Comment) directly from the ID itself.

const (
    TypeUser byte = iota + 1
    TypePost
)

// Create an ID with a type
id, _ := smolid.NewWithType(TypeUser)

// Check type later
if t, err := id.Type(); err == nil {
    switch t {
    case TypeUser:
        fmt.Println("This is a User ID")
    }
}

Integration

smolid implements several standard Go interfaces for seamless integration:

• JSON: json.Marshaler and json.Unmarshaler.
• Text: encoding.TextMarshaler and encoding.TextUnmarshaler.
• SQL: database/sql.Scanner and database/sql/driver.Valuer.
• Postgres: Native support for pgx (via pgtype.Int8Scanner/Valuer).
• GORM: Automatically identifies as bigint.

Considerations

Uniqueness and Collisions

smolid provides 13 to 20 bits of entropy per millisecond. This is "unique-enough" for many applications but is not a replacement for UUIDs in high-concurrency environments with massive write volumes (e.g., >1000 IDs per millisecond).

Database Compatibility

While smolid fits in a uint64, PostgreSQL bigint columns are signed. The timestamp will continue to work correctly and remain sortable until the year 2059, at which point the most significant bit flips and the values become negative from a signed perspective.

License

MIT

Documentation

Index

• Variables
• type ID
  • func FromString(s string) (_ ID, err error)
  • func Must(id ID, err error) ID
  • func New() ID
  • func NewWithType(typ byte) (ID, error)
  • func Nil() ID
  • func (id ID) Bytes() []byte
  • func (ID) GormDataType() string
  • func (id ID) Int64Value() (pgtype.Int8, error)
  • func (id ID) IsOfType(typ byte) (bool, error)
  • func (id ID) IsTyped() bool
  • func (id ID) MarshalJSON() ([]byte, error)
  • func (id ID) MarshalText() ([]byte, error)
  • func (id *ID) Scan(value any) error
  • func (id *ID) ScanInt64(v pgtype.Int8) error
  • func (id ID) String() string
  • func (id ID) Time() time.Time
  • func (id ID) Type() (byte, error)
  • func (id ID) Uint64() uint64
  • func (id *ID) UnmarshalJSON(data []byte) error
  • func (id *ID) UnmarshalText(text []byte) error
  • func (id ID) Value() (driver.Value, error)
  • func (id ID) Version() int

Variables

var ErrInvalidType = fmt.Errorf("invalid type id: must be less than or equal to %d", v1TypeSize)

var ErrUntyped = fmt.Errorf("no type id is embedded")

Types

type ID

type ID struct {
	// contains filtered or unexported fields
}

ID is a 64-bit (8-byte) value intended to be

• URL-Friendly; short and unobtrusive in its default unpadded base32 string encoding
• temporally sortable with strong index locality
• fast-enough and unique-enough for most use cases

Field Definitions

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          time_high                            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    time_low     |ver|t| rand  | type or rand|       rand      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Field Descriptions:

• Timestamp (41 bits): The most significant 41 bits represent a millisecond-precision timestamp The allowed timestamp range is 2025-01-01 00:00:00 - 2094-09-07 15:47:35
• Version (2 bits): Bits 41-42 are reserved for versioning. v1 is `01`
• Type Flag (1 bit): Bit 43 serves as a boolean flag. If set, the "Type/Rand" field is an embedded type identifier.
• Random (4 bits): The remaining 4 bits of the 6th byte are populated with pseudo-random data.
• Type/Random (7 bits): If the Type Flag is set, this field contains the Type Identifier. Otherwise, it is populated with pseudo-random data.
• Random (9 bits): The remaining byte is dedicated to pseudo-random data to _reasonably_ ensure uniqueness.

String Format

The string format is base32 with no padding. Canonically the string is lowercased. This decision is purely for
aesthetics, but the parser is case-insensitive and will accept uppercase base32 strings.

func FromString

func FromString(s string) (_ ID, err error)

FromString parses a smolid.ID from a string. While the canonical representation is all-lowercase, the parser is case-insensitive and will accept uppercase or mixed case without problems.

func Must

func Must(id ID, err error) ID

Must is a convenience function that panics if the given error is not nil. Useful for testing or scenarios where you know fully that an ID is valid, but don't want to deal with ignoring the error.

Stolen from gofrs/uuid

func New

func New() ID

New returns a new smolid.ID v1 with all defaults.

func NewWithType

func NewWithType(typ byte) (ID, error)

NewWithType returns a new smolid.ID v1 with the given type identifier embedded into the ID.

func Nil

func Nil() ID

func (ID) Bytes

func (id ID) Bytes() []byte

Bytes returns the raw 64-bit integer representation of the ID as a []byte.

func (ID) GormDataType

func (ID) GormDataType() string

GormDataType returns the data type for GORM.

func (ID) Int64Value

func (id ID) Int64Value() (pgtype.Int8, error)

Int64Value implements the pgtype.Int8Valuer interface (PostgreSQL BIGINT).

func (ID) IsOfType

func (id ID) IsOfType(typ byte) (bool, error)

IsOfType returns true if the ID is typed and matches the given type identifier.It will return an error if the ID was not created by NewWithType.

func (ID) IsTyped

func (id ID) IsTyped() bool

IsTyped returns true if the ID was created by NewWithType.

func (ID) MarshalJSON

func (id ID) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for ID.

func (ID) MarshalText

func (id ID) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface for ID.

func (*ID) Scan

func (id *ID) Scan(value any) error

Scan implements the sql.Scanner interface.

func (*ID) ScanInt64

func (id *ID) ScanInt64(v pgtype.Int8) error

ScanInt64 implements the pgtype.Int8Scanner interface.

func (ID) String

func (id ID) String() string

String returns the canonical string representation of the ID.

func (ID) Time

func (id ID) Time() time.Time

Time returns the time.Time embedded in the ID, with millisecond precision.

func (ID) Type

func (id ID) Type() (byte, error)

Type returns the type identifier embedded in the ID, if any. It will return an error if the ID was not created by NewWithType.

func (ID) Uint64

func (id ID) Uint64() uint64

Uint64 returns the raw 64-bit integer representation of the ID. Use this instead of Int64Value for most cases. The Int64Value method is provided for compatibility with the pgtype.Int8Valuer interface.

func (*ID) UnmarshalJSON

func (id *ID) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for ID.

func (*ID) UnmarshalText

func (id *ID) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface for ID.

func (ID) Value

func (id ID) Value() (driver.Value, error)

Value implements the driver.Valuer interface.

func (ID) Version

func (id ID) Version() int

Version returns the version of the ID