badgerx

package module
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 27, 2026 License: MIT Imports: 8 Imported by: 0

README

badgerx

CI Go Reference Go Report Card

badgerx is a thin, idiomatic Go wrapper around BadgerDB that abstracts away the repetitive encode/decode and compression boilerplate that every BadgerDB consumer ends up writing.

Why badgerx?

BadgerDB is a fast, embeddable key-value store — but it speaks only []byte. In practice, every application that uses it ends up writing the same scaffolding over and over:

// without badgerx — repeated at every call site
func (d *MyDb) StoreUser(id string, u User) error {
    val, err := gobEncode(u)
    if err != nil {
        return err
    }
    return d.db.Update(func(txn *badger.Txn) error {
        return txn.Set([]byte("user:"+id), val)
    })
}

func (d *MyDb) LoadUser(id string) (User, error) {
    var u User
    err := d.db.View(func(txn *badger.Txn) error {
        item, err := txn.Get([]byte("user:" + id))
        if err != nil {
            return err
        }
        return item.Value(func(val []byte) error {
            return gobDecode(val, &u)
        })
    })
    return u, err
}

That pattern — encode, transact, decode — is repeated for every type you store. badgerx collapses it into two calls while keeping full control over how values are encoded and compressed through swappable strategies.

// with badgerx
xdb := badgerx.NewBadgerXDb(db)

_ = xdb.Update([]byte("user:1"), u)

var got User
_ = xdb.View([]byte("user:1"), &got)

Features

  • Pluggable encoding — ships with gob and json. Bring your own (msgpack, protobuf, anything that serialises to []byte).
  • Pluggable compression — ships with zstd, snappy, and a no-op default. Implement the Compressor interface to add your own.
  • Strategy pattern — encoder and compressor are swapped via functional options at construction time, not scattered across call sites.
  • Safe defaults — gob encoding and no compression out of the box. Zero configuration required to get started.
  • No magic — badgerx does not wrap the full BadgerDB API. It exposes Update and View for encoded access, and the underlying badger.DB remains accessible for everything else.

Installation

go get github.com/somak2kai/badgerx

Requires Go 1.23 or later.

Quick Start

package main

import (
    "fmt"
    "log"

    badger "github.com/dgraph-io/badger/v4"
    "github.com/somak2kai/badgerx"
)

type User struct {
    Name string
    Age  int
}

func main() {
    db, err := badger.Open(badger.DefaultOptions("/tmp/mydb").WithLogger(nil))
    if err != nil {
        log.Fatal(err)
    }

    xdb := badgerx.NewBadgerXDb(db)
    defer xdb.Close()

    // store
    err = xdb.Update([]byte("user:1"), User{Name: "somak", Age: 30})
    if err != nil {
        log.Fatal(err)
    }

    // retrieve
    var u User
    err = xdb.View([]byte("user:1"), &u)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("%+v\n", u) // {Name:somak Age:30}
}

Encoders

GobEncoderDecoder (default)

Uses Go's built-in encoding/gob. Best for Go-to-Go communication where all readers and writers are Go programs.

xdb := badgerx.NewBadgerXDb(db) // gob is the default

If your structs contain interface{} fields, register the concrete types once at startup:

enc := &badgerx.GobEncoderDecoder{}
enc.RegisterType(ConcretePayload{})

xdb := badgerx.NewBadgerXDb(db, badgerx.WithEncoder(enc))
JsonEncoderDecoder

Uses encoding/json. Human-readable storage, good for cross-language interoperability or when you need to inspect values directly in the database.

xdb := badgerx.NewBadgerXDb(db, badgerx.WithEncoder(&badgerx.JsonEncoderDecoder{}))
Custom Encoder

Implement the Encoder interface to use any serialisation format:

type Encoder interface {
    Encode(v any) ([]byte, error)
    Decode(data []byte, v any) error
}

Compressors

DefaultNoOpCompressor (default)

No compression. Zero overhead. Use this when storage size is not a concern or when BadgerDB's built-in storage-level compression (zstd/snappy via Options.Compression) is sufficient.

ZstdCompressor

Zstandard compression. Excellent ratio with fast decompression. Best when storage efficiency matters.

zstdC, err := badgerx.NewZstdCompressor()
if err != nil {
    log.Fatal(err)
}

xdb := badgerx.NewBadgerXDb(db, badgerx.WithCompressor(zstdC))
defer xdb.Close() // releases zstd encoder/decoder resources
SnappyCompressor

Snappy compression. Prioritises speed over ratio. Good for latency-sensitive workloads with large values.

xdb := badgerx.NewBadgerXDb(db, badgerx.WithCompressor(&badgerx.SnappyCompressor{}))
Custom Compressor

Implement the Compressor interface to use any compression algorithm:

type Compressor interface {
    Compress(data []byte) ([]byte, error)
    Decompress(data []byte) ([]byte, error)
    Close() error
}

Combining Encoder and Compressor

Options compose freely:

zstdC, _ := badgerx.NewZstdCompressor()

xdb := badgerx.NewBadgerXDb(db,
    badgerx.WithEncoder(&badgerx.JsonEncoderDecoder{}),
    badgerx.WithCompressor(zstdC),
)
defer xdb.Close()

A note on compression layers

badgerx compression operates per value before the data reaches BadgerDB. This is distinct from BadgerDB's own storage-level compression (Options.Compression), which compresses value log blocks on disk. The two are complementary — you can use both simultaneously.

Accessing the underlying BadgerDB

badgerx intentionally wraps only Update and View. For everything else — transactions, iterators, TTLs, subscriptions — use the underlying badger.DB directly. badgerx does not try to re-wrap the full API.

Contributing

Contributions are welcome. Please open an issue before submitting a PR for significant changes. Run tests before pushing:

go test ./... -race

License

MIT. See LICENSE.

Documentation

Overview

Package badgerx extends github.com/dgraph-io/badger/v4 with pluggable encoding and compression strategies, removing the need to manually serialize and compress values at every call site.

Design

BadgerXDb wraps a badger.DB and applies an Encoder and a Compressor on every read and write. Both are swappable via functional options, defaulting to GobEncoderDecoder and DefaultNoOpCompressor when not specified.

Quick start 

db, _ := badger.Open(badger.DefaultOptions("/tmp/mydb"))

// default: gob encoding, no compression
xdb := badgerx.NewBadgerXDb(db)
defer xdb.Close()

type User struct{ Name string; Age int }

_ = xdb.Update([]byte("user:1"), User{Name: "somak", Age: 30})

var u User
_ = xdb.View([]byte("user:1"), &u)

Encoders

Two encoders are provided out of the box:

Implement the Encoder interface to supply your own (e.g. msgpack, protobuf).

Compressors

Three compressors are provided:

Implement the Compressor interface to supply your own.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BadgerXDb 

type BadgerXDb struct {
	// contains filtered or unexported fields
}

BadgerXDb wraps a badger.DB with pluggable encoding and compression. Values are encoded then compressed on write, and decompressed then decoded on read, transparently at every Update and View call.

Use NewBadgerXDb to create an instance. Always call BadgerXDb.Close when done to release resources held by the compressor and the underlying DB.

func NewBadgerXDb 

func NewBadgerXDb(db *badger.DB, opts ...BdOptions) *BadgerXDb

NewBadgerXDb creates a new BadgerXDb wrapping the given badger.DB. Defaults to GobEncoderDecoder and DefaultNoOpCompressor unless overridden via WithEncoder or WithCompressor. 

xdb := badgerx.NewBadgerXDb(db,
    badgerx.WithEncoder(&badgerx.JsonEncoderDecoder{}),
    badgerx.WithCompressor(zstdC),
)
Example

ExampleNewBadgerXDb demonstrates creating a BadgerXDb with default settings (gob encoding, no compression) and performing a basic store and retrieve. 

package main

import (
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

type User struct {
	Name string
	Age  int
}

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	db := openDB()

	xdb := badgerx.NewBadgerXDb(db)
	defer xdb.Close()

	_ = xdb.Update([]byte("user:1"), User{Name: "somak", Age: 30})

	var u User
	_ = xdb.View([]byte("user:1"), &u)

	fmt.Println(u.Name, u.Age)
}

Output:
somak 30

func (*BadgerXDb) Close 

func (d *BadgerXDb) Close() error

Close releases resources held by the compressor and closes the underlying badger DB. Always defer Close after creating a BadgerXDb: 

xdb := badgerx.NewBadgerXDb(db)
defer xdb.Close()

If both the compressor and the DB return errors on close, both are returned joined via errors.Join.

func (*BadgerXDb) IterateView added in v0.1.2 

func (d *BadgerXDb) IterateView(prefix []byte, opts badger.IteratorOptions, fn IterFunc) error

IterateView iterates over all keys sharing the given prefix, calling fn once per matching key. fn receives a DecodeFunc that decodes the current item's value into any pointer the caller provides.

opts controls iteration behaviour — use badger.DefaultIteratorOptions for forward iteration, or customise for reverse iteration, keys-only mode, or prefetch tuning: 

// forward iteration (default)
xdb.IterateView([]byte("user:"), badger.DefaultIteratorOptions, fn)

// reverse iteration
opts := badger.DefaultIteratorOptions
opts.Reverse = true
xdb.IterateView([]byte("user:"), opts, fn)

Create a fresh variable inside fn on each call — do not reuse a variable declared outside, as it will be overwritten on every iteration: 

var results []User
err := xdb.IterateView([]byte("user:"), badger.DefaultIteratorOptions, func(decode badgerx.DecodeFunc) error {
    var u User
    if err := decode(&u); err != nil {
        return err
    }
    results = append(results, u)
    return nil
})

Returning a non-nil error from fn stops iteration and surfaces that error as the return value of IterateView.

Example

ExampleBadgerXDb_IterateView demonstrates iterating over all keys sharing a common prefix and collecting the decoded values into a slice. 

package main

import (
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

type User struct {
	Name string
	Age  int
}

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	xdb := badgerx.NewBadgerXDb(openDB())
	defer xdb.Close()

	_ = xdb.Update([]byte("user:1"), User{Name: "alice", Age: 30})
	_ = xdb.Update([]byte("user:2"), User{Name: "bob", Age: 25})
	_ = xdb.Update([]byte("user:3"), User{Name: "carol", Age: 35})

	var users []User
	err := xdb.IterateView([]byte("user:"), badger.DefaultIteratorOptions, func(decode badgerx.DecodeFunc) error {
		var u User
		if err := decode(&u); err != nil {
			return err
		}
		users = append(users, u)
		return nil
	})
	if err != nil {
		log.Fatal(err)
	}

	for _, u := range users {
		fmt.Println(u.Name, u.Age)
	}
}

Output:
alice 30
bob 25
carol 35

func (*BadgerXDb) Update 

func (d *BadgerXDb) Update(key []byte, value any) error

Update encodes value using the configured Encoder, optionally compresses it using the configured Compressor, and stores the result under key.

The same encoder and compressor must be active when reading the value back via BadgerXDb.View.

Example

ExampleBadgerXDb_Update demonstrates storing a value under a key. 

package main

import (
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

type User struct {
	Name string
	Age  int
}

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	xdb := badgerx.NewBadgerXDb(openDB())
	defer xdb.Close()

	err := xdb.Update([]byte("user:1"), User{Name: "somak", Age: 30})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("stored")
}

Output:
stored

func (*BadgerXDb) View 

func (d *BadgerXDb) View(key []byte, v any) error

View retrieves the value stored under key, decompresses it, and decodes it into v. v must be a non-nil pointer to the same type that was passed to BadgerXDb.Update.

Returns badger.ErrKeyNotFound if the key does not exist. 

var u User
if err := xdb.View([]byte("user:1"), &u); errors.Is(err, badger.ErrKeyNotFound) {
    // key not found
}
Example

ExampleBadgerXDb_View demonstrates retrieving a value by key. Returns badger.ErrKeyNotFound when the key does not exist. 

package main

import (
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

type User struct {
	Name string
	Age  int
}

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	xdb := badgerx.NewBadgerXDb(openDB())
	defer xdb.Close()

	_ = xdb.Update([]byte("user:1"), User{Name: "somak", Age: 30})

	var u User
	err := xdb.View([]byte("user:1"), &u)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(u.Name, u.Age)
}

Output:
somak 30
Example (NotFound)

ExampleBadgerXDb_View_notFound demonstrates handling a missing key. 

package main

import (
	"errors"
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

type User struct {
	Name string
	Age  int
}

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	xdb := badgerx.NewBadgerXDb(openDB())
	defer xdb.Close()

	var u User
	err := xdb.View([]byte("user:missing"), &u)
	if errors.Is(err, badger.ErrKeyNotFound) {
		fmt.Println("not found")
	}
}

Output:
not found

type BdOptions 

type BdOptions func(*BadgerXDb)

BdOptions is a functional option for configuring a BadgerXDb. Use WithEncoder and WithCompressor to create options.

func WithCompressor 

func WithCompressor(c Compressor) BdOptions

WithCompressor returns a BdOptions that sets the compressor used by BadgerXDb. If not specified, DefaultNoOpCompressor is used by default (no compression).

func WithEncoder 

func WithEncoder(e Encoder) BdOptions

WithEncoder returns a BdOptions that sets the encoder used by BadgerXDb. If not specified, GobEncoderDecoder is used by default.

Example

ExampleWithEncoder demonstrates switching to JSON encoding. 

package main

import (
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

type User struct {
	Name string
	Age  int
}

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	xdb := badgerx.NewBadgerXDb(openDB(),
		badgerx.WithEncoder(&badgerx.JsonEncoderDecoder{}),
	)
	defer xdb.Close()

	_ = xdb.Update([]byte("user:1"), User{Name: "somak", Age: 30})

	var u User
	_ = xdb.View([]byte("user:1"), &u)

	fmt.Println(u.Name, u.Age)
}

Output:
somak 30

type Compressor 

type Compressor interface {
	// Compress compresses data and returns the compressed bytes.
	Compress(data []byte) ([]byte, error)
	// Decompress decompresses data and returns the original bytes.
	Decompress(data []byte) ([]byte, error)
	// Close releases any resources held by the compressor.
	Close() error
}

Compressor is the strategy interface for compressing and decompressing encoded byte slices before they are written to or after they are read from badger.

Implement this interface to provide a custom compression strategy. Close must release any resources held by the compressor.

type DecodeFunc added in v0.1.2 

type DecodeFunc func(v any) error

DecodeFunc is the function injected into each IterFunc call during BadgerXDb.IterateView. Call it with a non-nil pointer to decode the current item's value into your own variable: 

fn := func(decode badgerx.DecodeFunc) error {
    var u User
    return decode(&u)
}

type DefaultNoOpCompressor 

type DefaultNoOpCompressor struct{}

DefaultNoOpCompressor implements Compressor with no compression. It is the default compressor used by BadgerXDb when no compressor is specified. Use WithCompressor to swap in ZstdCompressor or SnappyCompressor when storage efficiency matters.

func (*DefaultNoOpCompressor) Close 

func (z *DefaultNoOpCompressor) Close() error

Close is a no-op for DefaultNoOpCompressor.

func (*DefaultNoOpCompressor) Compress 

func (z *DefaultNoOpCompressor) Compress(data []byte) ([]byte, error)

Compress returns data unmodified.

func (*DefaultNoOpCompressor) Decompress 

func (z *DefaultNoOpCompressor) Decompress(data []byte) ([]byte, error)

Decompress returns data unmodified.

type Encoder 

type Encoder interface {
	// Encode serializes v into a byte slice.
	Encode(v any) ([]byte, error)
	// Decode deserializes data into v. v must be a non-nil pointer.
	Decode(data []byte, v any) error
}

Encoder is the strategy interface for serializing and deserializing values.

Encode must convert v into a byte slice suitable for storage in badger. Decode must reconstruct the original value from data into v, where v is always a non-nil pointer to the target type (e.g. *MyStruct).

Implement this interface to provide a custom encoding strategy such as msgpack, protobuf, or any other binary format.

type GobEncoderDecoder 

type GobEncoderDecoder struct{}

GobEncoderDecoder implements Encoder using the standard encoding/gob package. It is the default encoder used by BadgerXDb when no encoder is specified.

For structs that contain interface{} fields, call GobEncoderDecoder.RegisterType once at startup for each concrete type that may appear in those fields.

func (*GobEncoderDecoder) Decode 

func (g *GobEncoderDecoder) Decode(data []byte, v any) error

Decode deserializes gob-encoded data into v. v must be a non-nil pointer.

func (*GobEncoderDecoder) Encode 

func (g *GobEncoderDecoder) Encode(v any) ([]byte, error)

Encode serializes v into gob-encoded bytes.

func (*GobEncoderDecoder) RegisterType 

func (g *GobEncoderDecoder) RegisterType(v any)

RegisterType registers a concrete type with gob so it can be correctly encoded and decoded when stored inside an interface{} field. This is only required when your structs contain interface{} fields. Call once at application startup — not on every read or write. 

enc := &badgerx.GobEncoderDecoder{}
enc.RegisterType(MyPayload{})
enc.RegisterType(AnotherPayload{})

xdb := badgerx.NewBadgerXDb(db, badgerx.WithEncoder(enc))
Example

ExampleGobEncoderDecoder_RegisterType demonstrates registering a concrete type for structs that contain interface{} fields. 

package main

import (
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	type Payload struct{ Value string }
	type Record struct {
		Name    string
		Payload any
	}

	enc := &badgerx.GobEncoderDecoder{}
	enc.RegisterType(Payload{})

	xdb := badgerx.NewBadgerXDb(openDB(), badgerx.WithEncoder(enc))
	defer xdb.Close()

	_ = xdb.Update([]byte("rec:1"), Record{Name: "badgerx", Payload: Payload{Value: "hello"}})

	var r Record
	_ = xdb.View([]byte("rec:1"), &r)

	fmt.Println(r.Name, r.Payload.(Payload).Value)
}

Output:
badgerx hello

type IterFunc added in v0.1.2 

type IterFunc func(decode DecodeFunc) error

IterFunc is the callback passed to BadgerXDb.IterateView. It is called once per matching key. The provided DecodeFunc decodes the current item only — create a fresh variable inside IterFunc on each call to avoid overwriting previous results: 

var results []User
fn := func(decode badgerx.DecodeFunc) error {
    var u User          // fresh every iteration
    if err := decode(&u); err != nil {
        return err
    }
    results = append(results, u)
    return nil
}

type JsonEncoderDecoder 

type JsonEncoderDecoder struct{}

JsonEncoderDecoder implements Encoder using the standard encoding/json package. Prefer this over GobEncoderDecoder when human-readable storage or cross-language interoperability is required. Values must be JSON-serializable (exported fields, json struct tags recommended).

func (*JsonEncoderDecoder) Decode 

func (j *JsonEncoderDecoder) Decode(data []byte, v any) error

Decode deserializes JSON-encoded data into v. v must be a non-nil pointer.

func (*JsonEncoderDecoder) Encode 

func (j *JsonEncoderDecoder) Encode(v any) ([]byte, error)

Encode serializes v into JSON-encoded bytes.

type SnappyCompressor 

type SnappyCompressor struct{}

SnappyCompressor implements Compressor using the Snappy algorithm. It prioritises speed over compression ratio, making it a good fit for latency-sensitive workloads with large values where some size reduction is still desirable.

func (*SnappyCompressor) Close 

func (s *SnappyCompressor) Close() error

Close is a no-op for SnappyCompressor.

func (*SnappyCompressor) Compress 

func (s *SnappyCompressor) Compress(data []byte) ([]byte, error)

Compress compresses data using the Snappy algorithm.

func (*SnappyCompressor) Decompress 

func (s *SnappyCompressor) Decompress(data []byte) ([]byte, error)

Decompress decompresses snappy-compressed data.

type ZstdCompressor 

type ZstdCompressor struct {
	// contains filtered or unexported fields
}

ZstdCompressor implements Compressor using the Zstandard algorithm. It offers an excellent compression ratio with fast decompression, making it well-suited for workloads where storage size matters more than write speed.

The encoder and decoder are created once and reused across calls — safe for concurrent use. Use NewZstdCompressor to create an instance.

func NewZstdCompressor 

func NewZstdCompressor() (*ZstdCompressor, error)

NewZstdCompressor creates a ZstdCompressor initialising reusable zstd encoder and decoder instances. The returned compressor is safe for concurrent use. Call ZstdCompressor.Close when done to release resources.

Example

ExampleNewZstdCompressor demonstrates using Zstandard compression alongside the default gob encoder. 

package main

import (
	"fmt"
	"log"

	badger "github.com/dgraph-io/badger/v4"
	badgerx "github.com/somak2kai/badgerx"
)

type User struct {
	Name string
	Age  int
}

func openDB() *badger.DB {
	db, err := badger.Open(badger.DefaultOptions("").WithInMemory(true).WithLogger(nil))
	if err != nil {
		log.Fatal(err)
	}
	return db
}

func main() {
	zstdC, err := badgerx.NewZstdCompressor()
	if err != nil {
		log.Fatal(err)
	}

	xdb := badgerx.NewBadgerXDb(openDB(), badgerx.WithCompressor(zstdC))
	defer xdb.Close()

	_ = xdb.Update([]byte("user:1"), User{Name: "somak", Age: 30})

	var u User
	_ = xdb.View([]byte("user:1"), &u)

	fmt.Println(u.Name, u.Age)
}

Output:
somak 30

func (*ZstdCompressor) Close 

func (z *ZstdCompressor) Close() error

Close releases resources held by the zstd encoder and decoder.

func (*ZstdCompressor) Compress 

func (z *ZstdCompressor) Compress(data []byte) ([]byte, error)

Compress compresses data using the Zstandard algorithm.

func (*ZstdCompressor) Decompress 

func (z *ZstdCompressor) Decompress(data []byte) ([]byte, error)

Decompress decompresses zstd-compressed data.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.