shadowbox

README

ShadowBox

ShadowBox is a lightweight, high-performance Go library for storing files inside a single portable database file. It is designed as a fast alternative to traditional databases when your primary need is file storage, retrieval, and transfer.

Installation

go get github.com/hdmain/shadowbox@v0.1.0

Requires Go 1.22+.

Packages

Package	Import	Description
shb	github.com/hdmain/shadowbox/shb	Standard ShadowBox database
sshb	github.com/hdmain/shadowbox/sshb	Encrypted Secure ShadowBox
box	github.com/hdmain/shadowbox/box	Shared storage engine
fusefs	github.com/hdmain/shadowbox/fusefs	Linux FUSE filesystem (mount)
root	github.com/hdmain/shadowbox	Version, errors, file detection

Quick Start — SHB

package main

import (
    "fmt"
    "log"

    "github.com/hdmain/shadowbox/shb"
)

func main() {
    box, err := shb.Create("mybox.shb", shb.DefaultOptions())
    if err != nil {
        log.Fatal(err)
    }
    defer box.Close()

    _, err = shb.Put(box, "photos/vacation.jpg", imageData, "image/jpeg")
    if err != nil {
        log.Fatal(err)
    }

    out, info, err := box.Get("photos/vacation.jpg")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("stored %s (%d bytes)\n", info.Key, info.Size)
    _ = out
}

Quick Start — SSHB (Encrypted)

box, err := sshb.Create("secure.sshb", "my-secret-password", sshb.DefaultOptions())
if err != nil { log.Fatal(err) }
defer box.Close()

_, err = sshb.Put(box, "confidential/report.pdf", pdfData, "application/pdf")

Error Handling

All packages export sentinel errors compatible with errors.Is:

import (
    "errors"
    "github.com/hdmain/shadowbox/shb"
)

data, _, err := box.Get("missing.txt")
if errors.Is(err, shb.ErrNotFound) {
    // handle missing file
}

File Type Detection

import "github.com/hdmain/shadowbox"

if shadowbox.IsSSHB("data.sshb") {
    // open with sshb.Open(...)
}

API Overview

Method	Description
Create / Open	Create or open a database file
Put / PutBytes	Store a file by key
Get	Retrieve file contents (uses cache)
GetReader	Stream file contents
OpenSeeker	io.ReadSeekCloser without full-file RAM buffer
Delete	Remove a file by key
Exists	Check if key exists
List	List all stored files
Stat	Get metadata for one file
Sync	Flush index and data to disk
Preload	Load all files into RAM cache
CacheStats	Inspect cache usage

Configuration

opts := shb.DefaultOptions()
opts.CacheSize = 512 << 20  // 512 MiB RAM cache (0 = disabled)
opts.ChunkSize = 4 << 20      // 4 MiB plaintext per chunk (default)
opts.ReadOnly = true          // open without write access
opts.MaxBlockSize = 1 << 30   // max 1 GiB per file

Streaming read (no full buffer)

rs, info, err := box.OpenSeeker("large/video.mp4")
if err != nil { log.Fatal(err) }
defer rs.Close()

rs.Seek(1<<20, io.SeekStart) // seek to 1 MiB
io.Copy(os.Stdout, rs)

Chunked writes use AES-GCM per chunk (SSHB) with configurable ChunkSize. SHB stores raw chunks with CRC32 integrity per block.

File Format

┌─────────────────────────────────────┐
│ Header (128 bytes)                  │
├─────────────────────────────────────┤
│ Data blocks (append-only)           │
├─────────────────────────────────────┤
│ Index (serialized catalog + CRC32)  │
└─────────────────────────────────────┘

• SHB magic: SHB\x01 — extension .shb
• SSHB magic: SSH\x01 — extension .sshb, AES-256-GCM + Argon2id

Examples

cd examples/basic
go run .

Mounting as a filesystem (Linux)

ShadowBox can be mounted as a real FUSE filesystem using the fusefs package and shadowbox CLI.

Prerequisites

Install FUSE on Linux:

# Debian/Ubuntu
sudo apt install fuse3 libfuse3-dev

# Fedora
sudo dnf install fuse3 fuse3-devel

Build the mount tool

go install ./cmd/shadowbox

Mount

# Standard box
shadowbox mount data.shb /mnt/shadowbox

# Encrypted box
shadowbox mount secrets.sshb /secure/vda --password 'your-password'

# Read-only
shadowbox mount data.shb /mnt/readonly --readonly

Unmount with Ctrl+C or fusermount -u /mnt/shadowbox.

How it works

FUSE operation	ShadowBox backend
Lookup / Readdir	In-memory directory tree from box keys (/ separator)
Open / Read	box.OpenSeeker(key) — one chunk in RAM at a time
Create / Write	Buffered in memory, flushed via box.Put on Release/Flush
Delete	box.Delete(key) + tree update
Getattr	File size from Stat(); uid/gid/mode/mtime from .fusemeta sidecar

POSIX metadata is stored in <boxfile>.fusemeta alongside the database file.

Run examples

Publishing

Tag a release to publish a new version:

git tag v0.1.0
git push origin v0.1.0

Consumers install with:

go get github.com/hdmain/shadowbox@v0.1.0

License

MIT — see LICENSE.

Documentation

Overview

Package shadowbox provides a single-file file storage library for Go.

ShadowBox stores arbitrary files inside one portable database file with fast in-memory indexing and an optional RAM cache.

Two database types are available:

• shb — standard ShadowBox (import "github.com/hdmain/shadowbox/shb")
• sshb — encrypted Secure ShadowBox (import "github.com/hdmain/shadowbox/sshb")

The box package exposes the shared storage engine used by both variants.

Index

• Constants
• Variables
• func IsSHB(path string) bool
• func IsSSHB(path string) bool
• type Type
  • func Detect(path string) (Type, error)

Constants

const (
	// TypeSHB is the standard ShadowBox format.
	TypeSHB = format.TypeSHB
	// TypeSSHB is the encrypted Secure ShadowBox format.
	TypeSSHB = format.TypeSSHB
)

const Version = "0.1.0"

Version is the current library version.

Variables

var (
	ErrNotFound            = box.ErrNotFound
	ErrClosed              = box.ErrClosed
	ErrReadOnly            = box.ErrReadOnly
	ErrIntegrity           = box.ErrIntegrity
	ErrBlockTooLarge       = box.ErrBlockTooLarge
	ErrPasswordRequired    = box.ErrPasswordRequired
	ErrPasswordNotExpected = box.ErrPasswordNotExpected
	ErrInvalidPassword     = box.ErrInvalidPassword
)

Re-exported sentinel errors shared across all packages.

var ErrInvalidKey = sbscrypto.ErrInvalidKey

ErrInvalidKey is returned when decryption fails due to a wrong key.

Functions

func IsSHB

func IsSHB(path string) bool

IsSHB reports whether path points to a standard ShadowBox file.

func IsSSHB

func IsSSHB(path string) bool

IsSSHB reports whether path points to a Secure ShadowBox file.

Types

type Type

type Type = format.Type

Type identifies a ShadowBox database format.

func Detect

func Detect(path string) (Type, error)

Detect reads the database header and returns its type.