README
¶
Bitcask Inspired KV Store: Append-Only Logs Meet In-Memory Hash Indexes
A lightweight, high-performance key-value store inspired by Bitcask. Built for simplicity, speed, and durability—perfect for learning system design, experimenting with storage engines, or using as a fast KV store in your projects.
Implementation
Read the full write-up: Building a Bitcask Inspired KV Store: Append-Only Logs Meet In-Memory Hash Indexes
The design prioritizes simplicity and correctness for educational clarity; production deployments would add compaction/merge, checksums, file rotation thresholds, and crash-consistency hardening.
Run Locally
Prerequisites
- Go 1.24+
- macOS/Linux recommended (Windows should work with minor path adjustments)
Clone
git clone https://github.com/himakhaitan/logkv-store.git
cd logkv-store
Build
Build the server and CLI binaries:
go build -o bin/logkvd ./cmd/logkvd
go build -o bin/logkv ./cmd/logkv-cli
Run the server
# Optional: customize bind address (default ":8080")
export LOGKV_ADDR=":8080"
./bin/logkvd
By default, data is persisted under the data/ directory at the project root.
Test
To run all unit tests in the project:
go test ./...
To measure how much of your code is covered by tests:
go test ./... -cover
For a detailed function-by-function coverage report:
go test ./... -coverprofile=coverage.out
go tool cover -func=coverage.out
#Optional If you want to see which lines are covered (green/red):
go tool cover -html=coverage.out
Interacting
- An HTTP server is started for basic operations.
- A CLI binary (
bin/logkv) is available for local interactions.
Note: This README intentionally omits specific CLI commands and API route details per project guidelines.
Configuration
- Address: set
LOGKV_ADDR(e.g.,:8080). - Data directory: defaults to
data/(seepkg/config/config.go).
Limitations (Current)
- No background compaction/merge to reclaim space.
- No checksums or CRC verification on records.
- Fixed single-process access pattern; no WAL fencing/locks for multi-process writers.
- Entire KeyDir must fit in memory.
- Basic HTTP surface; minimal validation.
Contributing
Issues and PRs are welcome. Keep changes focused and well-documented. Favor clarity over cleverness.
Feature Ideas You Can Add Next
- Compaction/merge job: coalesce latest values, drop tombstones, rewrite segments.
- Active segment rotation based on size/time thresholds.
- Checksums (CRC32/XXH3) for each record; verify on read.
- Bloom filter per segment to reduce unnecessary disk reads.
- Snapshot and restore utilities.
- Compression (per record or per segment).
- TTL/expiry and lazy cleanup.
- Metrics and tracing (Prometheus/OpenTelemetry).
- File-level and process-level locking for crash safety and multi-process protection.
- Configurable data directory via env/flags and structured config file.
- Batch operations (multi-set, multi-get) with pipelining.
- Simple authentication/authorization layer for the HTTP surface.
- Graceful schema evolution for record headers.
- Fsync strategies (always, interval, buffered) tuned via config.
Inspired by Bitcask
This project draws inspiration from the Bitcask storage model. See the original paper and community resources for deeper background.
Directories