Help us shape the future of Go! Take the 2026 Go Developer Survey

bedrock

package module
v0.0.0-...-8a0e250 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 24, 2025 License: Apache-2.0 Imports: 23 Imported by: 0

README

Bedrock: A Distributed, Transactional Key-Value Store

codecov

Disclaimer: Bedrock is a hobby project built for educational purposes. While it is designed to be robust and has a growing test suite, it has not been audited or tested for production environments. Please use it in production at your own risk.

Bedrock is a persistent key-value store written in Go, designed to be both a high-performance embedded engine and a fault-tolerant distributed database. It uses a Log-Structured Merge-Tree (LSM-Tree) for its storage engine and the Raft consensus algorithm for replication.

The primary goal of this project is to explore and implement the core concepts of modern distributed database systems.

Features

Version 0.3 (Current)

The current version of Bedrock is a complete, distributed, and transactional key-value store.

  • Distributed Consensus with Raft: Bedrock uses the HashiCorp Raft implementation to replicate data across a cluster of nodes. This provides high availability and fault tolerance, ensuring that the database remains operational and data is safe even if some servers in the cluster fail.

  • Multi-Key Transactions: Bedrock supports atomic multi-key transactions, providing ACID guarantees. A fine-grained, per-key lock manager provides isolation between concurrent transactions.

  • Durable Writes: All write operations are replicated via the Raft log, which is implemented on top of a durable Write-Ahead Log (WAL) on each node.

  • Crash Recovery & Snapshotting: Each node can recover its state quickly from its local WAL. The Raft implementation uses efficient snapshots to allow new or slow followers to catch up to the leader's state without replaying the entire history of the log.

  • Leveled Compaction: A background process on each node constantly performs Leveled Compaction on its local data, reclaiming disk space and keeping read performance high.

  • Fast Reads & Range Scans: On-disk data is indexed with in-memory sparse indexes, and the system supports efficient, sorted range scans via a merging iterator.

Usage Modes

Bedrock can be used in two primary modes: as an embedded library or as a distributed server.

1. Embedded (Single-Node) Mode

You can use Bedrock directly within your Go application as a durable, transactional, and concurrent key-value store library. All data will be stored in a local directory.

// --- Using Bedrock as an Embedded Library ---

kvConfig := bedrock.NewDefaultConfiguration() // Or create your own
db, err := bedrock.Open(kvConfig)
if err != nil {
    log.Fatalf("Failed to open Bedrock store: %s", err)
}
defer db.Close()

// 2. Perform operations using transactions.
txn := db.BeginTransaction()
txn.Put([]byte("hello"), []byte("embedded world"))
txn.Put([]byte("status"), []byte("ok"))
if err := txn.Commit(); err != nil {
    log.Fatalf("Commit failed: %s", err)
}

// 3. Read the data back.
readTxn := db.BeginTransaction()
val, ok := readTxn.Get([]byte("hello"))
if ok {
    fmt.Printf("Found value: %s\n", val) // "embedded world"
}
readTxn.Rollback() // Read-only transactions can be rolled back.
2. Distributed (Cluster) Mode

For high availability, Bedrock can be run as a server process that communicates with other nodes in a cluster. This mode exposes an HTTP API for client interaction.

Running the Server

You can start a single-node Bedrock server using the provided main function.

# From your project's root directory:
# Assuming that the `main` function lives in that directory
go run ./cmd/server/

By default, this will start the Raft communication on port 9000 and the client API on port 8080.

Writing Data (PUT)

To write or update a key, send a PUT request to the /put endpoint.

# Set the key "name" to the value "raft"
curl -X PUT -d 'raft' 'http://127.0.0.1:8080/put?key=name'
Reading Data (GET)

To write or update a key, send a GET request to the /get endpoint.

# Get the value for the key "name"
# Expected output: raft
curl 'http://127.0.0.1:8080/get?key=name'

Documentation

Index

Constants

View Source 
const (
	OP_PUT                           byte = 0x01
	OP_DELETE                        byte = 0x02
	SNAPSHOT_MANIFEST_FILE_TYPE      byte = 0x01
	SNAPSHOT_MANIFEST_FILE_PATH_TYPE byte = 0x02
	SNAPSHOT_MEMSTATE_FILE_TYPE      byte = 0x03
	SNAPSHOT_SEGMENT_FILE_TYPE       byte = 0x04
	SNAPSHOT_INDEX_FILE_TYPE         byte = 0x05
)
View Source 
const (

	// CheckpointSize = 64 * 1024 // 64KiB
	CheckpointSize = 1024 // 1KiB for testing
	AppendFlags    = os.O_RDWR | os.O_CREATE | os.O_APPEND
)

Variables

This section is empty.

Functions

func ComputeChecksum 

func ComputeChecksum(data []byte) uint32

ComputeChecksum calculates a CRC32 checksum of the given byte slice

func GenerateRandomString 

func GenerateRandomString(length int) string

GenerateRandomString generates a random string of a given length.

func GetPayloadForCommit 

func GetPayloadForCommit(ops [][]byte) []byte

GetPayloadForCommit returns the payload for a commit operation. The commit is a list of PUT operations.

func GetPayloadForPut 

func GetPayloadForPut(key []byte, value []byte) []byte

GetPayloadForPut returns the payload for a single, non-transactional write.

func GetSegmentIDFromIndexFilePath 

func GetSegmentIDFromIndexFilePath(filePath string) (uint64, error)

getSegmentIDFromSegmentFilePath returns the segment ID from a segment file path.

func GetSegmentIDFromManifestFileName 

func GetSegmentIDFromManifestFileName(fileName string) (uint64, error)

GetSegmentIDFromManifestFileName returns the segment ID from a manifest file name.

func GetSegmentIDFromSegmentFilePath 

func GetSegmentIDFromSegmentFilePath(filePath string) (uint64, error)

getSegmentIDFromSegmentFilePath returns the segment ID from a segment file path.

func GetSparseIndexBytes 

func GetSparseIndexBytes(segmentID uint64, key []byte, offset int64) []byte

GetSparseIndexBytes returns the bytes of the sparse index record.

func GetTempSegmentFileName 

func GetTempSegmentFileName(segmentId uint64) string

func GetTempSparseIndexFileName 

func GetTempSparseIndexFileName(segmentId uint64) string

func NewMergingIterator 

func NewMergingIterator(memtableIter Iterator, segmentIters []Iterator) *mergingIterator

func NewSegmentIterator 

func NewSegmentIterator(segmentPath string, index sparseIndex) (*segmentIterator, error)

func PreCreateDirectories 

func PreCreateDirectories(config *KVStoreConfig) error

PreCreateDirectories creates the directories for the KVStore if they don't exist.

func ScanRange 

func ScanRange(iter Iterator, startKey, endKey []byte, fn func(key, value []byte))

ScanRange scans the range of keys between startKey and endKey. It calls the provided function for each key-value pair in the range.

Note that the interval is [startKey, endKey), i.e. the endKey is not included.

func SerializeV2 

func SerializeV2(recordType byte, sequenceNum uint64, term uint64, payload []byte) []byte

SerializeV2 returns the bytes of the log record.

func StartRaft 

func StartRaft(kv *KVStore, config *ApplicationConfig, peers []string) (*raft.Raft, error)

StartRaft initializes and starts the Raft node, turning the KVStore into a distributed, fault-tolerant state machine.

Types

type ApplicationConfig 

type ApplicationConfig struct {
	NodeID            string
	RaftAddr          string
	BaseDir           string
	Bootstrap         bool
	SnapshotInterval  time.Duration
	SnapshotRetention int
	SnapshotThreshold int
}

ApplicationConfig is the configuration for the entire raft-based application.

See also: StartRaft()

func DefaultApplicationConfig 

func DefaultApplicationConfig(baseDir string) *ApplicationConfig

DefaultApplicationConfig creates a default configuration for a single-node, localhost setup. This is useful for testing.

func (*ApplicationConfig) GetBaseDir 

func (c *ApplicationConfig) GetBaseDir() string

GetBaseDir returns the base directory for the application.

It is used to store Raft snapshots.

type CompactionPlan 

type CompactionPlan struct {
	// contains filtered or unexported fields
}

CompactionPlan is the plan for the compaction. It contains the base segments and the overlapping segments.

func (*CompactionPlan) GetAllSegments 

func (c *CompactionPlan) GetAllSegments() []SegmentMetadata

GetAllSegments returns all the segments in the compaction plan.

type Iterator 

type Iterator interface {
	// Seek positions the iterator at the first key that is greater
	// than or equal to the given key.
	Seek(key []byte)

	// Next advances the iterator to the next key in the sequence.
	Next()

	// Key returns the key at the current iterator position.
	// The returned slice is only valid until the next call to Next().
	Key() []byte

	// Value returns the value at the current iterator position.
	// The returned slice is only valid until the next call to Next().
	Value() []byte

	// Valid returns true if the iterator is positioned at a valid key-value pair.
	Valid() bool
}

Iterator provides a way to seek and scan over key-value pairs.

type KVRecord 

type KVRecord struct {
	CheckSum  uint32
	KeySize   uint32
	ValueSize uint32
	Key       []byte
	Value     []byte
}

Simple abstraction for a log record. +----------+---------------+-----------+-------------+-----+-------+ | Checksum | Key Size | Value Size | Key | Value | | (4 bytes)| (4 bytes) | (4 bytes) | ... | ... | +----------+---------------+-----------+-------------+-----+-------+ | <----------- Header (12 bytes) -----------------> | <- Payload -> |

func DeserializeKVRecord 

func DeserializeKVRecord(data []byte) (KVRecord, error)

DeserializeKVRecord converts a byte array to a KVRecord according to the specified format.

func (*KVRecord) ComputeChecksumForRecord 

func (lr *KVRecord) ComputeChecksumForRecord() uint32

ComputeChecksumForRecord calculates checksum for the entire log record (excluding the checksum field itself)

func (*KVRecord) GetChecksum 

func (lv *KVRecord) GetChecksum() uint32

GetChecksum returns the checksum for the log record.

func (*KVRecord) Serialize 

func (lr *KVRecord) Serialize() []byte

Serialize converts the LogRecord to a byte array according to the specified format. The format is: Header (20 bytes) + Payload (Key + Value)

func (*KVRecord) SetChecksum 

func (lv *KVRecord) SetChecksum()

SetChecksum sets the checksum for the log record.

func (*KVRecord) Size 

func (lr *KVRecord) Size() int

Size returns the size of the log record.

type KVStore 

type KVStore struct {
	// contains filtered or unexported fields
}

func Open 

func Open(config *KVStoreConfig) (*KVStore, error)

Open opens/creates the log file and initializes the KVStore object.

1. Try to get the last checkpoint file path from the checkpoint file and the WAL files.

2. If the checkpoint is found, open it and read the last segment file path.

- Load the segment file into memtable.

- For all WAL files with segment ID greater than the last segment ID in the checkpoint file, replay them into memtable in ascending order.

- Truncate the WAL file (likely the last one) to the last good offset.

3. If the checkpoint is not found, simply start from the beginning of the WAL files and replay them into memtable in ascending order.

- Truncate the WAL file (likely the last one) to the last good offset.

4. Create and return the KVStore object.

func (*KVStore) AllocateNewSegmentID 

func (s *KVStore) AllocateNewSegmentID() uint64

This method assumes that the lock is currently not withheld.

func (*KVStore) AllocateSegmentIDUnsafe 

func (s *KVStore) AllocateSegmentIDUnsafe() uint64

This method assumes that the lock is currently withheld.

func (*KVStore) Apply 

func (s *KVStore) Apply(logEntry *raft.Log) interface{}

func (*KVStore) BeginTransaction 

func (kv *KVStore) BeginTransaction() *Transaction

BeginTransaction creates a new transaction.

func (*KVStore) CleanUpDirectories 

func (kv *KVStore) CleanUpDirectories() error

CleanUpDirectories cleans up the checkpoint and WAL directories.

func (*KVStore) Close 

func (kv *KVStore) Close() error

Close closes the WAL file.

func (*KVStore) CloseAndCleanUp 

func (kv *KVStore) CloseAndCleanUp() error

Mostly used in tests.

func (*KVStore) CollectRange 

func (kv *KVStore) CollectRange(startKey, endKey []byte) ([]KVRecord, error)

CollectRange collects the key-value pairs in the range between startKey and endKey.

It returns a slice of KVRecord.

func (*KVStore) Delete 

func (kv *KVStore) Delete(key []byte) error

Delete deletes a key-value pair from the KVStore. It creates a new transaction and commits it. It is a wrapper around Transaction.Delete.

func (*KVStore) Get 

func (kv *KVStore) Get(key []byte) ([]byte, bool)

Get returns the value for a given key. The value is nil if the key is not found. It creates a new transaction and commits it. It is a wrapper around Transaction.Get.

func (*KVStore) GetCurrentManifestFilePath 

func (s *KVStore) GetCurrentManifestFilePath() (string, error)

GetCurrentManifest returns the path to the current manifest file.

func (*KVStore) GetCurrentSegmentIDSafe 

func (s *KVStore) GetCurrentSegmentIDSafe() uint64

GetCurrentSegmentIDSafe returns the ID of the current segment.

func (*KVStore) GetCurrentSegmentIDUnsafe 

func (s *KVStore) GetCurrentSegmentIDUnsafe() uint64

This method assumes that the lock is currently withheld.

func (*KVStore) GetLastSequenceNum 

func (kv *KVStore) GetLastSequenceNum() uint64

GetLastSequenceNum returns the last sequence number.

func (*KVStore) NewIterator 

func (kv *KVStore) NewIterator() Iterator

NewIterator creates an iterator for range queries.

func (*KVStore) Print 

func (kv *KVStore) Print()

Print prints the memtable.

func (*KVStore) Put 

func (kv *KVStore) Put(key, value []byte) error

Put writes a key-value pair to the KVStore. It creates a new transaction and commits it. It is a wrapper around Transaction.Put.

func (*KVStore) Restore 

func (s *KVStore) Restore(snapshot io.ReadCloser) error

func (*KVStore) RollToNewSegment 

func (kv *KVStore) RollToNewSegment() (uint64, error)

RollToNewSegment rolls to a new segment. It creates a new WAL file and updates the WAL object. It also updates the current segment ID and the last sequence number.

func (*KVStore) Snapshot 

func (s *KVStore) Snapshot() (raft.FSMSnapshot, error)

type KVStoreConfig 

type KVStoreConfig struct {
	CheckpointSize             int64
	BaseDir                    string
	SegmentFileSizeThresholdLX int64
	MaintenanceIntervalMs      uint64
	MemtableSizeThreshold      int64
	EnableSyncCheckpoint       bool
	EnableCheckpoint           bool
	EnableCompaction           bool
	EnableMaintenance          bool
}

KVStoreConfig is the configuration for the KVStore.

func NewConfigurationNoMaintenance 

func NewConfigurationNoMaintenance() *KVStoreConfig

func NewDefaultConfiguration 

func NewDefaultConfiguration() *KVStoreConfig

NewDefaultConfiguration returns the default DB configuration.

func (*KVStoreConfig) GetBaseDir 

func (c *KVStoreConfig) GetBaseDir() string

GetBaseDir returns the base directory for the DB.

func (*KVStoreConfig) GetCheckpointSize 

func (c *KVStoreConfig) GetCheckpointSize() int64

GetCheckpointSize returns the checkpoint size for the DB.

func (*KVStoreConfig) GetMaintenanceIntervalMs 

func (c *KVStoreConfig) GetMaintenanceIntervalMs() uint64

GetCompactionIntervalMs returns the compaction interval for the DB.

func (*KVStoreConfig) GetMemtableSizeThreshold 

func (c *KVStoreConfig) GetMemtableSizeThreshold() int64

GetMemtableSizeThreshold returns the memtable size threshold for the DB.

func (*KVStoreConfig) GetSegmentFileSizeThresholdLX 

func (c *KVStoreConfig) GetSegmentFileSizeThresholdLX() int64

GetSegmentFileSizeThresholdLX returns the segment file size threshold for the DB.

func (*KVStoreConfig) WithBaseDir 

func (c *KVStoreConfig) WithBaseDir(dir string) *KVStoreConfig

WithBaseDir sets the base directory for the DB.

func (*KVStoreConfig) WithCheckpointSize 

func (c *KVStoreConfig) WithCheckpointSize(size int64) *KVStoreConfig

WithCheckpointSize sets the checkpoint size for the DB.

func (*KVStoreConfig) WithEnableCheckpoint 

func (c *KVStoreConfig) WithEnableCheckpoint(enable bool) *KVStoreConfig

func (*KVStoreConfig) WithEnableCompaction 

func (c *KVStoreConfig) WithEnableCompaction(enable bool) *KVStoreConfig

func (*KVStoreConfig) WithEnableMaintenance 

func (c *KVStoreConfig) WithEnableMaintenance(enable bool) *KVStoreConfig

func (*KVStoreConfig) WithEnableSyncCheckpoint 

func (c *KVStoreConfig) WithEnableSyncCheckpoint(enable bool) *KVStoreConfig

func (*KVStoreConfig) WithLog 

func (c *KVStoreConfig) WithLog() *KVStoreConfig

func (*KVStoreConfig) WithMaintenanceIntervalMs 

func (c *KVStoreConfig) WithMaintenanceIntervalMs(intervalMs uint64) *KVStoreConfig

func (*KVStoreConfig) WithMemtableSizeThreshold 

func (c *KVStoreConfig) WithMemtableSizeThreshold(size int64) *KVStoreConfig

WithMemtableSizeThreshold sets the memtable size threshold for the DB.

func (*KVStoreConfig) WithNoLog 

func (c *KVStoreConfig) WithNoLog() *KVStoreConfig

func (*KVStoreConfig) WithSegmentFileSizeThresholdLX 

func (c *KVStoreConfig) WithSegmentFileSizeThresholdLX(size int64) *KVStoreConfig

WithSegmentFileSizeThresholdLX sets the segment file size threshold for the DB.

type LockManager 

type LockManager struct {
	// contains filtered or unexported fields
}

LockManager is a manager for per-key locks.

func NewLockManager 

func NewLockManager() *LockManager

NewLockManager creates a new LockManager.

func (*LockManager) AcquireReadLock 

func (lm *LockManager) AcquireReadLock(key []byte)

func (*LockManager) AcquireWriteLock 

func (lm *LockManager) AcquireWriteLock(key []byte)

AcquireWriteLock acquires a write lock for the given key. If the key doesn't exist, the lock for it will be created.

func (*LockManager) ReleaseReadLocks 

func (lm *LockManager) ReleaseReadLocks(keys []string)

ReleaseReadLocks releases the read locks for the given keys.

func (*LockManager) ReleaseWriteLocks 

func (lm *LockManager) ReleaseWriteLocks(keys []string)

ReleaseWriteLocks releases the write locks for the given keys.

type LogRecord 

type LogRecord struct {
	CheckSum    uint32
	SequenceNum uint64
	KeySize     uint32
	ValueSize   uint32
	Key         []byte
	Value       []byte
}

Simple abstraction for a log record. +----------+---------------+-----------+-------------+-----+-------+ | Checksum | Sequence Num | Key Size | Value Size | Key | Value | | (4 bytes)| (8 bytes) | (4 bytes) | (4 bytes) | ... | ... | +----------+---------------+-----------+-------------+-----+-------+ | <----------- Header (20 bytes) -----------------> | <- Payload -> |

func Deserialize 

func Deserialize(data []byte) (LogRecord, error)

Deserialize converts a byte array to a LogRecord according to the specified format.

func (*LogRecord) ComputeChecksumForRecord 

func (lr *LogRecord) ComputeChecksumForRecord() uint32

ComputeChecksumForRecord calculates checksum for the entire log record (excluding the checksum field itself)

func (*LogRecord) Serialize 

func (lr *LogRecord) Serialize() []byte

Serialize converts the LogRecord to a byte array according to the specified format. The format is: Header (20 bytes) + Payload (Key + Value)

type LogRecordV2 

type LogRecordV2 struct {
	CheckSum    uint32
	SequenceNum uint64
	Term        uint64
	RecordType  byte
	PayloadSize uint32
	Payload     []byte
}

The new log record format in which the header is 25 bytes: 4 + 8 + 8 + 1 + 4 = 25 +----------+---------------+-----------+-----------+-------------+---------+ | Checksum | Sequence Num | Term | RecordType| PayloadSize | Payload | | (4 bytes)| (8 bytes) | (8 bytes) | (1 byte) | (4 bytes) | ... | +----------+---------------+-----------+-----------+-------------+---------+ | <----------- Header (25 bytes) ------------------------------> | | The record type can be: 0x01 -> Put -- a single, non-transactional write: [KeySize (4B)] [ValueSize (4B)] [Key (Var)] [Value (Var)] 0x02 -> Commit -- [NumOps (4B)] [Op1_KeySize] [Op1_ValueSize] [Op1_Key] [Op1_Value] [Op2_KeySize] ...

func DeserializeV2 

func DeserializeV2(data []byte) (LogRecordV2, error)

DeserializeV2 deserializes a log record from a byte array.

func (*LogRecordV2) GetOps 

func (r *LogRecordV2) GetOps() ([]KVRecord, error)

GetOps returns the operations in the log record. If the record type is a Put, it returns a single operation. If the record type is a Commit, it returns a list of operations. Note that each KVRecord is assumed to be valid at this point so it contains no checksum.

type MemState 

type MemState struct {
	// contains filtered or unexported fields
}

MemState is a simple in-memory key-value store. It doesn't have any concurrency protection.

func NewMemState 

func NewMemState() *MemState

func (*MemState) AddSparseIndexEntriesBulk 

func (m *MemState) AddSparseIndexEntriesBulk(segmentID uint64, sparseIndexEntries []sparseIndexEntry)

func (*MemState) AddSparseIndexEntry 

func (m *MemState) AddSparseIndexEntry(segmentID uint64, key []byte, offset int64)

AddSparseIndexEntry adds a new entry to the sparse index.

func (*MemState) ClearState 

func (m *MemState) ClearState()

ClearState clears the memtable and the sparse index.

func (*MemState) Delete 

func (m *MemState) Delete(key []byte)

func (*MemState) Deserialize 

func (m *MemState) Deserialize(data []byte) error

Deserialize deserializes the memtable from the data.

func (*MemState) FindKeyInSparseIndex 

func (m *MemState) FindKeyInSparseIndex(segmentID uint64, key []byte) int64

FindKeyInSparseIndex finds the offset of a key in the sparse index.

func (*MemState) Flush 

func (m *MemState) Flush(filePath string) ([]byte, []byte, error)

Flush flushes the memtable to the segment file. It writes the entries to the segment file in sorted order. TODO: It also updates the offset in the sparse index.

func (*MemState) FlushSparseIndex 

func (m *MemState) FlushSparseIndex(filePath string) error

FlushSparseIndex flushes the sparse index to the sparse index file.

func (*MemState) Get 

func (m *MemState) Get(key []byte) ([]byte, bool)

Get returns the value for a given key. If the key is not found, it returns an error.

func (*MemState) GetAllSegmentIDsDescendingOrder 

func (m *MemState) GetAllSegmentIDsDescendingOrder() ([]uint64, error)

GetAllSegmentFilePathsDescendingOrder gets all the segment files in the descending order of their segment IDs.

func (*MemState) GetAllSortedPairs 

func (m *MemState) GetAllSortedPairs() []pair

GetAllSortedPairs returns all the key-value pairs in the memtable sorted by key.

func (*MemState) GetSparseIndex 

func (m *MemState) GetSparseIndex(segmentID uint64) sparseIndex

GetSparseIndex returns the sparse index for a given segment ID.

func (*MemState) NewIterator 

func (m *MemState) NewIterator() *memtableIterator

func (*MemState) Print 

func (m *MemState) Print()

func (*MemState) Put 

func (m *MemState) Put(key, value []byte)

func (*MemState) RemoveSparseIndexEntry 

func (m *MemState) RemoveSparseIndexEntry(segmentID uint64)

func (*MemState) Serialize 

func (m *MemState) Serialize() ([]byte, error)

Serialize serializes the memtable to a byte slice.

func (*MemState) Size 

func (m *MemState) Size() int64

type MinHeap 

type MinHeap []*MinHeapRecord

MinHeap is a min heap of MinHeapRecord. If the keys are the same, the segment file path is used to break the tie.

func (MinHeap) Len 

func (h MinHeap) Len() int

func (MinHeap) Less 

func (h MinHeap) Less(i, j int) bool

func (*MinHeap) Peek 

func (h *MinHeap) Peek() interface{}

func (*MinHeap) Pop 

func (h *MinHeap) Pop() interface{}

func (*MinHeap) Push 

func (h *MinHeap) Push(x interface{})

func (MinHeap) Swap 

func (h MinHeap) Swap(i, j int)

type MinHeapRecord 

type MinHeapRecord struct {
	Record          KVRecord
	SegmentFilePath string
}

MinHeapRecord is a record that is used to store the records in the min heap. It is used to store the records in the min heap for the compaction process. The record is the KVRecord and the segment file path is the path to the segment file that contains the record.

type Pair 

type Pair struct {
	Key   string
	Value []byte
}

Pair is a key-value pair.

type SegmentMetadata 

type SegmentMetadata struct {
	// contains filtered or unexported fields
}

SegmentMetadata is the metadata for a segment file.

func ReadSegmentMetadata 

func ReadSegmentMetadata(file *os.File) (*SegmentMetadata, error)

ReadSegmentMetadata reads the segment metadata from the file. The bytes are in the following order: checksum (4 bytes), id (8 bytes), level (4 bytes), minKeySize (4 bytes), maxKeySize (4 bytes), filePathSize (4 bytes), minKey (variable), maxKey (variable), filePath (variable)

func (*SegmentMetadata) GetBytes 

func (s *SegmentMetadata) GetBytes() []byte

GetBytes returns the bytes representation of the segment metadata. The bytes are in the following order: checksum (4 bytes), id (8 bytes), level (4 bytes), minKeySize (4 bytes), maxKeySize (4 bytes), filePathSize (4 bytes), minKey (variable), maxKey (variable), filePath (variable)

func (*SegmentMetadata) GetSparseIndexFileName 

func (s *SegmentMetadata) GetSparseIndexFileName() string

GetSparseIndexFileName returns the file name of the sparse index file for the segment.

func (*SegmentMetadata) HasOverlappingKeys 

func (s *SegmentMetadata) HasOverlappingKeys(other *SegmentMetadata) bool

HasOverlappingKeys checks if the two segments have overlapping keys.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server is the main application server that wraps the Raft node and KVStore. It exposes an HTTP API for clients to interact with the database.

func NewServer 

func NewServer(kv *KVStore, raftNode *raft.Raft) *Server

NewServer creates a new server instance.

func (*Server) HandleGet 

func (s *Server) HandleGet(w http.ResponseWriter, r *http.Request)

handleGet is the HTTP handler for reading a key. It performs a strongly consistent read through the Raft leader.

func (*Server) HandlePut 

func (s *Server) HandlePut(w http.ResponseWriter, r *http.Request)

handlePut is the HTTP handler for writing a key. It proposes the change to the Raft cluster.

type SparseIndexRecord 

type SparseIndexRecord struct {
	Checksum  uint32 // 4 bytes
	SegmentID uint64 // 8 bytes
	KeySize   uint32 // 4 bytes
	Offset    int64  // 8 bytes
	Key       []byte // KeySize bytes
}

func DecodeSparseIndexHeader 

func DecodeSparseIndexHeader(headerBytes []byte) (*SparseIndexRecord, error)

type Transaction 

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction allows multiple operations to be performed atomically.

func NewTransaction 

func NewTransaction(store *KVStore) *Transaction

NewTransaction creates a new transaction.

func (*Transaction) Commit 

func (txn *Transaction) Commit() error

Release Per-Key Locks: The final step is to tell the LockManager to release all the individual read and write locks that this transaction was holding. This allows other waiting transactions to proceed.

func (*Transaction) Delete 

func (txn *Transaction) Delete(key []byte) error

Delete deletes a key from the KVStore.

func (*Transaction) Get 

func (txn *Transaction) Get(key []byte) ([]byte, bool)

Get gets a value from the KVStore.

func (*Transaction) GetForUpdate 

func (txn *Transaction) GetForUpdate(key []byte) ([]byte, bool)

GetForUpdate gets a value from the KVStore and acquires a write lock. This should be used when you plan to update the key later in the same transaction. It prevents the deadlock that occurs when Get() acquires a read lock and Put() tries to acquire a write lock.

func (*Transaction) Put 

func (txn *Transaction) Put(key, value []byte) error

Put writes a key-value pair to the KVStore.

func (*Transaction) Rollback 

func (txn *Transaction) Rollback() error

Rollback rolls back the transaction.

type WAL 

type WAL struct {
	// contains filtered or unexported fields
}

func (*WAL) Append 

func (l *WAL) Append(key, value []byte) error

Append appends a new record to the log.

func (*WAL) AppendTransaction 

func (l *WAL) AppendTransaction(term uint64, payload []byte) error

AppendTransaction appends a transaction record to the log.

func (*WAL) Close 

func (l *WAL) Close() error

Close shuts down the log file.

Source Files

View all Source files

Directories

Path Synopsis
cmd
server command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.