tidesdb_go

package module
v0.8.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 13, 2026 License: MPL-2.0 Imports: 4 Imported by: 0

README

tidesdb-go

tidesdb-go is the official GO binding for TidesDB.

TidesDB is a fast and efficient key-value storage engine library written in C. The underlying data structure is based on a log-structured merge-tree (LSM-tree). This GO binding provides a safe, idiomatic GO interface to TidesDB with full support for all features.

Features

  • Full ACID transaction support with savepoints
  • MVCC with five isolation levels from READ UNCOMMITTED to SERIALIZABLE
  • Column families aka isolated key-value stores with independent configuration
  • Bidirectional iterators with forward/backward traversal with seek support
  • TTL(time to live) support with automatic key expiration and internal garbage collection
  • LZ4, LZ4 Fast, ZSTD, Snappy, or no compression
  • Bloom filters with configurable false positive rates
  • Global block CLOCK cache for hot blocks
  • Automatic with configurable thread pools (sorted runs, compaction)
  • Six built-in comparators plus custom registration

For GO usage you can go to the TidesDB GO Reference here.

License

Multiple licenses apply:

Mozilla Public License Version 2.0 (TidesDB)

BSD 3 Clause (Snappy)
BSD 2 (LZ4)
BSD 2 (xxHash - Yann Collet)
BSD (Zstandard)

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Support

For issues, questions, or discussions

Documentation

Overview

Package tidesdb_go Copyright (C) TidesDB

Original Author: Alex Gaetano Padula

Licensed under the Mozilla Public License, v. 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

https://www.mozilla.org/en-US/MPL/2.0/

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Package tidesdb_go Copyright (C) TidesDB

Original Author: Alex Gaetano Padula

Licensed under the Mozilla Public License, v. 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

https://www.mozilla.org/en-US/MPL/2.0/

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

Constants

View Source 
const (
	ErrSuccess     = C.TDB_SUCCESS
	ErrMemory      = C.TDB_ERR_MEMORY
	ErrInvalidArgs = C.TDB_ERR_INVALID_ARGS
	ErrNotFound    = C.TDB_ERR_NOT_FOUND
	ErrIO          = C.TDB_ERR_IO
	ErrCorruption  = C.TDB_ERR_CORRUPTION
	ErrExists      = C.TDB_ERR_EXISTS
	ErrConflict    = C.TDB_ERR_CONFLICT
	ErrTooLarge    = C.TDB_ERR_TOO_LARGE
	ErrMemoryLimit = C.TDB_ERR_MEMORY_LIMIT
	ErrInvalidDB   = C.TDB_ERR_INVALID_DB
	ErrUnknown     = C.TDB_ERR_UNKNOWN
	ErrLocked      = C.TDB_ERR_LOCKED
)

Error codes from TidesDB

Variables

This section is empty.

Functions

This section is empty.

Types

type CacheStats added in v0.6.0 

type CacheStats struct {
	Enabled       bool
	TotalEntries  uint64
	TotalBytes    uint64
	Hits          uint64
	Misses        uint64
	HitRate       float64
	NumPartitions uint64
}

CacheStats is statistics about the block cache.

type ColumnFamily added in v0.5.0 

type ColumnFamily struct {
	// contains filtered or unexported fields
}

ColumnFamily is a TidesDB column family.

func (*ColumnFamily) ClearCommitHook added in v0.8.5 

func (cf *ColumnFamily) ClearCommitHook() error

ClearCommitHook removes the commit hook callback from the column family.

func (*ColumnFamily) Compact added in v0.5.0 

func (cf *ColumnFamily) Compact() error

Compact manually triggers compaction for a column family.

func (*ColumnFamily) FlushMemtable added in v0.6.0 

func (cf *ColumnFamily) FlushMemtable() error

FlushMemtable manually triggers memtable flush for a column family.

func (*ColumnFamily) GetStats added in v0.6.0 

func (cf *ColumnFamily) GetStats() (*Stats, error)

GetStats retrieves statistics about a column family.

func (*ColumnFamily) IsCompacting added in v0.7.0 

func (cf *ColumnFamily) IsCompacting() bool

IsCompacting checks if a column family has a compaction operation in progress.

func (*ColumnFamily) IsFlushing added in v0.7.0 

func (cf *ColumnFamily) IsFlushing() bool

IsFlushing checks if a column family has a flush operation in progress.

func (*ColumnFamily) PurgeCF added in v0.8.7 

func (cf *ColumnFamily) PurgeCF() error

PurgeCF forces a synchronous flush and aggressive compaction for a single column family. Unlike FlushMemtable and Compact (which are non-blocking), PurgeCF blocks until all flush and compaction I/O is complete.

func (*ColumnFamily) RangeCost added in v0.8.4 

func (cf *ColumnFamily) RangeCost(keyA, keyB []byte) (float64, error)

RangeCost estimates the computational cost of iterating between two keys in a column family. The returned value is an opaque double — meaningful only for comparison with other values from the same function. It uses only in-memory metadata and performs no disk I/O. Key order does not matter — the function normalizes the range internally.

func (*ColumnFamily) SetCommitHook added in v0.8.5 

func (cf *ColumnFamily) SetCommitHook(fn CommitHookFunc) error

SetCommitHook sets a commit hook callback for the column family. The hook fires synchronously after WAL write, memtable apply, and commit status marking. Hook failure is logged but does not roll back the commit (data is already durable). Setting a new hook replaces any previously set hook for this column family. Pass nil to disable the hook (equivalent to ClearCommitHook).

func (*ColumnFamily) SyncWal added in v0.8.7 

func (cf *ColumnFamily) SyncWal() error

SyncWal forces an immediate fsync of the active write-ahead log for a column family. This is useful for explicit durability control when using SyncNone or SyncInterval modes.

func (*ColumnFamily) UpdateRuntimeConfig added in v0.7.0 

func (cf *ColumnFamily) UpdateRuntimeConfig(config ColumnFamilyConfig, persistToDisk bool) error

UpdateRuntimeConfig updates runtime-safe configuration settings for a column family. If persistToDisk is true, changes are saved to config.ini in the column family directory.

type ColumnFamilyConfig added in v0.3.0 

type ColumnFamilyConfig struct {
	Name                  string
	WriteBufferSize       uint64
	LevelSizeRatio        uint64
	MinLevels             int
	DividingLevelOffset   int
	KlogValueThreshold    uint64
	CompressionAlgorithm  CompressionAlgorithm
	EnableBloomFilter     bool
	BloomFPR              float64
	EnableBlockIndexes    bool
	IndexSampleRatio      int
	BlockIndexPrefixLen   int
	SyncMode              SyncMode
	SyncIntervalUs        uint64
	ComparatorName        string
	SkipListMaxLevel      int
	SkipListProbability   float32
	DefaultIsolationLevel IsolationLevel
	MinDiskSpace          uint64
	L1FileCountTrigger    int
	L0QueueStallThreshold int
	UseBtree              int
}

ColumnFamilyConfig is the configuration for a column family.

func DefaultColumnFamilyConfig added in v0.5.0 

func DefaultColumnFamilyConfig() ColumnFamilyConfig

DefaultColumnFamilyConfig returns a default column family configuration.

type CommitHookFunc added in v0.8.5 

type CommitHookFunc func(ops []CommitOp, commitSeq uint64) int

CommitHookFunc is the callback invoked synchronously after a transaction commits to a column family. ops contains the full batch of operations for that CF. commitSeq is a monotonic commit sequence number usable as a replication cursor. Return 0 on success, non-zero on failure (logged as warning by TidesDB).

type CommitOp added in v0.8.5 

type CommitOp struct {
	Key      []byte // Key data (copied; safe to retain after callback returns)
	Value    []byte // Value data (nil for deletes; copied; safe to retain)
	TTL      int64  // Time-to-live Unix timestamp (0 = no expiry)
	IsDelete bool   // true if this is a delete operation, false for put
}

CommitOp represents a single operation in a committed transaction batch.

type CompressionAlgorithm added in v0.6.0 

type CompressionAlgorithm int

CompressionAlgorithm the compression algorithm type. 

const (
	NoCompression      CompressionAlgorithm = C.TDB_COMPRESS_NONE
	SnappyCompression  CompressionAlgorithm = C.TDB_COMPRESS_SNAPPY
	LZ4Compression     CompressionAlgorithm = C.TDB_COMPRESS_LZ4
	ZstdCompression    CompressionAlgorithm = C.TDB_COMPRESS_ZSTD
	LZ4FastCompression CompressionAlgorithm = C.TDB_COMPRESS_LZ4_FAST
)

type Config added in v0.5.0 

type Config struct {
	DBPath               string
	NumFlushThreads      int
	NumCompactionThreads int
	LogLevel             LogLevel
	BlockCacheSize       uint64
	MaxOpenSSTables      uint64
	MaxMemoryUsage       uint64
	LogToFile            bool
	LogTruncationAt      uint64
}

Config is the configuration for opening a TidesDB instance.

func DefaultConfig added in v0.6.0 

func DefaultConfig() Config

DefaultConfig returns a default database configuration.

type DbStats added in v0.8.7 

type DbStats struct {
	NumColumnFamilies   int
	TotalMemory         uint64
	AvailableMemory     uint64
	ResolvedMemoryLimit uint64
	MemoryPressureLevel int
	FlushPendingCount   int
	TotalMemtableBytes  int64
	TotalImmutableCount int
	TotalSstableCount   int
	TotalDataSizeBytes  uint64
	NumOpenSstables     int
	GlobalSeq           uint64
	TxnMemoryBytes      int64
	CompactionQueueSize uint64
	FlushQueueSize      uint64
}

DbStats is aggregate statistics across the entire database instance.

type IsolationLevel added in v0.6.0 

type IsolationLevel int

IsolationLevel the transaction isolation level. 

const (
	IsolationReadUncommitted IsolationLevel = C.TDB_ISOLATION_READ_UNCOMMITTED
	IsolationReadCommitted   IsolationLevel = C.TDB_ISOLATION_READ_COMMITTED
	IsolationRepeatableRead  IsolationLevel = C.TDB_ISOLATION_REPEATABLE_READ
	IsolationSnapshot        IsolationLevel = C.TDB_ISOLATION_SNAPSHOT
	IsolationSerializable    IsolationLevel = C.TDB_ISOLATION_SERIALIZABLE
)

type Iterator added in v0.5.0 

type Iterator struct {
	// contains filtered or unexported fields
}

Iterator is a TidesDB iterator.

func (*Iterator) Free added in v0.5.0 

func (iter *Iterator) Free()

Free frees the iterator resources.

func (*Iterator) Key added in v0.5.0 

func (iter *Iterator) Key() ([]byte, error)

Key retrieves the current key from the iterator.

func (*Iterator) Next added in v0.5.0 

func (iter *Iterator) Next() error

Next moves the iterator to the next entry.

func (*Iterator) Prev added in v0.5.0 

func (iter *Iterator) Prev() error

Prev moves the iterator to the previous entry.

func (*Iterator) Seek added in v0.6.0 

func (iter *Iterator) Seek(key []byte) error

Seek positions the iterator at the first key >= target key.

func (*Iterator) SeekForPrev added in v0.6.0 

func (iter *Iterator) SeekForPrev(key []byte) error

SeekForPrev positions the iterator at the last key <= target key.

func (*Iterator) SeekToFirst added in v0.5.0 

func (iter *Iterator) SeekToFirst() error

SeekToFirst positions the iterator at the first key.

func (*Iterator) SeekToLast added in v0.5.0 

func (iter *Iterator) SeekToLast() error

SeekToLast positions the iterator at the last key.

func (*Iterator) Valid added in v0.5.0 

func (iter *Iterator) Valid() bool

Valid returns true if the iterator is positioned at a valid entry.

func (*Iterator) Value added in v0.5.0 

func (iter *Iterator) Value() ([]byte, error)

Value retrieves the current value from the iterator.

type LogLevel added in v0.6.0 

type LogLevel int

LogLevel the logging level. 

const (
	LogDebug LogLevel = C.TDB_LOG_DEBUG
	LogInfo  LogLevel = C.TDB_LOG_INFO
	LogWarn  LogLevel = C.TDB_LOG_WARN
	LogError LogLevel = C.TDB_LOG_ERROR
	LogFatal LogLevel = C.TDB_LOG_FATAL
	LogNone  LogLevel = C.TDB_LOG_NONE
)

type Stats added in v0.6.0 

type Stats struct {
	NumLevels        int
	MemtableSize     uint64
	LevelSizes       []uint64
	LevelNumSSTables []int
	Config           *ColumnFamilyConfig
	TotalKeys        uint64
	TotalDataSize    uint64
	AvgKeySize       float64
	AvgValueSize     float64
	LevelKeyCounts   []uint64
	ReadAmp          float64
	HitRate          float64
	UseBtree         bool
	BtreeTotalNodes  uint64
	BtreeMaxHeight   uint32
	BtreeAvgHeight   float64
}

Stats is statistics about a column family.

type SyncMode added in v0.6.0 

type SyncMode int

SyncMode the sync mode for durability. 

const (
	SyncNone     SyncMode = C.TDB_SYNC_NONE
	SyncFull     SyncMode = C.TDB_SYNC_FULL
	SyncInterval SyncMode = C.TDB_SYNC_INTERVAL
)

type TidesDB 

type TidesDB struct {
	// contains filtered or unexported fields
}

TidesDB is a TidesDB instance.

func Open 

func Open(config Config) (*TidesDB, error)

Open opens a TidesDB instance with the given configuration.

func (*TidesDB) Backup added in v0.7.0 

func (db *TidesDB) Backup(dir string) error

Backup creates an on-disk snapshot of an open database without blocking normal reads/writes. The dir parameter must be a non-existent directory or an empty directory.

func (*TidesDB) BeginTxn 

func (db *TidesDB) BeginTxn() (*Transaction, error)

BeginTxn begins a new transaction with default isolation level.

func (*TidesDB) BeginTxnWithIsolation added in v0.6.0 

func (db *TidesDB) BeginTxnWithIsolation(isolation IsolationLevel) (*Transaction, error)

BeginTxnWithIsolation begins a new transaction with the specified isolation level.

func (*TidesDB) Checkpoint added in v0.8.2 

func (db *TidesDB) Checkpoint(checkpointDir string) error

Checkpoint creates a lightweight, near-instant snapshot of an open database using hard links instead of copying SSTable data. The checkpointDir parameter must be a non-existent directory or an empty directory. The checkpoint can be opened as a normal TidesDB database.

func (*TidesDB) CloneColumnFamily added in v0.8.1 

func (db *TidesDB) CloneColumnFamily(sourceName, destName string) error

CloneColumnFamily creates a complete copy of an existing column family with a new name. The clone contains all the data from the source at the time of cloning. Both column families exist independently after cloning.

func (*TidesDB) Close 

func (db *TidesDB) Close() error

Close closes a TidesDB instance.

func (*TidesDB) CreateColumnFamily 

func (db *TidesDB) CreateColumnFamily(name string, config ColumnFamilyConfig) error

CreateColumnFamily creates a new column family with the given configuration.

func (*TidesDB) DeleteColumnFamily added in v0.8.4 

func (db *TidesDB) DeleteColumnFamily(cf *ColumnFamily) error

DeleteColumnFamily drops a column family by pointer. This is faster than DropColumnFamily when you already hold the ColumnFamily pointer, as it skips the internal name lookup.

func (*TidesDB) DropColumnFamily 

func (db *TidesDB) DropColumnFamily(name string) error

DropColumnFamily drops a column family and all associated data.

func (*TidesDB) GetCacheStats added in v0.6.0 

func (db *TidesDB) GetCacheStats() (*CacheStats, error)

GetCacheStats retrieves statistics about the block cache.

func (*TidesDB) GetColumnFamily added in v0.5.0 

func (db *TidesDB) GetColumnFamily(name string) (*ColumnFamily, error)

GetColumnFamily retrieves a column family by name.

func (*TidesDB) GetComparator added in v0.8.6 

func (db *TidesDB) GetComparator(name string) (bool, error)

GetComparator retrieves a registered comparator by name. Returns true if the comparator is registered, false otherwise.

func (*TidesDB) GetDbStats added in v0.8.7 

func (db *TidesDB) GetDbStats() (*DbStats, error)

GetDbStats retrieves aggregate statistics across the entire database instance. Unlike GetStats (which heap-allocates), GetDbStats fills a caller-provided struct on the stack. No free is needed.

func (*TidesDB) ListColumnFamilies 

func (db *TidesDB) ListColumnFamilies() ([]string, error)

ListColumnFamilies lists all column families in the database.

func (*TidesDB) Purge added in v0.8.7 

func (db *TidesDB) Purge() error

Purge forces a synchronous flush and aggressive compaction for all column families, then drains both the global flush and compaction queues. This blocks until all work is complete.

func (*TidesDB) RegisterComparator added in v0.6.0 

func (db *TidesDB) RegisterComparator(name string, ctxStr string) error

RegisterComparator registers a custom comparator with the database.

func (*TidesDB) RenameColumnFamily added in v0.7.0 

func (db *TidesDB) RenameColumnFamily(oldName, newName string) error

RenameColumnFamily atomically renames a column family and its underlying directory. Waits for any in-progress flush/compaction to complete before renaming.

type Transaction 

type Transaction struct {
	// contains filtered or unexported fields
}

Transaction is a TidesDB transaction.

func (*Transaction) Commit 

func (txn *Transaction) Commit() error

Commit commits the transaction.

func (*Transaction) Delete 

func (txn *Transaction) Delete(cf *ColumnFamily, key []byte) error

Delete removes a key-value pair from the transaction.

func (*Transaction) Free 

func (txn *Transaction) Free()

Free frees the transaction resources.

func (*Transaction) Get 

func (txn *Transaction) Get(cf *ColumnFamily, key []byte) ([]byte, error)

Get retrieves a value from the transaction.

func (*Transaction) NewIterator added in v0.5.0 

func (txn *Transaction) NewIterator(cf *ColumnFamily) (*Iterator, error)

NewIterator creates a new iterator for a column family within a transaction.

func (*Transaction) Put 

func (txn *Transaction) Put(cf *ColumnFamily, key, value []byte, ttl int64) error

Put adds a key-value pair to the transaction. TTL is Unix timestamp (seconds since epoch) for expiration, or -1 for no expiration.

func (*Transaction) ReleaseSavepoint added in v0.6.0 

func (txn *Transaction) ReleaseSavepoint(name string) error

ReleaseSavepoint releases a savepoint without rolling back.

func (*Transaction) Reset added in v0.8.1 

func (txn *Transaction) Reset(isolation IsolationLevel) error

Reset resets a committed or aborted transaction for reuse with a new isolation level. This avoids the overhead of freeing and reallocating transaction resources in hot loops. The transaction must be committed or aborted before reset; resetting an active transaction returns an error.

func (*Transaction) Rollback 

func (txn *Transaction) Rollback() error

Rollback rolls back the transaction.

func (*Transaction) RollbackToSavepoint added in v0.6.0 

func (txn *Transaction) RollbackToSavepoint(name string) error

RollbackToSavepoint rolls back the transaction to a savepoint.

func (*Transaction) Savepoint added in v0.6.0 

func (txn *Transaction) Savepoint(name string) error

Savepoint creates a savepoint within the transaction.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.