ultimate_db

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 29, 2026 License: MIT Imports: 21 Imported by: 0

README

UltimateDB (UDB)

UltimateDB is a high-performance, embedded, transactional database engine designed for edge-cases requiring both relational integrity and high-speed document retrieval. It features a hybrid architecture: a durable, ARIES-compliant slotted-page storage engine for long-term audit storage, paired with an MVCC/OCC lock-free cache for real-time analytical throughput.

Core Architecture

1. Storage Engine
  • Slotted Page Layout: Implements forward-growing slot directories and backward-growing data payloads, preventing fragmentation and ensuring optimal I/O alignment.
  • ARIES-Style WAL: Includes full write-ahead logging with CHECKPOINT support, ensuring ACID compliance and crash recovery.
  • CRC Integrity: Every physical page contains a checksum envelope to detect data corruption (bit rot) at the I/O layer.
2. Transactional & Concurrency Model
  • Dual-Tier Transactionality: Uses a pessimistic, disk-backed durability layer for structural consistency and a non-blocking MVCC (Multi-Version Concurrency Control) cache for performance.
  • OCC Validation: Implements Optimistic Concurrency Control with a ValidateAndCommit phase, allowing high-throughput read/write concurrency without the bottlenecks of standard global locks.
3. Unified Query Language (UQL)

UDB features a custom-built, SQL-inspired engine that abstracts complex CRUD operations into a clean, human-readable syntax.

  • CRUD Operations: INSERT, SELECT, UPDATE, DELETE.
  • Relational Mechanics: Supports JOIN operations and boolean filter expressions.
  • Diagnostic Tools: Includes SHOW METRICS and RECOVER commands for real-time observability and crash recovery.
4. ORM Layer

The built-in ORM uses Go reflection to map structs directly to physical storage. It handles:

  • Automatic JSON serialization.
  • Type-safe retrieval (Find).
  • Tombstone-based deletion.

Getting Started

Installation

Add UDB to your Go project:

# Assuming local development for your startup pipeline
import "ultimate_db"
Basic Usage Pattern
// Initialize the engine and ORM
db := udb.NewDB(bp, wal, metrics)
orm := udb.NewORM(db, index, searcher, walPath)

// Define your model
type IncidentReport struct {
    ID     uint64 `json:"id"`
    Target string `json:"target"`
}

// Perform type-safe CRUD
report := IncidentReport{ID: 101, Target: "Gemma2-9b"}
orm.Insert(report)

// Execute analytical UQL
stmt, _ := udb.ParseUQL("SELECT * FROM incidentreports WHERE adversarial")
results, _ := stmt.Execute(db, index, searcher, nil, nil, walPath)

Metrics & Observability

UDB provides built-in instrumentation. Executing SHOW METRICS returns:

  • Buffer Pool Cache Hit Efficiency.
  • Active Transaction counts.
  • Average Slotted Page Compaction latencies.

Performance

Designed for low-latency retrieval, UDB achieves full pipeline integration (CRUD + Indexing + ORM mapping) in ~0.33ms per complete test lifecycle, making it an ideal candidate for real-time cybersecurity threat analysis and AI auditing.

License

Confidential - Proprietary Database Engine for [Your Startup Name] Infrastructure.

Documentation

Index

Constants

View Source 
const (
	PageSize = 32768

	// Expanded to 32 bytes to comfortably seat a 4-byte CRC, 4-byte Type,
	// 4-byte Slot Count, 4-byte Lower, 4-byte Upper, and an 8-byte NextPageID pointer.
	PageHeaderSize = 32

	RecordHeaderSize = 24
	BTreeHeaderSize  = 24
)
View Source 
const (
	PageTypeInternal = 1
	PageTypeLeaf     = 2
	PrefixHash       = "H:"
)
View Source 
const (
	MaxBlockSize  = 32700
	MagicHeader   = 0x5348
	LookaheadBits = 8
	LookaheadMask = 0xFF
	LookaheadSize = 256
	MaxQueryDepth = 50
)
View Source 
const (
	LogTypeBegin uint8 = iota
	LogTypeUpdate
	LogTypeCommit
	LogTypeAbort
	LogTypeCLR
	LogTypeCheckpoint
)
View Source 
const (
	IndexPageID         PageID = 10
	MetadataPageID      PageID = 11
	PostingsChunkSize          = 256
	DefaultVirtualNodes        = 64
)

Internal Registry Reservations

Variables

View Source 
var ErrPageFull = errors.New("page requires splitting")
View Source 
var GlobalCacheStore = &MVCCCacheStore{
	recs:     make(map[string][]MVCCRecord),
	activeTx: make(map[uint64]uint64),
}
View Source 
var GlobalEncoderTable [256]HuffmanIndexEntry
View Source 
var GlobalForestTable [LookaheadSize]ForestDensityEntry
View Source 
var GlobalOverflowTree [1024]int16

Functions

func BuildDynamicForest added in v1.0.3 

func BuildDynamicForest(lengths []byte)

func Compress 

func Compress(src []byte, dst []byte) (int, error)

func Decompress 

func Decompress(src []byte, dst []byte) (int, error)

func PerformRecovery added in v1.1.0 

func PerformRecovery(db *DB, walPath string) error

PerformRecovery executes the complete ARIES three-pass crash recovery suite. This must be called exactly once during the initialization of the DB struct.

func Tokenize 

func Tokenize(text string) []string

Tokenize acts as the core parsing utility for data indexing and factor isolation.

Types

type AndQuery 

type AndQuery struct{ Left, Right Query }

func (*AndQuery) Execute 

func (q *AndQuery) Execute(s *SegmentSearcher) *RoaringBitmap

type AtomicMetrics added in v1.1.0 

type AtomicMetrics struct {
	// contains filtered or unexported fields
}

AtomicMetrics implements the EngineMetrics interface using high-performance, lock-free atomic operations suitable for concurrent production workloads.

func NewAtomicMetrics added in v1.1.0 

func NewAtomicMetrics() *AtomicMetrics

NewAtomicMetrics initializes an empty metrics container.

func (*AtomicMetrics) Capture added in v1.1.0 

func (m *AtomicMetrics) Capture() EngineMetricsSnapshot

Capture returns a point-in-time snapshot of the database diagnostics.

func (*AtomicMetrics) IncrBufferPoolHit added in v1.1.0 

func (m *AtomicMetrics) IncrBufferPoolHit()

func (*AtomicMetrics) IncrBufferPoolMiss added in v1.1.0 

func (m *AtomicMetrics) IncrBufferPoolMiss()

func (*AtomicMetrics) RecordPageCompactionTime added in v1.1.0 

func (m *AtomicMetrics) RecordPageCompactionTime(d time.Duration)

func (*AtomicMetrics) RecordWalFlushLatency added in v1.1.0 

func (m *AtomicMetrics) RecordWalFlushLatency(d time.Duration)

func (*AtomicMetrics) SetActiveTransactions added in v1.1.0 

func (m *AtomicMetrics) SetActiveTransactions(count int64)

type AuditInterceptor added in v1.2.0 

type AuditInterceptor interface {
	VerifyAccess(subject []byte, action, resource string) bool
	LogAudit(actor, action, msg string)
}

type BM25Scorer added in v1.2.0 

type BM25Scorer struct {
	// contains filtered or unexported fields
}

func NewBM25Scorer added in v1.2.0 

func NewBM25Scorer() *BM25Scorer

func (*BM25Scorer) Score added in v1.2.0 

func (s *BM25Scorer) Score(tf, docLen, avgDocLen float64, totalDocs, docFreq int) float64

type BTree 

type BTree struct {
	// contains filtered or unexported fields
}

func NewBTree 

func NewBTree(bp *BufferPool, rootID PageID) *BTree

func (*BTree) FindLeaf 

func (tree *BTree) FindLeaf(key []byte) (*BTreePage, error)

func (*BTree) Insert 

func (tree *BTree) Insert(key, value []byte) error

func (*BTree) Scan 

func (tree *BTree) Scan(prefix string) ([][]byte, [][]byte, error)

func (*BTree) SplitInternalNode 

func (tree *BTree) SplitInternalNode(node *BTreePage, lockedAncestors []*Page) error

func (*BTree) SplitLeaf 

func (tree *BTree) SplitLeaf(node *BTreePage, lockedAncestors []*Page) error

type BTreeCursor 

type BTreeCursor struct {
	// contains filtered or unexported fields
}

func NewBTreeCursor 

func NewBTreeCursor(tree *BTree) (*BTreeCursor, error)

func (*BTreeCursor) Close 

func (c *BTreeCursor) Close()

func (*BTreeCursor) Next 

func (c *BTreeCursor) Next() ([]byte, []byte, error)

type BTreePage 

type BTreePage struct{ *Page }

func (*BTreePage) BTreeInit 

func (p *BTreePage) BTreeInit()

func (*BTreePage) IsSafeForInsert 

func (p *BTreePage) IsSafeForInsert(requiredBytes uint32) bool

func (*BTreePage) NextLeafID 

func (p *BTreePage) NextLeafID() PageID

func (*BTreePage) NumCells 

func (p *BTreePage) NumCells() uint16

func (*BTreePage) PageType 

func (p *BTreePage) PageType() uint16

func (*BTreePage) ParentID 

func (p *BTreePage) ParentID() PageID

func (*BTreePage) RightmostChildID 

func (p *BTreePage) RightmostChildID() PageID

func (*BTreePage) SetNextLeafID 

func (p *BTreePage) SetNextLeafID(id PageID)

func (*BTreePage) SetNumCells 

func (p *BTreePage) SetNumCells(n uint16)

func (*BTreePage) SetPageType 

func (p *BTreePage) SetPageType(t uint16)

func (*BTreePage) SetParentID 

func (p *BTreePage) SetParentID(id PageID)

func (*BTreePage) SetRightmostChildID 

func (p *BTreePage) SetRightmostChildID(id PageID)

type BatchingWAL 

type BatchingWAL struct {
	// contains filtered or unexported fields
}

func NewBatchingWAL 

func NewBatchingWAL(path string) (*BatchingWAL, error)

func (*BatchingWAL) Append 

func (w *BatchingWAL) Append(txnID uint64, logType uint8, pageID PageID, key, oldValue, newValue []byte) (LogSequenceNumber, error)

Append puts an operational update update vector into the group commit flusher loop.

func (*BatchingWAL) Checkpoint 

func (w *BatchingWAL) Checkpoint() error

func (*BatchingWAL) Close 

func (w *BatchingWAL) Close() error

type BlockDevice added in v1.1.0 

type BlockDevice interface {
	ReadAt(p []byte, off int64) (n int, err error)
	WriteAt(p []byte, off int64) (n int, err error)
	Sync() error
	Stat() (os.FileInfo, error)
	Close() error
}

BlockDevice abstracts raw, random-access I/O operations away from the OS file system. This allows the buffer pool to write to local files, memory maps, or in-memory test arrays.

type BufferPool 

type BufferPool struct {
	// contains filtered or unexported fields
}

func NewBufferPool 

func NewBufferPool(disk *DiskManager, poolSize int, evictor EvictionPolicy, metrics EngineMetrics) *BufferPool

func (*BufferPool) FetchPage 

func (bp *BufferPool) FetchPage(id PageID) (*Page, error)

func (*BufferPool) FlushAll 

func (bp *BufferPool) FlushAll() error

func (*BufferPool) NewPage 

func (bp *BufferPool) NewPage() (*Page, error)

func (*BufferPool) UnpinPage 

func (bp *BufferPool) UnpinPage(id PageID, isDirty bool)

type CheckpointStatement added in v1.1.0 

type CheckpointStatement struct{}

func (*CheckpointStatement) Execute added in v1.1.0 

func (stmt *CheckpointStatement) Execute(db *DB, _ *MemIndex, _ *SegmentSearcher, _ LockManager, _ Codec, _ string) ([][]byte, error)

type ClusterQuery added in v1.2.0 

type ClusterQuery struct {
	QueryID   string `json:"query_id"`
	QueryText string `json:"query_text"`
	Limit     int    `json:"limit"`
}

type Codec added in v1.1.0 

type Codec interface {
	// ID returns a unique identification byte stored as a structural prefix flag.
	ID() uint8
	// Encode compresses raw payload slices into compressed bytes.
	Encode(src []byte) ([]byte, error)
	// Decode decompresses compressed payloads back into their original layout.
	Decode(src []byte) ([]byte, error)
}

Codec standardizes block and record compression layers. Different namespaces can wrap different codec implementations depending on the performance/ratio tradeoff required.

type DB 

type DB struct {
	// contains filtered or unexported fields
}

func NewDB 

func NewDB(bp *BufferPool, wal *BatchingWAL, metrics EngineMetrics) *DB

func (*DB) BeginTxn 

func (db *DB) BeginTxn() uint64

func (*DB) Close 

func (db *DB) Close() error

func (*DB) CommitTxn added in v1.0.2 

func (db *DB) CommitTxn(txnID uint64)

func (*DB) HSet 

func (db *DB) HSet(pageID PageID, txnID uint64, hashKey, field, value []byte, ttl time.Duration) error

func (*DB) NewRangeCursor added in v1.2.0 

func (db *DB) NewRangeCursor(pageID PageID, txnID uint64, startKey, endKey []byte) *RangeCursor

NewRangeCursor initializes a bounded iteration scanner over a specific page matrix

func (*DB) Read 

func (db *DB) Read(pageID PageID, readTxnID uint64, key []byte) ([]byte, error)

Read extracts payloads by searching active slot locations under MVCC rules

func (*DB) ReadCompressed 

func (db *DB) ReadCompressed(pageID PageID, readTxnID uint64, key []byte, codec Codec) ([]byte, error)

func (*DB) Scan 

func (db *DB) Scan(pageID PageID, readTxnID uint64, prefix []byte, iter func(key, value []byte) bool) error

func (*DB) ScanCompressed 

func (db *DB) ScanCompressed(pageID PageID, readTxnID uint64, prefix []byte, codec Codec, iter func(key, value []byte) bool) error

func (*DB) Write 

func (db *DB) Write(pageID PageID, txnID uint64, key, value []byte, ttl time.Duration) error

Write implements full slotted payload insertions guarded by ARIES undo/redo logging frameworks

func (*DB) WriteCompressed 

func (db *DB) WriteCompressed(pageID PageID, txnID uint64, key, value []byte, ttl time.Duration, codec Codec) error

type DeleteStatement added in v1.1.0 

type DeleteStatement struct {
	Filter Query
}

func (*DeleteStatement) Execute added in v1.1.0 

func (stmt *DeleteStatement) Execute(db *DB, _ *MemIndex, s *SegmentSearcher, lockMgr LockManager, _ Codec, _ string) ([][]byte, error)

type DiskManager 

type DiskManager struct {
	// contains filtered or unexported fields
}

func NewDiskManager 

func NewDiskManager(device BlockDevice) *DiskManager

func (*DiskManager) ReadPage 

func (d *DiskManager) ReadPage(id PageID, data *[PageSize]byte) error

func (*DiskManager) WritePage 

func (d *DiskManager) WritePage(id PageID, data *[PageSize]byte) error

type EngineMetrics added in v1.1.0 

type EngineMetrics interface {
	IncrBufferPoolHit()
	IncrBufferPoolMiss()
	RecordWalFlushLatency(d time.Duration)
	RecordPageCompactionTime(d time.Duration)
	SetActiveTransactions(count int64)
}

EngineMetrics handles zero-allocation event tracking across internal data paths. Injecting this interface gives you production visibility into concurrent components.

type EngineMetricsSnapshot added in v1.1.0 

type EngineMetricsSnapshot struct {
	BufferPoolHits       uint64
	BufferPoolMisses     uint64
	BufferPoolHitRatio   float64
	AvgWalFlushLatency   time.Duration
	AvgCompactionLatency time.Duration
	ActiveTransactions   int64
}

Snapshot represents a static copy of system vital signs captured at a specific point in time.

type EngineState added in v1.2.0 

type EngineState struct {
	TotalDocs int     `json:"total_docs"`
	AvgDocLen float64 `json:"avg_doc_len"`
}

type EvictionPolicy added in v1.1.0 

type EvictionPolicy interface {
	// RecordAccess updates the internal tracking metadata whenever a page is touched.
	RecordAccess(id PageID)
	// Evict selects the next page candidate for recycling based on the underlying strategy.
	Evict() (PageID, bool)
	// Remove explicitly deletes a page from the tracking history (e.g., on page drops).
	Remove(id PageID)
}

EvictionPolicy manages the page cache access history and decides which unpinned frames can be safely recycled when the buffer pool is exhausted.

type ForestDensityEntry 

type ForestDensityEntry uint16

func NewForestEntry 

func NewForestEntry(literal byte, consumedBits byte) ForestDensityEntry

func (ForestDensityEntry) Consumed 

func (f ForestDensityEntry) Consumed() byte

func (ForestDensityEntry) Literal 

func (f ForestDensityEntry) Literal() byte

type HSetStatement added in v1.1.0 

type HSetStatement struct {
	HashKey string
	Field   string
	Value   string
}

func (*HSetStatement) Execute added in v1.1.0 

func (stmt *HSetStatement) Execute(db *DB, _ *MemIndex, _ *SegmentSearcher, lockMgr LockManager, _ Codec, _ string) ([][]byte, error)

type HuffmanIndexEntry 

type HuffmanIndexEntry uint64

func NewHuffmanEntry 

func NewHuffmanEntry(code uint64, length byte) HuffmanIndexEntry

func (HuffmanIndexEntry) Code 

func (e HuffmanIndexEntry) Code() uint64

func (HuffmanIndexEntry) Length 

func (e HuffmanIndexEntry) Length() byte

type InsertStatement added in v1.1.0 

type InsertStatement struct {
	DocID        uint64
	Value        string
	IsCompressed bool
}

func (*InsertStatement) Execute added in v1.1.0 

func (stmt *InsertStatement) Execute(db *DB, index *MemIndex, _ *SegmentSearcher, lockMgr LockManager, codec Codec, _ string) ([][]byte, error)

type IntegratedEngine added in v1.2.0 

type IntegratedEngine struct {
	DB          *DB
	Transport   NetworkTransport
	Interceptor AuditInterceptor
	Analyzer    *InternalAnalyzer
	Scorer      *BM25Scorer

	TotalDocs int
	AvgDocLen float64
	// contains filtered or unexported fields
}

func NewIntegratedEngine added in v1.2.0 

func NewIntegratedEngine(db *DB, transport NetworkTransport, interceptor AuditInterceptor) (*IntegratedEngine, error)

func (*IntegratedEngine) AddClusterNode added in v1.2.0 

func (ie *IntegratedEngine) AddClusterNode(nodeID, address string)

func (*IntegratedEngine) InsertDocument added in v1.2.0 

func (ie *IntegratedEngine) InsertDocument(pageID PageID, docID string, text string) error

InsertDocument runs a single-pass transaction, writing data blocks and updating index chunks simultaneously

func (*IntegratedEngine) LocalSearch added in v1.2.0 

func (ie *IntegratedEngine) LocalSearch(queryText string, limit int) ([]SearchResult, error)

func (*IntegratedEngine) ScatterGather added in v1.2.0 

func (ie *IntegratedEngine) ScatterGather(ctx context.Context, subjectID []byte, queryText string, limit int) ([]SearchResult, error)

type InternalAnalyzer added in v1.2.0 

type InternalAnalyzer struct {
	// contains filtered or unexported fields
}

func NewInternalAnalyzer added in v1.2.0 

func NewInternalAnalyzer() *InternalAnalyzer

func (*InternalAnalyzer) Tokenize added in v1.2.0 

func (a *InternalAnalyzer) Tokenize(text string) []string

type JoinResult 

type JoinResult struct {
	Key        []byte
	LeftValue  []byte
	RightValue []byte
}

func SortMergeJoin 

func SortMergeJoin(leftTree, rightTree *BTree) ([]JoinResult, error)

type KVIterator added in v1.1.0 

type KVIterator interface {
	Next() (key []byte, value []byte, err error)
	Close()
}

KVIterator provides cursor-based access across ordered structural keys.

type KVStore added in v1.1.0 

type KVStore interface {
	Begin() TxnHandle
	Get(txn TxnHandle, key []byte) ([]byte, error)
	Put(txn TxnHandle, key []byte, value []byte, ttl time.Duration) error
	Delete(txn TxnHandle, key []byte) error
	NewIterator(txn TxnHandle, prefix []byte) KVIterator
}

KVStore abstracts your storage engine data operations. This allows full-text search indices or execution parsers to sit on top of any backend engine configuration.

type LRUEvictionPolicy added in v1.1.0 

type LRUEvictionPolicy struct {
	// contains filtered or unexported fields
}

LRUEvictionPolicy implements the EvictionPolicy interface using a standard Least Recently Used (LRU) algorithm cache layout.

func NewLRUEvictionPolicy added in v1.1.0 

func NewLRUEvictionPolicy() *LRUEvictionPolicy

NewLRUEvictionPolicy initializes an empty LRU policy tracking space.

func (*LRUEvictionPolicy) Evict added in v1.1.0 

func (p *LRUEvictionPolicy) Evict() (PageID, bool)

Evict identifies and extracts the oldest unpinned page from the back of the tracking history list to be recycled by the buffer pool.

func (*LRUEvictionPolicy) RecordAccess added in v1.1.0 

func (p *LRUEvictionPolicy) RecordAccess(id PageID)

RecordAccess promotes a page to the front of the tracking list, marking it as the most recently used element.

func (*LRUEvictionPolicy) Remove added in v1.1.0 

func (p *LRUEvictionPolicy) Remove(id PageID)

Remove explicitly extracts a page identifier from the eviction history list. This is critical when files are truncated, tables dropped, or pages forced out.

type Lexer added in v1.1.0 

type Lexer struct {
	// contains filtered or unexported fields
}

func NewLexer added in v1.1.0 

func NewLexer(input string) *Lexer

func (*Lexer) NextToken added in v1.1.0 

func (l *Lexer) NextToken() Token

type LockManager added in v1.1.0 

type LockManager interface {
	// Acquire requests an isolation token for a specific transaction handle.
	// It blocks until the lock mode is granted or a deadlock detection timeout fires.
	Acquire(txnID uint64, key string, mode LockMode) error

	// Release explicitly drops an active lock held by a completed transaction.
	Release(txnID uint64, key string) error

	// ReleaseAll handles bulk transaction unlocks during a Commit or Abort phase (Strict 2PL).
	ReleaseAll(txnID uint64) error
}

LockManager enforces concurrency boundaries across logical keys. Implementing this contract protects the engine from race conditions and write skew.

type LockMode added in v1.1.0 

type LockMode uint8

LockMode defines the access privilege requested by a concurrent transaction. 

const (
	LockShared    LockMode = iota // Shared (S) Lock for Read isolation
	LockExclusive                 // Exclusive (X) Lock for Write isolation
)

type LogSequenceNumber added in v1.1.0 

type LogSequenceNumber uint64

type MVCCCacheStore added in v1.1.0 

type MVCCCacheStore struct {
	// contains filtered or unexported fields
}

func (*MVCCCacheStore) BeginOCC added in v1.1.0 

func (c *MVCCCacheStore) BeginOCC() uint64

func (*MVCCCacheStore) Read added in v1.1.0 

func (c *MVCCCacheStore) Read(txID uint64, key string) ([]byte, error)

func (*MVCCCacheStore) ValidateAndCommit added in v1.1.0 

func (c *MVCCCacheStore) ValidateAndCommit(txID uint64, writeSet map[string][]byte, ttl time.Duration) error

type MVCCRecord added in v1.1.0 

type MVCCRecord struct {
	Version   uint64
	TxnID     uint64
	Value     []byte
	Deleted   bool
	ExpiredAt time.Time
}

type MemDevice added in v1.1.0 

type MemDevice struct {
	// contains filtered or unexported fields
}

MemDevice implements a fully concurrent virtual BlockDevice in memory.

func NewMemDevice added in v1.1.0 

func NewMemDevice() *MemDevice

NewMemDevice initializes an empty, virtualized block storage device.

func (*MemDevice) Close added in v1.1.0 

func (m *MemDevice) Close() error

func (*MemDevice) ReadAt added in v1.1.0 

func (m *MemDevice) ReadAt(p []byte, off int64) (int, error)

func (*MemDevice) Stat added in v1.1.0 

func (m *MemDevice) Stat() (os.FileInfo, error)

func (*MemDevice) Sync added in v1.1.0 

func (m *MemDevice) Sync() error

func (*MemDevice) WriteAt added in v1.1.0 

func (m *MemDevice) WriteAt(p []byte, off int64) (int, error)

type MemIndex 

type MemIndex struct {
	// contains filtered or unexported fields
}

func NewMemIndex 

func NewMemIndex() *MemIndex

func (*MemIndex) Add 

func (m *MemIndex) Add(docID uint64, text string)

Add splits inbound text fields and appends unique Document IDs to the postings list.

func (*MemIndex) WriteSegment 

func (m *MemIndex) WriteSegment(path string) error

WriteSegment serializes the postings map into a performance-sorted binary disk segment.

type MetricsStatement added in v1.1.0 

type MetricsStatement struct{}

func (*MetricsStatement) Execute added in v1.1.0 

func (stmt *MetricsStatement) Execute(db *DB, _ *MemIndex, _ *SegmentSearcher, _ LockManager, _ Codec, _ string) ([][]byte, error)

type NetworkTransport added in v1.2.0 

type NetworkTransport interface {
	BroadcastQuery(ctx context.Context, queryPayload []byte) ([][]byte, error)
	GetLocalNodeID() string
}

type NotQuery 

type NotQuery struct{ Left, Right Query }

func (*NotQuery) Execute 

func (q *NotQuery) Execute(s *SegmentSearcher) *RoaringBitmap

type ORM added in v1.1.0 

type ORM struct {
	// contains filtered or unexported fields
}

ORM provides a type-safe data mapping layer on top of ultimate_db.

func NewORM added in v1.1.0 

func NewORM(db *DB, index *MemIndex, searcher *SegmentSearcher, walPath string) *ORM

NewORM instantiates a clean object mapping wrapper.

func (*ORM) Delete added in v1.1.0 

func (o *ORM) Delete(model interface{}) error

Delete writes an immediate tombstong vector expiration marker across targeted keys.

func (*ORM) Find added in v1.1.0 

func (o *ORM) Find(id uint64, out interface{}) error

Find performs an ID-bound direct lookup, mapping raw bytes back to structured runtime objects.

func (*ORM) Insert added in v1.1.0 

func (o *ORM) Insert(model interface{}) error

Insert routes an object through a synthesized UQL command to verify query logging behavior.

func (*ORM) Update added in v1.1.0 

func (o *ORM) Update(model interface{}) error

Update enforces atomicity by writing revised data states over structural slots directly.

type OSFileDevice added in v1.1.0 

type OSFileDevice struct {
	// contains filtered or unexported fields
}

OSFileDevice implements BlockDevice using actual operating system files.

func NewOSFileDevice added in v1.1.0 

func NewOSFileDevice(path string) (*OSFileDevice, error)

NewOSFileDevice initializes a file-backed physical block device.

func (*OSFileDevice) Close added in v1.1.0 

func (d *OSFileDevice) Close() error

func (*OSFileDevice) ReadAt added in v1.1.0 

func (d *OSFileDevice) ReadAt(p []byte, off int64) (int, error)

func (*OSFileDevice) Stat added in v1.1.0 

func (d *OSFileDevice) Stat() (os.FileInfo, error)

func (*OSFileDevice) Sync added in v1.1.0 

func (d *OSFileDevice) Sync() error

func (*OSFileDevice) WriteAt added in v1.1.0 

func (d *OSFileDevice) WriteAt(p []byte, off int64) (int, error)

type OrQuery 

type OrQuery struct{ Left, Right Query }

func (*OrQuery) Execute 

func (q *OrQuery) Execute(s *SegmentSearcher) *RoaringBitmap

type Page 

type Page struct {
	ID         PageID
	Data       [PageSize]byte
	PinCount   atomic.Int32
	IsDirty    bool
	Latch      sync.RWMutex
	MemVersion uint64 // Used for Optimistic Concurrency Control (OCC) validation
}

Page represents a production-hardened 32KB slotted data block frame.

The 32-byte header layout structure on disk is mapped as follows: [0:4] CRC32 Checksum (Validates whole-page byte integrity) [4:8] Page Type Flag (Leaf vs. Internal Node layout) [8:12] Slot Count (Number of tracking elements in forwarding array) [12:16] Lower Free Space Boundary (Offset where slots directory ends; grows forward) [16:20] Upper Free Space Boundary (Offset where record payloads begin; grows backward) [24:32] Next Leaf Page ID (8-byte sequence pointer for B+ Tree leaf chain indexing)

func (*Page) ComputeChecksum added in v1.1.0 

func (p *Page) ComputeChecksum() uint32

ComputeChecksum calculates the IEEE CRC32 across payload data bytes (skipping index 0:4).

func (*Page) GetChecksum added in v1.1.0 

func (p *Page) GetChecksum() uint32

func (*Page) GetLowerBoundary added in v1.1.0 

func (p *Page) GetLowerBoundary() uint32

func (*Page) GetNextPageID added in v1.1.0 

func (p *Page) GetNextPageID() PageID

func (*Page) GetPageType added in v1.1.0 

func (p *Page) GetPageType() uint32

func (*Page) GetSlot added in v1.1.0 

func (p *Page) GetSlot(idx uint32) (Slot, error)

func (*Page) GetSlotCount added in v1.1.0 

func (p *Page) GetSlotCount() uint32

func (*Page) GetUpperBoundary added in v1.1.0 

func (p *Page) GetUpperBoundary() uint32

func (*Page) Init 

func (p *Page) Init()

Init configures an empty, completely zeroed slotted page layout.

func (*Page) IsSafeForInsert added in v1.1.0 

func (p *Page) IsSafeForInsert(requiredBytes uint32) bool

IsSafeForInsert verifies if fragmented free space remains to comfortably host structural expansion.

func (*Page) SetChecksum added in v1.1.0 

func (p *Page) SetChecksum(val uint32)

func (*Page) SetLowerBoundary added in v1.1.0 

func (p *Page) SetLowerBoundary(offset uint32)

func (*Page) SetNextPageID added in v1.1.0 

func (p *Page) SetNextPageID(id PageID)

func (*Page) SetPageType added in v1.1.0 

func (p *Page) SetPageType(t uint32)

func (*Page) SetSlotCount added in v1.1.0 

func (p *Page) SetSlotCount(count uint32)

func (*Page) SetUpperBoundary added in v1.1.0 

func (p *Page) SetUpperBoundary(offset uint32)

func (*Page) WriteSlot added in v1.1.0 

func (p *Page) WriteSlot(idx uint32, s Slot)

type PageID 

type PageID uint64

type Parser 

type Parser struct {
	// contains filtered or unexported fields
}

type ParserUQL added in v1.1.0 

type ParserUQL struct {
	// contains filtered or unexported fields
}

func NewParserUQL added in v1.1.0 

func NewParserUQL(input string) *ParserUQL

type Posting added in v1.2.0 

type Posting struct {
	DocID string  `json:"doc_id"`
	TF    float64 `json:"tf"`
}

type PostingsChunk added in v1.2.0 

type PostingsChunk struct {
	ChunkID  uint32    `json:"chunk_id"`
	Postings []Posting `json:"postings"`
}

type Query 

type Query interface {
	Execute(s *SegmentSearcher) *RoaringBitmap
}

func ParseQuery 

func ParseQuery(input string) (Query, error)

type RangeCursor added in v1.2.0 

type RangeCursor struct {
	// contains filtered or unexported fields
}

RangeCursor handles safe, page-crossing sequential iteration tracking

func (*RangeCursor) Next added in v1.2.0 

func (rc *RangeCursor) Next() ([]byte, []byte, error)

Next advances the cursor to the next valid slot record, seamlessly stepping across underlying physical splits

type RecoverStatement added in v1.1.0 

type RecoverStatement struct{}

func (*RecoverStatement) Execute added in v1.1.0 

func (stmt *RecoverStatement) Execute(db *DB, _ *MemIndex, _ *SegmentSearcher, _ LockManager, _ Codec, walPath string) ([][]byte, error)

type RoaringBitmap added in v1.1.0 

type RoaringBitmap struct {
	Chunks map[uint64][]uint16
}

func NewRoaringBitmap added in v1.1.0 

func NewRoaringBitmap() *RoaringBitmap

func RoaringDifference added in v1.1.0 

func RoaringDifference(r1, r2 *RoaringBitmap) *RoaringBitmap

func RoaringIntersect added in v1.1.0 

func RoaringIntersect(r1, r2 *RoaringBitmap) *RoaringBitmap

func RoaringUnion added in v1.1.0 

func RoaringUnion(r1, r2 *RoaringBitmap) *RoaringBitmap

func (*RoaringBitmap) Add added in v1.1.0 

func (r *RoaringBitmap) Add(val uint64)

func (*RoaringBitmap) Deserialize added in v1.1.0 

func (r *RoaringBitmap) Deserialize(data []byte) error

func (*RoaringBitmap) Len added in v1.1.0 

func (r *RoaringBitmap) Len() int

Add this so you can call len(res) in your tests

func (*RoaringBitmap) Serialize added in v1.1.0 

func (r *RoaringBitmap) Serialize() []byte

func (*RoaringBitmap) ToArray added in v1.1.0 

func (r *RoaringBitmap) ToArray() []uint64

type RoutingEntry added in v1.2.0 

type RoutingEntry struct {
	ID      string
	Address string
	Healthy bool
}

type SearchResult added in v1.2.0 

type SearchResult struct {
	DocID string  `json:"doc_id"`
	Score float64 `json:"score"`
}

type SegmentSearcher 

type SegmentSearcher struct {
	Data []byte
	// contains filtered or unexported fields
}

func (*SegmentSearcher) FetchPostings 

func (s *SegmentSearcher) FetchPostings(target string) []uint64

FetchPostings extracts a raw list of sorted Document IDs matching a targeted vocabulary term string.

func (*SegmentSearcher) Search 

func (s *SegmentSearcher) Search(queryString string) (*RoaringBitmap, error)

Search coordinates query resolution by running the AST execution logic directly.

type SelectStatement added in v1.1.0 

type SelectStatement struct {
	Filter       Query
	Limit        int
	IsCompressed bool
	JoinTable    string
	JoinOnLeft   string
	JoinOnRight  string
}

func (*SelectStatement) Execute added in v1.1.0 

func (stmt *SelectStatement) Execute(db *DB, _ *MemIndex, s *SegmentSearcher, lockMgr LockManager, codec Codec, _ string) ([][]byte, error)

type Slot added in v1.1.0 

type Slot struct {
	Offset uint16
	Length uint16
}

Slot represents a directory entry mapping inside the forwarding array. Each slot consumes exactly 4 bytes: [0:2] Payload Offset, [2:4] Payload Length.

type TermQuery 

type TermQuery struct{ Term string }

func (*TermQuery) Execute 

func (q *TermQuery) Execute(s *SegmentSearcher) *RoaringBitmap

type Token added in v1.1.0 

type Token struct {
	Type    TokenType
	Literal string
}

type TokenType added in v1.1.0 

type TokenType int
const (
	TokenIllegal TokenType = iota
	TokenEOF
	TokenIdent
	TokenString
	TokenNumber

	// Data Keywords (CRUD)
	TokenSelect
	TokenInsert
	TokenInto
	TokenValues
	TokenUpdate
	TokenSet
	TokenDelete
	TokenFrom
	TokenWhere
	TokenJoin
	TokenOn
	TokenAnd
	TokenOr
	TokenNot
	TokenLimit

	// Engine Feature Keywords
	TokenHSet
	TokenCompressed
	TokenRecover
	TokenCheckpoint
	TokenShow
	TokenMetrics

	// Syntax Operators
	TokenLParen
	TokenRParen
	TokenAsterisk
	TokenComma
	TokenEqual
	TokenColon
)

func LookupIdent added in v1.1.0 

func LookupIdent(ident string) TokenType

type TxnHandle added in v1.1.0 

type TxnHandle interface {
	ID() uint64
	Commit() error
	Abort() error
}

TxnHandle represents a logical snapshot state across isolated operations. Updated to expose its ID to lower-level subsystems like the lock manager and recovery loops.

type UQLStatement added in v1.1.0 

type UQLStatement interface {
	Execute(db *DB, index *MemIndex, s *SegmentSearcher, lockMgr LockManager, codec Codec, walPath string) ([][]byte, error)
}

func ParseUQL added in v1.1.0 

func ParseUQL(input string) (UQLStatement, error)

type UpdateStatement added in v1.1.0 

type UpdateStatement struct {
	Filter   Query
	NewValue string
}

func (*UpdateStatement) Execute added in v1.1.0 

func (stmt *UpdateStatement) Execute(db *DB, index *MemIndex, s *SegmentSearcher, lockMgr LockManager, _ Codec, _ string) ([][]byte, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.