Documentation
¶
Overview ¶
Package octetdb provides a durable embedded Go OLTP database.
New applications open one user-selected directory with OpenCatalog, then declare the durable logical topology:
Database
└── Bucket
└── Dataset
└── Records
Record identity is a Dataset plus an application key. KeyedCommand identity is database-wide because one Database.Mutate callback can atomically read and write records in several datasets. A successful mutation has one durable accepted or rejected decision and is exactly retryable while its command ID remains inside the configured dedupe horizon.
Dataset.Get decodes one record into an ordinary Go value. Dataset.Scan and ScanDataset visit detached values in ascending record-key order from one stable logical snapshot. Scans are read-only, honor context cancellation between records, and stop synchronously when a callback returns ScanStop. The current serialized snapshot implementation blocks mutations for the duration of a scan.
Oct is optional. Go applications can use every database and scan capability in this package without the Oct compiler or runtime. Oct's separate filter/map/take query syntax is an advanced authoring path that lowers to its FLOW runtime; Oct compiler internals are not dependencies of this package.
The v0.1 Open and DB account API remains supported. OpenKeyed is a deprecated compatibility path for the distinct, unreleased pre-v0.2 global-key format; new code should use OpenCatalog. Formats are detected fail-closed and are never silently reinterpreted or migrated.
OctetDB is single-process and single-replica. It is not SQL, a network service, an ORM, or a replicated system. The application must ensure that at most one process or handle opens a database directory.
Example ¶
package main
import (
"context"
"fmt"
"log"
"os"
"github.com/yuechen-li-dev/octetdb"
)
func main() {
dir, err := os.MkdirTemp("", "octetdb-example-")
if err != nil {
log.Fatal(err)
}
defer os.RemoveAll(dir)
db, err := octetdb.Open(context.Background(), octetdb.Options{Path: dir})
if err != nil {
log.Fatal(err)
}
defer db.Close()
result, err := db.Submit(context.Background(), octetdb.Command{
ID: "create-1", Kind: octetdb.Create, AccountID: 1, Amount: 100,
})
if err != nil {
log.Fatal(err)
}
account, found := db.Get(1)
fmt.Println(result.Accepted, account.Balance, found)
}
Output: true 100 true
Index ¶
- Constants
- func DecodeResult(decision KeyedDecision, destination any) error
- func Reject(code string) error
- func RejectWithResult(code string, result any) error
- func ScanDataset[T any](ctx context.Context, dataset *Dataset, ...) error
- type Account
- type Bucket
- type BucketInfo
- type Catalog
- type Command
- type CommandKind
- type DB
- func (db *DB) Close() error
- func (db *DB) Get(id uint64) (Account, bool)
- func (db *DB) Snapshot(ctx context.Context) error
- func (db *DB) Stats() Stats
- func (db *DB) Submit(ctx context.Context, command Command) (Result, error)
- func (db *DB) SubmitBatch(ctx context.Context, commands []Command) ([]Result, error)
- type Database
- func (db *Database) Bucket(ctx context.Context, name string) (*Bucket, error)
- func (db *Database) Catalog(ctx context.Context) (Catalog, error)
- func (db *Database) Close() error
- func (db *Database) DatabaseID() string
- func (db *Database) ListBuckets(ctx context.Context) ([]BucketInfo, error)
- func (db *Database) Mutate(ctx context.Context, command KeyedCommand, mutation Mutation) (KeyedDecision, error)
- func (db *Database) Snapshot(ctx context.Context) error
- type DatabaseInfo
- type Dataset
- type DatasetBounds
- type DatasetInfo
- type DatasetKind
- type DatasetOptions
- type DatasetOrigin
- type DatasetRecord
- type Error
- type ErrorKind
- type KeyedCommand
- type KeyedDBdeprecated
- func (db *KeyedDB) Close() error
- func (db *KeyedDB) GetKeyed(ctx context.Context, key string, destination any) (bool, error)deprecated
- func (db *KeyedDB) SnapshotKeyed(ctx context.Context) errordeprecated
- func (db *KeyedDB) SubmitKeyed(ctx context.Context, command KeyedCommand, mutation KeyedMutation) (KeyedDecision, error)deprecated
- type KeyedDecision
- type KeyedMutationdeprecated
- type KeyedOptions
- type KeyedRejection
- type KeyedTxdeprecated
- type Mutation
- type Options
- type Reason
- type Result
- type ScanAction
- type Stats
- type Tx
Examples ¶
Constants ¶
const (
// FormatVersion is the on-disk database format written by this release.
FormatVersion = 1
)
Variables ¶
This section is empty.
Functions ¶
func DecodeResult ¶ added in v0.2.0
func DecodeResult(decision KeyedDecision, destination any) error
DecodeResult decodes a decision result into an application-defined Go value.
func Reject ¶ added in v0.2.0
Reject returns an error that makes SubmitKeyed durably reject a command.
func RejectWithResult ¶ added in v0.2.0
RejectWithResult is Reject with an application-defined JSON result.
func ScanDataset ¶ added in v0.2.0
func ScanDataset[T any](ctx context.Context, dataset *Dataset, visit func(key string, value T) (ScanAction, error)) error
ScanDataset is the typed KeyedJSON scan helper. It decodes each detached record into a new T and creates no intermediate result slice. Ordering, snapshot, cancellation, read-only, and synchronous-stop semantics are the same as Dataset.Scan. A decode failure fails the whole scan; already-observed callback values cannot be revoked.
Types ¶
type Account ¶
type Account struct {
// ID is the account's application-assigned key.
ID uint64
// Balance is the current authoritative integer balance.
Balance int64
// Frozen reports whether outgoing value operations are disabled.
Frozen bool
// Version increments for each applied state change.
Version uint64
}
Account is the authoritative state for one account.
type Bucket ¶ added in v0.2.0
type Bucket struct {
// contains filtered or unexported fields
}
Bucket is a stable handle to a first-level catalog namespace.
func (*Bucket) Dataset ¶ added in v0.2.0
func (bucket *Bucket) Dataset(ctx context.Context, name string, options DatasetOptions) (*Dataset, error)
Dataset opens or durably creates a leaf dataset. Reopening with incompatible kind, type identity, or semantic bounds fails closed.
func (*Bucket) ListDatasets ¶ added in v0.2.0
func (bucket *Bucket) ListDatasets(ctx context.Context) ([]DatasetInfo, error)
ListDatasets returns this bucket's dataset metadata in name order.
type BucketInfo ¶ added in v0.2.0
type BucketInfo struct {
Name string `json:"name"`
}
BucketInfo describes one first-level catalog namespace.
type Catalog ¶ added in v0.2.0
type Catalog struct {
Database DatabaseInfo `json:"database"`
Buckets []BucketInfo `json:"buckets"`
Datasets []DatasetInfo `json:"datasets"`
}
Catalog is a detached, stable view of logical database topology.
type Command ¶
type Command struct {
// ID is the application-assigned idempotency key.
ID string
// Kind selects the account operation.
Kind CommandKind
// AccountID is the primary account or transfer source.
AccountID uint64
// OtherAccountID is the destination for multi-account commands.
OtherAccountID uint64
// Amount is the opening balance or amount operated on, depending on Kind.
Amount int64
}
Command is a uniquely identified account operation. Command IDs provide exact idempotency while retained in the configured dedupe horizon.
type CommandKind ¶
type CommandKind uint8
CommandKind selects one operation in the v0.1 account domain.
const ( // Create creates an account with Amount as its non-negative opening balance. Create CommandKind = iota + 1 // Deposit adds a positive Amount to an existing account. Deposit // Withdraw removes a positive Amount when the account is open and funded. Withdraw // Transfer moves a positive Amount from AccountID to OtherAccountID. Transfer // Freeze prevents withdrawals and transfers from an account. Freeze // Unfreeze restores withdrawal and transfer eligibility. Unfreeze // BeginTransfer records a pending transfer for later confirmation. BeginTransfer // Confirm applies the matching pending transfer. Confirm // Cancel clears a pending transfer without changing balances. Cancel )
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is an open OctetDB handle. A database directory must have at most one open handle across all processes; v0.1 does not yet provide a lock file.
func Open ¶
Open creates or recovers a durable database. Context cancellation is checked before recovery begins; once recovery starts, Open completes it or returns a storage/recovery error so the caller never receives a partially recovered DB.
func (*DB) Get ¶
Get reads the current authoritative account state. It returns false for a missing account and after the DB is closed.
func (*DB) Snapshot ¶
Snapshot atomically installs a snapshot and starts a fresh WAL. Cancellation is honored while waiting for admission, not after snapshot installation starts.
func (*DB) SubmitBatch ¶
SubmitBatch evaluates commands in order and commits all new decisions in one WAL frame and one synchronization. Cancellation can abort while waiting for admission. After admission, the operation runs to a definitive durable result.
type Database ¶ added in v0.2.0
type Database struct {
// contains filtered or unexported fields
}
Database is a durable database with a shallow Database/Bucket/Dataset catalog. Commands and their deduplication identity are database-wide.
func OpenCatalog ¶ added in v0.2.0
OpenCatalog creates or recovers a conventional catalog-aware database. OctetDB owns the product files beneath path.
func (*Database) Bucket ¶ added in v0.2.0
Bucket opens or durably creates a bucket. Structural changes are not data transactions and are complete before this method returns.
func (*Database) Catalog ¶ added in v0.2.0
Catalog returns a deterministic detached topology snapshot.
func (*Database) DatabaseID ¶ added in v0.2.0
DatabaseID returns the durable generated identity of this database.
func (*Database) ListBuckets ¶ added in v0.2.0
func (db *Database) ListBuckets(ctx context.Context) ([]BucketInfo, error)
ListBuckets returns catalog bucket names in deterministic order.
func (*Database) Mutate ¶ added in v0.2.0
func (db *Database) Mutate(ctx context.Context, command KeyedCommand, mutation Mutation) (KeyedDecision, error)
Mutate submits one durable atomic command that may touch several datasets. Command IDs and exact retry decisions are database-wide.
type DatabaseInfo ¶ added in v0.2.0
type DatabaseInfo struct {
ID string `json:"id"`
}
DatabaseInfo describes the logical database identity.
type Dataset ¶ added in v0.2.0
type Dataset struct {
// contains filtered or unexported fields
}
Dataset is a stable handle to a leaf keyed-record collection.
func (*Dataset) Info ¶ added in v0.2.0
func (dataset *Dataset) Info() DatasetInfo
Info returns detached dataset metadata.
func (*Dataset) Scan ¶ added in v0.2.0
func (dataset *Dataset) Scan(ctx context.Context, visit func(DatasetRecord) (ScanAction, error)) error
Scan visits detached records in record-key ascending order. It is read-only: it does not append to the WAL, advance the sequence, or change dedupe state. The scan holds the database admission boundary and observes one stable committed state. Mutations cannot interleave and must not be called from visit. Context cancellation is checked between records.
visit should be deterministic, local, and side-effect-free. It runs synchronously and may return ScanStop for First, Any, or Take behavior.
type DatasetBounds ¶ added in v0.2.0
type DatasetBounds struct {
MaxRecords int `json:"max_records"`
MaxValueBytes int `json:"max_value_bytes"`
}
DatasetBounds are semantic hard limits, not allocation hints.
type DatasetInfo ¶ added in v0.2.0
type DatasetInfo struct {
ID uint64 `json:"id"`
BucketName string `json:"bucket_name"`
Name string `json:"name"`
Kind DatasetKind `json:"kind"`
Origin DatasetOrigin `json:"origin"`
TypeIdentity string `json:"type_identity,omitempty"`
Bounds DatasetBounds `json:"bounds"`
}
DatasetInfo describes one leaf storage object.
type DatasetKind ¶ added in v0.2.0
type DatasetKind string
DatasetKind identifies a product-owned logical dataset representation. M2B intentionally supports only the conventional JSON record path.
const ( // KeyedJSON stores application-defined Go values encoded with encoding/json. KeyedJSON DatasetKind = "keyed_json" )
type DatasetOptions ¶ added in v0.2.0
type DatasetOptions struct {
Kind DatasetKind
TypeIdentity string
MaxRecords int
MaxValueBytes int
}
DatasetOptions declare the stable compatibility identity and bounds of a dataset. Zero bounds inherit the database's configured keyed bounds.
func DefaultDatasetOptions ¶ added in v0.2.0
func DefaultDatasetOptions() DatasetOptions
DefaultDatasetOptions selects opaque keyed JSON and inherited bounds.
type DatasetOrigin ¶ added in v0.2.0
type DatasetOrigin string
DatasetOrigin identifies how a catalog entry was declared.
const ( // GoCatalog means the sane-default Go API created the entry. GoCatalog DatasetOrigin = "go" )
type DatasetRecord ¶ added in v0.2.0
type DatasetRecord struct {
Key string
JSON json.RawMessage
}
DatasetRecord is one detached logical KeyedJSON record. JSON never aliases OctetDB's internal record storage.
func (DatasetRecord) Decode ¶ added in v0.2.0
func (record DatasetRecord) Decode(destination any) error
Decode decodes this logical record into destination.
type Error ¶
type Error struct {
// Kind is the stable category for programmatic handling.
Kind ErrorKind
// Op is the public operation that failed.
Op string
// contains filtered or unexported fields
}
Error describes an OctetDB operation failure without making diagnostic text part of the API contract.
type ErrorKind ¶
type ErrorKind string
ErrorKind identifies an error category suitable for programmatic decisions.
const ( // ErrorInvalidInput means options or a command were malformed. ErrorInvalidInput ErrorKind = "invalid_input" // ErrorCapacity means a configured account or batch bound was exceeded. ErrorCapacity ErrorKind = "capacity" // ErrorStorage means a write, synchronization, or other storage operation failed. ErrorStorage ErrorKind = "storage" // ErrorCorruption means checksums or structural validation found damaged data. ErrorCorruption ErrorKind = "corruption" // ErrorIncompatible means the database or behavioral model cannot be read by this version. ErrorIncompatible ErrorKind = "incompatible" // ErrorClosed means an operation was attempted after Close. ErrorClosed ErrorKind = "closed" // ErrorPoisoned means an earlier durability failure made further writes unsafe. ErrorPoisoned ErrorKind = "poisoned" )
type KeyedCommand ¶ added in v0.2.0
type KeyedCommand struct {
ID string
}
KeyedCommand identifies an application mutation. ID must be stable across retries; OctetDB never silently generates retry identity.
type KeyedDB
deprecated
added in
v0.2.0
type KeyedDB struct {
// contains filtered or unexported fields
}
KeyedDB is a durable, single-process database for application-defined JSON records. Mutations are serialized and atomic across all keys they touch.
Deprecated: use Database with OpenCatalog. KeyedDB retains the distinct pre-v0.2 global-key format only for compatibility.
func OpenKeyed
deprecated
added in
v0.2.0
OpenKeyed creates or recovers a conventional keyed-state database beneath path. OctetDB owns the product files in that directory.
Deprecated: use OpenCatalog. OpenKeyed exists only for the distinct pre-v0.2 global-key development format and never opens a catalog database.
func (*KeyedDB) Close ¶ added in v0.2.0
Close snapshots current keyed state, closes storage, and is idempotent.
func (*KeyedDB) SnapshotKeyed
deprecated
added in
v0.2.0
func (*KeyedDB) SubmitKeyed
deprecated
added in
v0.2.0
func (db *KeyedDB) SubmitKeyed(ctx context.Context, command KeyedCommand, mutation KeyedMutation) (KeyedDecision, error)
SubmitKeyed executes one atomic, durable, exactly deduplicated mutation.
Deprecated: use Database.Mutate.
type KeyedDecision ¶ added in v0.2.0
type KeyedDecision struct {
Sequence uint64
CommandID string
Applied bool
Code string
Result json.RawMessage
Duplicate bool
}
KeyedDecision is the durable outcome of one keyed command. Result is the JSON encoding returned by the mutation function or by RejectWithResult.
type KeyedMutation
deprecated
added in
v0.2.0
KeyedMutation atomically reads and writes application-defined records. A nil error applies all writes. Reject or RejectWithResult records an exact durable rejection and discards all writes. Other errors abort without recording an ID.
Deprecated: use Mutation with OpenCatalog.
type KeyedOptions ¶ added in v0.2.0
type KeyedOptions struct {
// MaxRecords bounds live keys. Zero selects 100,000.
MaxRecords int
// DedupeHorizon bounds retained exact command decisions. Zero selects 100,000.
DedupeHorizon int
// MaxValueBytes bounds one encoded Go value. Zero selects 1 MiB.
MaxValueBytes int
// MaxTransactionBytes bounds all encoded writes in one command. Zero selects 4 MiB.
MaxTransactionBytes int
}
KeyedOptions configures the conventional application-defined keyed-state path. Zero values select bounded product defaults.
func DefaultKeyedOptions ¶ added in v0.2.0
func DefaultKeyedOptions() KeyedOptions
DefaultKeyedOptions returns the documented bounded defaults. It exists to make the default path explicit; the zero KeyedOptions value is equivalent.
type KeyedRejection ¶ added in v0.2.0
type KeyedRejection struct {
Code string
// contains filtered or unexported fields
}
KeyedRejection is an application-domain rejection persisted for exact retry.
func (*KeyedRejection) Error ¶ added in v0.2.0
func (r *KeyedRejection) Error() string
type KeyedTx
deprecated
added in
v0.2.0
type KeyedTx struct {
// contains filtered or unexported fields
}
KeyedTx is the application-facing transaction passed to a KeyedMutation. It is valid only during that callback.
Deprecated: use Tx with Database.Mutate.
type Options ¶
type Options struct {
// Path is the required database directory.
Path string
// MaxAccounts bounds dense account slots; zero selects 100,000.
MaxAccounts int
// DedupeHorizon bounds retained exact command results; zero selects 100,000.
DedupeHorizon int
// BatchMax bounds commands in one SubmitBatch call; zero selects 512.
BatchMax int
}
Options configures a bounded durable database.
type Reason ¶
type Reason string
Reason explains an accepted or rejected domain decision.
const ( // ReasonApplied means the requested state change was applied. ReasonApplied Reason = "applied" // ReasonAwaitingConfirmation means a pending transfer was recorded. ReasonAwaitingConfirmation Reason = "awaiting_confirmation" // ReasonCancelled means a pending transfer was cleared. ReasonCancelled Reason = "cancelled" // ReasonInvalidAmount means the domain rejected the amount. ReasonInvalidAmount Reason = "invalid_amount" // ReasonAccountMissing means a referenced account does not exist. ReasonAccountMissing Reason = "account_missing" // ReasonAccountExists means Create named an existing account. ReasonAccountExists Reason = "account_exists" // ReasonAccountFrozen means a frozen account cannot perform the operation. ReasonAccountFrozen Reason = "account_frozen" // ReasonInsufficientFunds means the source balance was too small. ReasonInsufficientFunds Reason = "insufficient_funds" // ReasonInvalidWorkflow means a pending-transfer transition did not match. ReasonInvalidWorkflow Reason = "invalid_workflow" )
type Result ¶
type Result struct {
// Sequence is the durable decision's monotonic database sequence.
Sequence uint64
// CommandID is the submitted idempotency key.
CommandID string
// Accepted reports whether the domain applied the requested transition.
Accepted bool
// Reason explains the domain decision.
Reason Reason
// Duplicate reports that a retained prior result was returned.
Duplicate bool
}
Result is the durable decision for one command.
type ScanAction ¶ added in v0.2.0
type ScanAction uint8
ScanAction tells Dataset.Scan whether to continue or stop successfully. Stopping is synchronous: no later record is examined or decoded.
const ( // ScanContinue advances to the next record. ScanContinue ScanAction = iota // ScanStop completes the scan successfully after the current record. ScanStop )
type Stats ¶
type Stats struct {
// CommittedSequence is the latest durable decision sequence.
CommittedSequence uint64
// WALBytesWritten is WAL write volume in the current process.
WALBytesWritten uint64
// SnapshotSequence is the installed snapshot sequence, or zero if absent.
SnapshotSequence uint64
// DedupeEntries is the number of currently retained command results.
DedupeEntries int
// AccountCount is the number of allocated account identities.
AccountCount int
}
Stats is a consistent snapshot of the smallest reliable operational counters.
type Tx ¶ added in v0.2.0
type Tx struct {
// contains filtered or unexported fields
}
Tx is valid only inside a Mutation callback. It can access several datasets in one database-wide atomic command.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
bench
command
|
|
|
layoutprobe
command
|
|
|
m5bench
command
|
|
|
m5gen
command
|
|
|
m5probed
command
|
|
|
m5probee
command
|
|
|
m5probeempty
command
|
|
|
m5proberuntime
command
|
|
|
m6gen
command
|
|
|
m7bench
command
|
|
|
m7trace
command
|
|
|
m8bench
command
|
|
|
m8recovery
command
|
|
|
m9storage
command
|
|
|
tigercompare
command
|
|
|
examples
|
|
|
minimal
command
|
|
|
quickstart
command
|
|
|
restart
command
|
|
|
internal
|
|
|
core
Package core implements the canonical, bounded OctetDB account engine.
|
Package core implements the canonical, bounded OctetDB account engine. |
|
m5compiled
Code generated from experiments/M5/generated/snapshot.octest by Oct artifact.
|
Code generated from experiments/M5/generated/snapshot.octest by Oct artifact. |
|
model
Code generated by Oct's compiled backend.
|
Code generated by Oct's compiled backend. |
|
scheduled
Code generated by Oct's compiled backend.
|
Code generated by Oct's compiled backend. |