meldbase

package module
v0.1.0-alpha.13 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 24, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

README

Meldbase

A local document database that keeps application data live.

Meldbase is an experimental, embedded reactive document database for Go and TypeScript applications. It stores documents in one local durable file, exposes typed queries and indexes, and can keep those queries live over HTTP and WebSockets when an application needs a server boundary.

It is designed for product teams that want a small, application-owned data layer—not a hosted MongoDB clone, an ORM, or a distributed database to operate.

Read the documentation →

Why Meldbase

  • Start local. Open one file from Go; there is no separate database service required for the embedded path.
  • Keep reads live. The same query model powers local collections, server fetches, and ordered realtime updates.
  • Own the boundary. A declarative collection policy enforces JWT-derived workspace and owner scope; your application keeps ownership of users, roles, and identity.
  • Operate with evidence. Health probes, an authenticated dashboard, physical backup/restore, logical export/import, offline verification, and a single-node runbook are part of the project—not afterthoughts.

Start in two minutes

Embed it in Go
go get github.com/crapthings/meldbase@latest
package main

import (
    "context"
    "log"

    "github.com/crapthings/meldbase"
)

func main() {
    ctx := context.Background()
    db, err := meldbase.Open("app.meld")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    users := db.Collection("users")
    _, err = users.InsertOne(ctx, meldbase.Document{
        "name": meldbase.String("Ada"),
        "role": meldbase.String("engineer"),
    })
    if err != nil {
        log.Fatal(err)
    }
}

For a runnable tour of durable writes, indexes, reactive queries, reopen and verification:

go run ./cmd/meld demo
Create a secured local node

meld init creates a new, non-overwriting single-node bundle with private credentials, a data directory, backup/rehearsal directories, and a loopback launcher. It does not create users or weaken authentication.

go build -o ./meld ./cmd/meld
./meld init --dir ./meldbase-local
MELDBASE_BIN="$(pwd)/meld" ./meldbase-local/start.sh

The API starts on 127.0.0.1:8080 and the embedded operator dashboard on 127.0.0.1:9091. Your identity service signs the JWTs; the server verifies them and applies the configured collection-access boundary. Use meld --help and the generated API reference while the task-oriented documentation is rebuilt.

What you can build today

Need Meldbase provides
Product data Typed documents, CRUD, safe filters and updates, compound and unique indexes.
Live UI Reactive queries with snapshots, ordered deltas, resume tokens, and a React adapter.
Application API HTTP fetch/mutation endpoints plus ticket-authenticated WebSocket realtime.
Workspace boundary JWT-derived workspace/owner scoping, optional field limits, and RPC-only collections; clients never choose a trusted workspace.
Business logic Typed RPC, durable idempotency, and an optional authenticated Node.js worker boundary.
Operations /livez, /readyz, authenticated metrics/dashboard, inspect, verify, backup, restore, export/import, and restore drills.
TypeScript and React

The TypeScript packages are currently a repository-workspace preview; they are not yet published to npm. The client uses one data-only query contract locally and remotely:

import { MeldbaseClient } from "@meldbase/client"

const db = new MeldbaseClient({
  baseUrl: "https://data.example.com",
  accessToken: () => auth.currentAccessToken(),
})

const query = db.collection("todos").find({ done: false })
const stop = query.subscribe(render, {
  onStatus: (status) => showSyncState(status.state),
})

const todo = await db.collection("todos").insertOne({
  title: "Build something live",
  done: false,
})

React is a thin adapter over the same query object:

const query = useMemo(() => db.collection("todos").find({ done: false }), [db])
const { documents, status, error } = useLiveQuery(query)

See examples/realtime-todos for a runnable browser example.

How the pieces fit

your Go process                    your application boundary
┌──────────────────┐              ┌─────────────────────────┐
│ core             │              │ HTTP + WebSocket server │
│ documents/indexes│── optional ─▶│ JWT + workspace policy  │
│ durable .meld    │              │ SDKs / browser clients  │
└──────────────────┘              └─────────────────────────┘

The embedded core is useful on its own. Add the server only when another process, browser, or service needs access. The server does not become your user directory: it trusts a verified identity provider, derives an actor and active workspace from the token, and constrains configured business collections.

Use it when

Meldbase is a good fit when you want application-owned documents, reactive UI state, and a small operational footprint—for example, a local-first product component, an internal tool, an edge-adjacent service, or a Go application that needs durable live queries without introducing a separate database product.

It is deliberately not a fit when you need MongoDB wire compatibility, sharding, distributed transactions, automatic HA/failover, offline conflict resolution, complex aggregation, or built-in end-user identity. Those are not hidden roadmap promises.

Durable data and operations

Meldbase uses a checksummed copy-on-write format with a durable Commit Log, bounded history, crash recovery, and full offline verification. A physical backup retains database identity and history for recovery; Compact creates an independent file with a new identity when that is what you need.

# The source must be offline: both commands take the exclusive process lock.
meld backup --db /srv/meldbase/data/app.meld \
  --out /srv/meldbase/backups/app.meld > app.receipt.json
meld restore --in /srv/meldbase/backups/app.meld \
  --receipt app.receipt.json \
  --out /srv/meldbase/rehearsals/app-restored.meld
meld verify --db /srv/meldbase/rehearsals/app-restored.meld

For lower storage and transfer cost in an offline window, add --compression gzip to meld backup; its receipt records a digest of the compressed artifact and meld restore verifies it before decompressing. This does not change the full-snapshot RPO or the source writer pause. See single-node backup and restore.

For a running durable server, serve can create the same verified physical artifacts without taking the HTTP service offline. Each scheduled copy briefly blocks writers, leaves readers available, writes a private JSON receipt beside the artifact, and only prunes older receipt-backed pairs. All four scheduling flags are required together so an accidental partial configuration never starts a retention job:

meld serve --db /srv/meldbase/data/app.meld \
  --jwt-hs256-secret-file /etc/meldbase/jwt-hs256.secret \
  --jwt-issuer example.internal \
  --jwt-audience meldbase-api \
  --access-policy-file /etc/meldbase/access-policy.json \
  --backup-dir /srv/meldbase/backups \
  --backup-interval 1h \
  --backup-keep 168 \
  --backup-timeout 10m

The interval is the upper bound for the backup policy's RPO only after a successful backup has been copied off the host; it is not a durability or availability guarantee. RTO is deployment-specific and must be measured by a regular restore drill using a separate rehearsal path. Point-in-time restore between physical backup artifacts is not supported. Keep the source database, backup directory, and rehearsal directory on deliberately chosen storage with restricted permissions.

export and import provide a logical archive for portable data snapshots and import rehearsals. It carries collections, typed documents, and index definitions—not pages, database identity, commit history, or credentials. It is not a replacement for the physical recovery backup above; both paths must be new.

meld export --db /srv/meldbase/data/app.meld \
  --out /srv/meldbase/archives/app.jsonl
meld import --in /srv/meldbase/archives/app.jsonl \
  --out /srv/meldbase/rehearsals/app-imported.meld
meld verify --db /srv/meldbase/rehearsals/app-imported.meld

Before operating real data, verify backup and restore on a disposable copy and inspect the command help for the exact operational contract.

Current alpha status

Meldbase is early-stage and should not yet hold production data. The current format is revision 3 and intentionally evolves during alpha; older alpha files are unsupported. There is no cross-version compatibility promise yet: a future breaking change will receive release-specific guidance when it is actually planned. The project has one current storage path—there is no legacy runtime or fallback engine.

The core, server, SDKs, and single-node tooling are implemented and tested, but the project does not claim blanket power-loss qualification for every filesystem, production-grade automatic HA, or the deferred features listed above. Production qualification describes the repeatable single-node soak, benchmark, query-observer, and destructive fault-injection evidence, along with its safety boundaries. The supported single-node deployment runbook covers the systemd template, probes, configuration, scheduled backups, and safe upgrade/rollback path.

ROADMAP.md is the evidence-oriented readiness checklist. It does not expand the supported single-node alpha boundary.

Documentation

The documentation is being rebuilt around the current product contract. The generated TypeScript API remains available; use the source, tests, and CONTRIBUTING.md as the current implementation reference.

Contributing and verification

go test ./...
go test -race ./...
go vet ./...
pnpm check
pnpm test
pnpm build:example

See CONTRIBUTING.md for contribution guidance, SECURITY.md for private vulnerability reporting.

License

Licensed under the Apache License 2.0.

Documentation

Overview

Package meldbase provides an embedded, durable, realtime document database.

Stability: Alpha. The Go database API can change between alpha releases; operators and applications must follow the release notes and migration instructions when upgrading. Versioned HTTP/WebSocket protocol contracts are documented separately at https://meldbase.com/protocol-stability. HTTP/WebSocket serving is provided by github.com/crapthings/meldbase/server; optional deployment adapters live below github.com/crapthings/meldbase/integrations.

Index

Constants

View Source
const ArrayKind = database.ArrayKind
View Source
const BinaryKind = database.BinaryKind
View Source
const BoolKind = database.BoolKind
View Source
const CreateCollectionOperation = database.CreateCollectionOperation

CreateCollectionOperation is emitted only by the durable database change feed. Ordinary collection creation remains implicit for CRUD callers.

View Source
const CreateIndexOperation = database.CreateIndexOperation
View Source
const DefaultCommitCoordinatorMaxBatch = database.DefaultCommitCoordinatorMaxBatch
View Source
const DefaultCommitCoordinatorMaxDelay = database.DefaultCommitCoordinatorMaxDelay
View Source
const DefaultCommitCoordinatorMaxPending = database.DefaultCommitCoordinatorMaxPending
View Source
const DefaultCommitRetentionMaxBytes = database.DefaultCommitRetentionMaxBytes
View Source
const DefaultCommitRetentionMaxCommits = database.DefaultCommitRetentionMaxCommits
View Source
const DefaultMaxDocumentBytes = database.DefaultMaxDocumentBytes
View Source
const DefaultMaxFileBytes = database.DefaultMaxFileBytes
View Source
const DefaultMaxIndexBuildBytes = database.DefaultMaxIndexBuildBytes
View Source
const DefaultMaxIndexBuildEntries = database.DefaultMaxIndexBuildEntries
View Source
const DefaultMaxQueryCandidates = database.DefaultMaxQueryCandidates
View Source
const DefaultMaxQueryDocumentsExamined = database.DefaultMaxQueryDocumentsExamined
View Source
const DefaultMaxQueryKeysExamined = database.DefaultMaxQueryKeysExamined
View Source
const DefaultMaxQuerySkip = database.DefaultMaxQuerySkip
View Source
const DefaultMaxQuerySortBytes = database.DefaultMaxQuerySortBytes
View Source
const DefaultMaxReactiveViewBytes = database.DefaultMaxReactiveViewBytes
View Source
const DefaultMaxReactiveViewDocuments = database.DefaultMaxReactiveViewDocuments

Reactive views retain matching document versions for incremental ordering and updates, not merely the page currently emitted to a subscriber.

View Source
const DefaultMaxTransactionBytes = database.DefaultMaxTransactionBytes
View Source
const DefaultMaxTransactionChanges = database.DefaultMaxTransactionChanges
View Source
const DefaultReplayDeliveryTimeout = database.DefaultReplayDeliveryTimeout

DefaultReplayDeliveryTimeout bounds how long a replay source can wait for a full caller buffer before it releases the retained-history lease.

View Source
const DefaultReplicationMaxFrameBytes = database.DefaultReplicationMaxFrameBytes
View Source
const DefaultRollbackAnchorOperationTimeout = database.DefaultRollbackAnchorOperationTimeout

DefaultRollbackAnchorOperationTimeout prevents a failed remote trust service from indefinitely holding database publication acknowledgement.

View Source
const DeleteOperation = database.DeleteOperation
View Source
const DiagnosticCanceled = database.DiagnosticCanceled
View Source
const DiagnosticCommit = database.DiagnosticCommit
View Source
const DiagnosticFailure = database.DiagnosticFailure
View Source
const DiagnosticQuery = database.DiagnosticQuery
View Source
const DiagnosticSuccess = database.DiagnosticSuccess
View Source
const Float64Kind = database.Float64Kind
View Source
const IDKind = database.IDKind
View Source
const IndexBuildFailureCanceled = database.IndexBuildFailureCanceled
View Source
const IndexBuildFailureHistoryLost = database.IndexBuildFailureHistoryLost
View Source
const IndexBuildFailureInvalidIndex = database.IndexBuildFailureInvalidIndex
View Source
const IndexBuildFailureNone = database.IndexBuildFailureNone
View Source
const IndexBuildFailureResourceLimit = database.IndexBuildFailureResourceLimit
View Source
const IndexBuildFailureUniqueConflict = database.IndexBuildFailureUniqueConflict
View Source
const IndexBuildPhaseCatchUp = database.IndexBuildPhaseCatchUp
View Source
const IndexBuildPhaseFailed = database.IndexBuildPhaseFailed
View Source
const IndexBuildPhaseReady = database.IndexBuildPhaseReady
View Source
const IndexBuildPhaseScan = database.IndexBuildPhaseScan
View Source
const InsertOperation = database.InsertOperation
View Source
const Int64Kind = database.Int64Kind
View Source
const NullKind = database.NullKind
View Source
const ObjectKind = database.ObjectKind
View Source
const PageSize = database.PageSize
View Source
const QueryDeltaAdd = database.QueryDeltaAdd
View Source
const QueryDeltaChange = database.QueryDeltaChange
View Source
const QueryDeltaMove = database.QueryDeltaMove
View Source
const QueryDeltaRemove = database.QueryDeltaRemove
View Source
const RecoveryAutomatic = database.RecoveryAutomatic
View Source
const RecoveryRequireClean = database.RecoveryRequireClean
View Source
const ReplaceIndexOperation = database.ReplaceIndexOperation
View Source
const ReplicationAckFrame = database.ReplicationAckFrame
View Source
const ReplicationBatchFrame = database.ReplicationBatchFrame
View Source
const ReplicationHelloFrame = database.ReplicationHelloFrame
View Source
const ReplicationProtocolVersion = database.ReplicationProtocolVersion

ReplicationProtocolVersion is intentionally separate from the browser realtime protocol. It transports durable database positions between trusted servers, not end-user query subscriptions.

View Source
const ReplicationResyncFrame = database.ReplicationResyncFrame
View Source
const StorageFormatCurrent = database.StorageFormatCurrent
View Source
const StorageFormatUnknown = database.StorageFormatUnknown
View Source
const StringKind = database.StringKind
View Source
const TimeKind = database.TimeKind
View Source
const UpdateOperation = database.UpdateOperation

Variables

View Source
var DefaultQueryLimits = database.DefaultQueryLimits
View Source
var ErrBackupDestinationExists = database.ErrBackupDestinationExists
View Source
var ErrBackupUnsupported = database.ErrBackupUnsupported
View Source
var ErrClosed = database.ErrClosed
View Source
var ErrCommitOutcomeUnknown = database.ErrCommitOutcomeUnknown

ErrCommitOutcomeUnknown means cancellation or a lost caller connection raced an already-admitted durable write. Callers must reconcile by the returned document ID(s), rather than retrying business logic blindly.

View Source
var ErrCompactionDestinationExists = database.ErrCompactionDestinationExists
View Source
var ErrCompactionUnsupported = database.ErrCompactionUnsupported
View Source
var ErrCompoundIndexUnsupported = database.ErrCompoundIndexUnsupported
View Source
var ErrCorrupt = database.ErrCorrupt
View Source
var ErrDatabaseIdentity = database.ErrDatabaseIdentity
View Source
var ErrDatabaseLocked = database.ErrDatabaseLocked
View Source
var ErrDestinationExists = database.ErrDestinationExists
View Source
var ErrDiagnosticsActive = database.ErrDiagnosticsActive
View Source
var ErrDuplicateID = database.ErrDuplicateID
View Source
var ErrDuplicateKey = database.ErrDuplicateKey
View Source
var ErrDurability = database.ErrDurability
View Source
var ErrDurableConsumerExists = database.ErrDurableConsumerExists
View Source
var ErrDurableConsumerNotFound = database.ErrDurableConsumerNotFound
View Source
var ErrDurableConsumerUnsupported = database.ErrDurableConsumerUnsupported
View Source
var ErrHistoryLost = database.ErrHistoryLost
View Source
var ErrImmutableID = database.ErrImmutableID
View Source
var ErrIndexBuildExists = database.ErrIndexBuildExists
View Source
var ErrIndexBuildFailed = database.ErrIndexBuildFailed
View Source
var ErrIndexBuildNotFound = database.ErrIndexBuildNotFound
View Source
var ErrIndexBuildSchedulerRunning = database.ErrIndexBuildSchedulerRunning
View Source
var ErrIndexBuildUnsupported = database.ErrIndexBuildUnsupported
View Source
var ErrInsecureFileMode = database.ErrInsecureFileMode
View Source
var ErrInvalidCollection = database.ErrInvalidCollection
View Source
var ErrInvalidCommitCoordinatorOptions = database.ErrInvalidCommitCoordinatorOptions
View Source
var ErrInvalidDelta = database.ErrInvalidDelta
View Source
var ErrInvalidDocument = database.ErrInvalidDocument
View Source
var ErrInvalidFilter = database.ErrInvalidFilter
View Source
var ErrInvalidIndex = database.ErrInvalidIndex
View Source
var ErrInvalidIndexBuildSchedulerOptions = database.ErrInvalidIndexBuildSchedulerOptions
View Source
var ErrInvalidReclamationOptions = database.ErrInvalidReclamationOptions
View Source
var ErrInvalidReplayDeliveryTimeout = database.ErrInvalidReplayDeliveryTimeout
View Source
var ErrInvalidResourceLimits = database.ErrInvalidResourceLimits
View Source
var ErrInvalidRollbackProtection = database.ErrInvalidRollbackProtection
View Source
var ErrInvalidUpdate = database.ErrInvalidUpdate
View Source
var ErrLogicalArchiveDestinationExists = database.ErrLogicalArchiveDestinationExists
View Source
var ErrLogicalArchiveUnsupported = database.ErrLogicalArchiveUnsupported
View Source
var ErrMutationLimit = database.ErrMutationLimit
View Source
var ErrNotFound = database.ErrNotFound
View Source
var ErrPrimaryWriteFence = database.ErrPrimaryWriteFence
View Source
var ErrQueryBudget = database.ErrQueryBudget

ErrQueryBudget reports that one query exceeded an execution-work budget. Callers can use errors.Is to distinguish this from invalid query input.

View Source
var ErrReclamationConflict = database.ErrReclamationConflict
View Source
var ErrReclamationUnsupported = database.ErrReclamationUnsupported
View Source
var ErrRecoveryRequired = database.ErrRecoveryRequired
View Source
var ErrReplicaPromoted = database.ErrReplicaPromoted
View Source
var ErrReplicaPromotionAuthority = database.ErrReplicaPromotionAuthority
View Source
var ErrReplicaPromotionFence = database.ErrReplicaPromotionFence
View Source
var ErrReplicaPromotionWriteFence = database.ErrReplicaPromotionWriteFence
View Source
var ErrReplicaProtocol = database.ErrReplicaProtocol
View Source
var ErrReplicaReadOnly = database.ErrReplicaReadOnly
View Source
var ErrReplicaSequence = database.ErrReplicaSequence
View Source
var ErrReplicaSourceActive = database.ErrReplicaSourceActive
View Source
var ErrResourceLimit = database.ErrResourceLimit
View Source
var ErrRollbackAnchor = database.ErrRollbackAnchor
View Source
var ErrRollbackAnchorRequired = database.ErrRollbackAnchorRequired
View Source
var ErrRollbackDetected = database.ErrRollbackDetected
View Source
var ErrSlowConsumer = database.ErrSlowConsumer
View Source
var ErrUnsupportedFormat = database.ErrUnsupportedFormat
View Source
var ErrVerificationUnsupported = database.ErrVerificationUnsupported
View Source
var ErrWriteConflict = database.ErrWriteConflict
View Source
var ErrWriteTransactionUnsupported = database.ErrWriteTransactionUnsupported

Functions

func MarshalQuerySpecJSON

func MarshalQuerySpecJSON(query QuerySpec) ([]byte, error)

MarshalQuerySpecJSON emits the canonical, data-only wire representation used for transport fingerprints and cross-language conformance.

func MarshalReplicationFrame

func MarshalReplicationFrame(frame ReplicationFrame, limits ReplicationFrameLimits) ([]byte, error)

MarshalReplicationFrame returns a strict JSON frame with canonical document images encoded as base64 of the storage-independent typed document codec. It is suitable for WebSocket binary/text messages, QUIC streams or framed RPC, but it does not provide authentication or encryption itself.

func MarshalWireDocument

func MarshalWireDocument(document Document) ([]byte, error)

func MarshalWireValue

func MarshalWireValue(value Value) ([]byte, error)

func ValidateStrictJSON

func ValidateStrictJSON(data []byte, maxBytes int) error

ValidateStrictJSON rejects oversized, trailing, deeply nested, and duplicate-key JSON before a transport decodes it into structs or maps.

Types

type ArchiveBootstrap

type ArchiveBootstrap = database.ArchiveBootstrap

ArchiveBootstrap binds an exact verified physical snapshot to the durable database change feed that was pinned before that snapshot began.

A receiver must persist and verify Backup, then drain and Ack every batch up through SnapshotToken without applying it (the snapshot already contains those effects). It can then apply and Ack later batches in order. This avoids the bootstrap/tail gap without inventing a second, weaker history contract.

type BackupResult

type BackupResult = database.BackupResult

func ImportPhysicalBackup

func ImportPhysicalBackup(ctx context.Context, source io.Reader, destination string, expected BackupResult, options PhysicalBackupImportOptions) (BackupResult, error)

ImportPhysicalBackup receives one exact Backup artifact into a new local path. It writes a private temporary file, checks the claimed byte count and SHA-256 while streaming, runs the complete offline graph/index verifier, then publishes with the same no-overwrite link-and-directory-sync commit point as backup and migration.

source is intentionally transport-neutral. A WebSocket, HTTP response, QUIC stream, or removable-media reader may supply it, but transport cancellation must close or honor ctx itself: a generic io.Reader cannot be interrupted while blocked in Read. The destination is never opened as a writable DB by this function; callers normally open the successfully imported file through OpenFollower before applying a replication tail.

type BackupStats

type BackupStats = database.BackupStats

type Change

type Change = database.Change

type ChangeBatch

type ChangeBatch = database.ChangeBatch

type Collection

type Collection = database.Collection

type CommitCoordinatorOptions

type CommitCoordinatorOptions = database.CommitCoordinatorOptions

CommitCoordinatorOptions controls optional group commit for ordinary InsertMany, filter Update and filter Delete operations. It is disabled by default, so opening an existing database never changes write scheduling unexpectedly.

A coordinator group has one physical Meta publication but retains one logical commit token for every admitted write request. Public write transactions, atomic RPC, index builds and other maintenance operations remain exclusive commits. When rollback protection is configured, the coordinator advances the external anchor only after the group's final Meta publication is durable and before acknowledging any member.

type CommitCoordinatorStats

type CommitCoordinatorStats = database.CommitCoordinatorStats

CommitCoordinatorStats is a fixed-cardinality snapshot of the optional

write-admission scheduler. It is included in DBStats and the versioned

admin schema, so applications can alert on admission pressure without inspecting a mutable queue or adding application labels.

type CommitRetentionPolicy

type CommitRetentionPolicy = database.CommitRetentionPolicy

CommitRetentionPolicy bounds logical Commit Log history by both commit count and canonical encoded bytes. Zero fields select production defaults. Active replay leases may temporarily exceed either budget rather than losing history under a reader.

type CommitStats

type CommitStats = database.CommitStats

type CompactionOptions

type CompactionOptions = database.CompactionOptions

CompactionOptions configures newly written replacement or compaction files. ResourceLimits govern transient index construction as well as the reopened destination handle; zero fields select production defaults.

type CompactionStats

type CompactionStats = database.CompactionStats

type Cursor

type Cursor = database.Cursor

type DB

type DB = database.DB

func New

func New() *DB

func NewWithOptions

func NewWithOptions(options DatabaseOptions) (*DB, error)

NewWithOptions creates an in-memory database with explicit resource limits.

func Open

func Open(path string) (*DB, error)

Open creates or opens a Meldbase durable database in the current format.

func OpenWithOptions

func OpenWithOptions(path string, options OpenOptions) (*DB, error)

type DBStats

type DBStats = database.DBStats

DBStats is a point-in-time, allocation-bounded view of database health. Counters are process-lifetime values and reset when the database is reopened. Persistent state such as CommitSequence is read from the database itself.

Stats deliberately exposes no user values, document IDs, query parameters, or callbacks. It is safe for an admin sampler to call periodically, but it is not intended to be called on every database operation.

type DatabaseOptions

type DatabaseOptions = database.DatabaseOptions

DatabaseOptions configures an in-memory database.

type DeleteResult

type DeleteResult = database.DeleteResult

type DiagnosticEvent

type DiagnosticEvent = database.DiagnosticEvent

type DiagnosticKind

type DiagnosticKind = database.DiagnosticKind

type DiagnosticOutcome

type DiagnosticOutcome = database.DiagnosticOutcome

type DiagnosticSnapshot

type DiagnosticSnapshot = database.DiagnosticSnapshot

type DiagnosticStats

type DiagnosticStats = database.DiagnosticStats

type Diagnostics

type Diagnostics = database.Diagnostics

Diagnostics owns a fixed-capacity event ring. Close disables future timing and recording but keeps the retained snapshot readable by its owner.

type DiagnosticsOptions

type DiagnosticsOptions = database.DiagnosticsOptions

DiagnosticsOptions controls opt-in detailed events. Defaults retain 256 events and record failed, >=50ms queries and >=100ms durable commits. Setting RecordAll is intended only for short development sessions. SampleEvery adds a deterministic one-in-N sample of otherwise fast successful operations.

type Document

type Document = database.Document

func NewDocument

func NewDocument(fields map[string]any) (Document, error)

func UnmarshalWireDocument

func UnmarshalWireDocument(data []byte, limits QueryLimits) (Document, error)

func UnmarshalWireInputDocument

func UnmarshalWireInputDocument(data []byte, limits QueryLimits) (Document, error)

type DocumentCacheStats

type DocumentCacheStats = database.DocumentCacheStats

type DocumentID

type DocumentID = database.DocumentID

func NewDocumentID

func NewDocumentID() (DocumentID, error)

func ParseDocumentID

func ParseDocumentID(s string) (DocumentID, error)

type DurabilityStats

type DurabilityStats = database.DurabilityStats

DurabilityStats is retained in the admin wire contract. Current-format databases do not use a WAL or checkpoints, so every field is zero.

type DurableChangeBatch

type DurableChangeBatch = database.DurableChangeBatch

DurableChangeBatch is one globally ordered Commit Log position projected to one collection. Changes is empty when another collection or private catalog change advanced the durable position; callers must still Ack that Token after processing it so the checkpoint can advance without pinning history forever.

This is deliberately a document-change feed, not a full replication protocol: it does not expose private System records, index definitions, collection lifecycle or raw storage bytes.

type DurableChangeSubscription

type DurableChangeSubscription = database.DurableChangeSubscription

DurableChangeSubscription is a pull/acknowledge bridge over a durable checkpoint. Batches remain ordered. Ack must be called only after the consumer's external side effect for that token is durable.

type DurableDatabaseChangeBatch

type DurableDatabaseChangeBatch = database.DurableDatabaseChangeBatch

DurableDatabaseChangeBatch is one globally ordered Commit Log position projected into public document and catalog events. It is the semantic source for archive and single-writer-follower protocols: callers must Ack only after the externally applied effect for Token is durable.

Private System records, raw pages, index-build progress and retention control records are deliberately excluded. A batch can therefore be empty when a private record advanced a retained position; it must still be Acked.

type DurableDatabaseChangeSubscription

type DurableDatabaseChangeSubscription = database.DurableDatabaseChangeSubscription

DurableDatabaseChangeSubscription is a crash-resumable pull/acknowledge feed over the complete public database. It exposes collection creation, index publication and document changes in exact Commit Log order. It does not itself copy a bootstrap snapshot or apply changes to a follower; those transport and ownership contracts are intentionally separate.

type ExplainAccessSource

type ExplainAccessSource = database.ExplainAccessSource

type ExplainAdvice

type ExplainAdvice = database.ExplainAdvice

type ExplainBound

type ExplainBound = database.ExplainBound

type ExplainBudget

type ExplainBudget = database.ExplainBudget

type ExplainResult

type ExplainResult = database.ExplainResult

type Filter

type Filter = database.Filter

type Follower

type Follower = database.Follower

Follower owns a local, read-only database that advances only through validated DurableDatabaseChangeBatch values. It is the local application half of a future remote replication protocol; transport authentication, snapshot transfer and promotion deliberately remain outside this type.

func OpenFollower

func OpenFollower(path string, options OpenOptions) (*Follower, error)

OpenFollower opens a physical archive/bootstrap copy as a replica. Normal public mutations on DB return ErrReplicaReadOnly; use Apply to advance the next source token. The returned DB remains fully queryable and reactive.

type FollowerPromotionAuthority

type FollowerPromotionAuthority = database.FollowerPromotionAuthority

FollowerPromotionAuthority must make its returned fence durable before it returns. Implementations normally revoke a primary lease through a quorum controller or external consensus store.

type FollowerPromotionFence

type FollowerPromotionFence = database.FollowerPromotionFence

FollowerPromotionFence is a controller-issued, non-empty epoch proving the old primary's write authority was fenced for this database/token. Epoch is deliberately opaque to Meldbase: a controller may use it as an epoch ID or a compact signed lease certificate (as integrations/primarylease does). Meldbase does not invent a local substitute for that distributed safety decision.

type FollowerPromotionFenceBinder

type FollowerPromotionFenceBinder = database.FollowerPromotionFenceBinder

FollowerPromotionFenceBinder binds one controller-issued promotion fence to the local primary-write guard before a follower becomes writable. The binder may update caller-owned local lease/epoch state, but must not enable writes until it has accepted the exact fence. It runs on the promotion control path, outside the DB writer lock; unlike ValidatePrimaryWrite it may coordinate with the controller if the implementation needs to.

A promoted follower requires this interface in addition to PrimaryWriteFence. Otherwise an unrelated always-allow guard could make a one-time promotion certificate appear to grant permanent write authority.

type FollowerPromotionRequest

type FollowerPromotionRequest = database.FollowerPromotionRequest

FollowerPromotionRequest is the exact local state an external fencing system must certify before this process can become writable primary.

type IndexBuildFailure

type IndexBuildFailure = database.IndexBuildFailure

type IndexBuildID

type IndexBuildID = database.IndexBuildID

IndexBuildID identifies one durable, resumable Storage index build.

func ParseIndexBuildID

func ParseIndexBuildID(value string) (IndexBuildID, error)

type IndexBuildPhase

type IndexBuildPhase = database.IndexBuildPhase

type IndexBuildScheduler

type IndexBuildScheduler = database.IndexBuildScheduler

type IndexBuildSchedulerOptions

type IndexBuildSchedulerOptions = database.IndexBuildSchedulerOptions

IndexBuildSchedulerOptions configures an explicit default-off runner. Each task receives a bounded time quantum, then yields durable progress so CRUD and other builds can proceed between quanta.

type IndexBuildSchedulerStats

type IndexBuildSchedulerStats = database.IndexBuildSchedulerStats

type IndexBuildStats

type IndexBuildStats = database.IndexBuildStats

type IndexBuildStatus

type IndexBuildStatus = database.IndexBuildStatus

IndexBuildStatus is durable progress. EntryCount and CanonicalBytes describe the current private Secondary tree, not transient Go heap usage.

type IndexCatalogEntry

type IndexCatalogEntry = database.IndexCatalogEntry

IndexCatalogEntry is an immutable operator-facing description of one published index. It contains no document keys, values, or cardinalities. Index management remains a deployment concern; this is intentionally a read-only catalog for CLIs and protected operator surfaces.

type IndexDefinition

type IndexDefinition = database.IndexDefinition

type IndexField

type IndexField = database.IndexField

IndexField is one ordered component of an index definition. Order must be 1 (ascending) or -1 (descending); fields are evaluated left to right.

type IndexOptions

type IndexOptions = database.IndexOptions

IndexOptions controls complete-tuple uniqueness.

type Kind

type Kind = database.Kind

type LogicalArchiveImportOptions

type LogicalArchiveImportOptions = database.LogicalArchiveImportOptions

LogicalArchiveImportOptions bounds an untrusted logical archive. Zero MaxBytes selects the normal storage-file limit; the receiver owns this cap.

type LogicalArchiveResult

type LogicalArchiveResult = database.LogicalArchiveResult

LogicalArchiveResult is the portable, data-only archive receipt. SHA256 covers every JSONL record before the final end record; the end record stores the same digest so an importer can reject truncation or alteration.

func ImportLogicalArchive

func ImportLogicalArchive(ctx context.Context, source io.Reader, destination string, options LogicalArchiveImportOptions) (result LogicalArchiveResult, resultErr error)

ImportLogicalArchive validates and applies a portable archive into a private temporary database, verifies that database offline, then atomically publishes it at destination. A malformed archive never leaves a destination database.

type Maintenance

type Maintenance = database.Maintenance

Maintenance owns one background reclamation loop. Stop is idempotent and waits for an active scan to observe cancellation. Closing the DB also stops the loop through the DB lifecycle channel.

type MaintenanceOptions

type MaintenanceOptions = database.MaintenanceOptions

MaintenanceOptions configures an explicit default-off maintenance loop. Every run uses online optimistic reclamation; runs never overlap.

type MaintenanceStats

type MaintenanceStats = database.MaintenanceStats

type MutationSpec

type MutationSpec = database.MutationSpec

func CompileUpdate

func CompileUpdate(update Update) (MutationSpec, error)

func DecodeMutationSpecJSON

func DecodeMutationSpecJSON(data []byte, limits QueryLimits) (MutationSpec, error)

type OpenOptions

type OpenOptions = database.OpenOptions

OpenOptions configures the current durable storage format.

type Operation

type Operation = database.Operation

type OperationalState

type OperationalState = database.OperationalState

OperationalState is a minimal, allocation-free serving-state snapshot. A fail-stop durability error preserves reads from the last committed state but disables writes; a closed database is neither readable nor writable.

type PageCacheStats

type PageCacheStats = database.PageCacheStats

type PhysicalBackupImportOptions

type PhysicalBackupImportOptions = database.PhysicalBackupImportOptions

PhysicalBackupImportOptions bounds an untrusted physical-backup stream before it can consume local disk. Zero selects the normal file limit. Deployments with a deliberately larger database must set MaxBytes explicitly on the receiving side; a sender never chooses that authority.

type PrimaryWriteFence

type PrimaryWriteFence = database.PrimaryWriteFence

PrimaryWriteFence is the local enforcement hook for an external primary election/fencing system. Its implementation normally checks an atomically refreshed lease epoch and expiry, not the network. Returning an error rejects the whole logical commit before storage mutation; it never poisons the database or advances a token.

Implementations must not call back into DB and must return promptly: the check runs while the writer has admitted a commit. Election, renewal, certificate rotation and old-primary revocation remain external concerns.

type PrimaryWriteFenceRequest

type PrimaryWriteFenceRequest = database.PrimaryWriteFenceRequest

PrimaryWriteFenceRequest binds a proposed primary mutation to this database identity and exact next logical commit sequence. A lease implementation must reject when its external authority/epoch/expiry no longer permits that write.

type PrimaryWriteFenceStats

type PrimaryWriteFenceStats = database.PrimaryWriteFenceStats

PrimaryWriteFenceStats is a fixed-cardinality view of the optional external primary-write guard. Configured means a guard was supplied at open; Enforced is false while a read-only follower applies validated source history. Checks and Rejected count only actual primary write admissions. No lease, epoch, endpoint, database ID, or controller detail is exposed.

type QueryDelta

type QueryDelta = database.QueryDelta

QueryDelta transforms exactly FromToken into Token. Operations are ordered: removals first, followed by reverse-order add/move anchors and document changes. Applying them in slice order is deterministic.

type QueryDeltaOperation

type QueryDeltaOperation = database.QueryDeltaOperation

QueryDeltaOperation mutates an ordered query result. A zero BeforeID means the end of the result; database document IDs are never zero.

type QueryDeltaOperationKind

type QueryDeltaOperationKind = database.QueryDeltaOperationKind

type QueryDeltaSubscription

type QueryDeltaSubscription = database.QueryDeltaSubscription

QueryDeltaSubscription returns one safe initial snapshot and then ordered deltas. It is the preferred core stream for transports and reactive clients; QuerySubscription remains the full-snapshot compatibility adapter.

type QueryLimits

type QueryLimits = database.QueryLimits

type QueryOptions

type QueryOptions = database.QueryOptions

type QueryReplaySource

type QueryReplaySource = database.QueryReplaySource

QueryReplaySource atomically reconstructs a query at afterToken and tails later ordered revisions. Initial.Token must equal afterToken. Implementations return ErrHistoryLost when retention can no longer satisfy that contract.

type QueryReplaySubscription

type QueryReplaySubscription = database.QueryReplaySubscription

type QuerySnapshot

type QuerySnapshot = database.QuerySnapshot

func ApplyQueryDelta

func ApplyQueryDelta(snapshot QuerySnapshot, delta QueryDelta) (QuerySnapshot, error)

ApplyQueryDelta strictly validates and applies an ordered delta without mutating the input snapshot.

type QuerySpec

type QuerySpec = database.QuerySpec

func CompileQuery

func CompileQuery(filter Filter, options QueryOptions) (QuerySpec, error)

func DecodeQuerySpecJSON

func DecodeQuerySpecJSON(data []byte, limits QueryLimits) (QuerySpec, error)

type QueryStats

type QueryStats = database.QueryStats

type QuerySubscription

type QuerySubscription = database.QuerySubscription

type RealtimeStats

type RealtimeStats = database.RealtimeStats

type ReclaimOptions

type ReclaimOptions = database.ReclaimOptions

ReclaimOptions controls explicit page reclamation. Online scans a duplicate read handle without holding the storage writer lock and installs its result only if the Meta generation is unchanged. MaxAttempts bounds complete graph rescans after concurrent commits; zero selects three attempts.

type ReclaimResult

type ReclaimResult = database.ReclaimResult

type ReclamationStats

type ReclamationStats = database.ReclamationStats

type RecoveryMode

type RecoveryMode = database.RecoveryMode

RecoveryMode controls whether Open may perform only the bounded recovery actions described by RecoveryReport. Zero selects the normal automatic mode.

type RecoveryReport

type RecoveryReport = database.RecoveryReport

RecoveryReport is an immutable, non-sensitive receipt for decisions made while opening a database. It reports only actions that were completed before Open returned successfully; corruption and unsupported formats still fail Open instead of being described as recovered.

type ReplicationFrame

type ReplicationFrame = database.ReplicationFrame

ReplicationFrame is a transport-neutral protocol envelope. A transport must authenticate both peers (for example with mTLS) before it accepts frames; DatabaseID binds every frame to one durable source identity.

func UnmarshalReplicationFrame

func UnmarshalReplicationFrame(data []byte, limits ReplicationFrameLimits) (ReplicationFrame, error)

UnmarshalReplicationFrame rejects unknown fields, duplicate JSON keys, malformed base64, invalid typed documents and non-canonical identities before a receiver reaches the follower state machine.

type ReplicationFrameLimits

type ReplicationFrameLimits = database.ReplicationFrameLimits

ReplicationFrameLimits bounds one already-decompressed protocol frame. The default accommodates the configured 64 MiB canonical transaction limit plus JSON/base64 overhead, while still rejecting unbounded peer allocation.

type ReplicationSourceLease

type ReplicationSourceLease = database.ReplicationSourceLease

ReplicationSourceLease gives one authenticated source-side replica identity exclusive process-local ownership of its durable consumer. A lease prevents duplicate concurrent transports from racing one checkpoint; it does not establish distributed primary authority or replace follower-promotion fencing.

type ReplicationSourceSession

type ReplicationSourceSession = database.ReplicationSourceSession

ReplicationSourceSession is the primary-side state machine for one authenticated peer. It deliberately permits one unacknowledged batch at a time: this is both bounded flow control and the proof that a durable ACK can never skip an unseen source token.

func NewReplicationSourceSession

func NewReplicationSourceSession(db *DB, subscription *DurableDatabaseChangeSubscription, limits ReplicationFrameLimits) (*ReplicationSourceSession, error)

NewReplicationSourceSession binds an existing named durable database feed to one source identity. The caller owns peer authentication and must close the session when that authenticated connection ends.

type ResourceLimits

type ResourceLimits = database.ResourceLimits

ResourceLimits bounds work admitted by writes, index maintenance, and query execution. Zero values select production defaults; limits cannot be disabled accidentally. Byte limits use the canonical typed binary representation, independent of Go heap layout, JSON spelling, storage generation, or transport compression.

type ResourceStats

type ResourceStats = database.ResourceStats

type RollbackAnchor

type RollbackAnchor = database.RollbackAnchor

RollbackAnchor is trusted state retained outside the database device. A server must never accept the same identity below either an acknowledged logical commit sequence or physical maintenance generation after restart. The coordinates are independently monotonic: one group may advance several logical sequences while publishing a single physical generation.

type RollbackAnchorStatusProvider

type RollbackAnchorStatusProvider = database.RollbackAnchorStatusProvider

RollbackAnchorStatusProvider is an optional lock-free observability contract for RollbackAnchorStore implementations.

type RollbackAnchorStore

type RollbackAnchorStore = database.RollbackAnchorStore

RollbackAnchorStore durably loads and atomically advances one database's monotonic anchor. Advance must not return until the anchor is persistent and must reject identity changes or regression of either monotonic coordinate. Implementations must be safe for concurrent callers and honor cancellation. An Advance error does not prove that state was unchanged: persistence may have completed before a response, deadline or cancellation was observed.

func NewFileRollbackAnchorStore

func NewFileRollbackAnchorStore(path string) (RollbackAnchorStore, error)

NewFileRollbackAnchorStore returns a fail-closed, atomically replaced anchor file. The parent directory must already exist. For rollback protection, that directory must be backed by storage trusted independently from the database.

type RollbackAnchorStoreStatus

type RollbackAnchorStoreStatus = database.RollbackAnchorStoreStatus

RollbackAnchorStoreStatus is a bounded, identity-free process-session view of an anchor backend. Counters are diagnostic and never participate in recovery.

type RollbackProtection

type RollbackProtection = database.RollbackProtection

RollbackProtection configures fail-closed database identity and sequence checks. AnchorStore should live on an independently trusted device or remote quorum; placing it beside the database cannot detect whole-device rollback. InitializeAnchor explicitly trusts the database currently at Path when the store is empty and should only be used during provisioning or audited restore.

type SortField

type SortField = database.SortField

type StorageFormat

type StorageFormat = database.StorageFormat

StorageFormat identifies the sole supported on-disk engine. Unknown denotes a missing or zero-length path, not an unrecognized non-empty file.

func DetectStorageFormat

func DetectStorageFormat(path string) (StorageFormat, error)

DetectStorageFormat performs only enough inspection to distinguish a new path from the current database format. Old database files deliberately fail closed: this build contains no legacy reader or automatic migration path.

type StorageFormatInfo

type StorageFormatInfo = database.StorageFormatInfo

StorageFormatInfo is a read-only negotiation view, not a full graph audit. ReaderCompatible says this binary understands the reported revision and all required feature bits; callers must still Open the database before use.

func InspectStorageFormat

func InspectStorageFormat(path string) (StorageFormatInfo, error)

InspectStorageFormat validates current Meta checksums and reports its newest readable envelope without opening the database for mutation.

type StorageLimits

type StorageLimits = database.StorageLimits

StorageLimits bounds the physical single-file high-water mark. Zero selects DefaultMaxFileBytes. The value must be a 16 KiB page multiple.

type StorageStats

type StorageStats = database.StorageStats

StorageStats describes the selected physical backend. Session counters reset on reopen; physical state and cache counters come from the backend itself.

type Update

type Update = database.Update

type UpdateResult

type UpdateResult = database.UpdateResult

type Value

type Value = database.Value

Value is a closed tagged value. Its representation is private so callers cannot construct a tag/payload mismatch.

func Array

func Array(v ...Value) Value

func Binary

func Binary(v []byte) Value

func Bool

func Bool(v bool) Value

func Float

func Float(v float64) Value

func ID

func ID(v DocumentID) Value

func Int

func Int(v int64) Value

func Null

func Null() Value

func Object

func Object(v Document) Value

func String

func String(v string) Value

func Time

func Time(v time.Time) Value

Time stores millisecond precision, matching JavaScript Date and the wire contract. Precision is normalized at construction rather than silently lost during transport.

func UnmarshalWireValue

func UnmarshalWireValue(data []byte, limits QueryLimits) (Value, error)

UnmarshalWireValue decodes one closed, typed wire value using the same depth/item/byte limits as query operands. It is suitable for data-only protocol arguments such as RPC; it never evaluates source or callbacks.

func ValueOf

func ValueOf(x any) (Value, error)

type VerificationReport

type VerificationReport = database.VerificationReport

VerificationReport is a schema-versioned receipt for a full, read-only protected-page graph and published-index semantic audit. ReclaimablePages is informational; verification never installs a free pool or publishes maintenance metadata.

func VerifyFile

func VerifyFile(ctx context.Context, path string) (VerificationReport, error)

VerifyFile performs an offline, non-mutating audit of an existing file. It takes a non-blocking shared advisory lock, so an active writer fails with ErrDatabaseLocked. It never creates, truncates, repairs, reclaims, or advances the database. Meta inspection alone is cheaper; this method walks every page protected by both valid Meta roots, recomputes published and provable shadow Secondary keys from canonical Primary documents in both directions, and hashes the file. Legacy caught-up builds lacking an applied CatalogRoot remain readable but report IndexBuildContentsVerified=false.

type WriteTransaction

type WriteTransaction = database.WriteTransaction

WriteTransaction is a short-lived snapshot write view. It provides point operations with optimistic serializable commit validation. Values returned from it are isolated clones.

A transaction is active only during its handler callback. Handlers must not retain it or call normal DB/Collection methods from inside the callback.

type WriteTransactionStats

type WriteTransactionStats = database.WriteTransactionStats

WriteTransactionStats describes public optimistic point transactions. Every started callback reaches exactly one terminal counter. These aggregates do not contain collection, document, actor, or callback identifiers.

Directories

Path Synopsis
Package admin provides optional, bounded observability consumers for Meldbase.
Package admin provides optional, bounded observability consumers for Meldbase.
cmd
meld command
integrations
anchorhttp
Package anchorhttp provides an authenticated HTTPS quorum implementation of meldbase.RollbackAnchorStore.
Package anchorhttp provides an authenticated HTTPS quorum implementation of meldbase.RollbackAnchorStore.
authorityhttp
Package authorityhttp exposes a narrow, mTLS-authenticated primary-lease Authority control endpoint.
Package authorityhttp exposes a narrow, mTLS-authenticated primary-lease Authority control endpoint.
leasehttp
Package leasehttp exposes one authenticated primarylease.LeaseStore member over strict HTTPS/mTLS JSON.
Package leasehttp exposes one authenticated primarylease.LeaseStore member over strict HTTPS/mTLS JSON.
otel
Package meldotel exports Meldbase's bounded admin snapshots through the stable OpenTelemetry Metrics API.
Package meldotel exports Meldbase's bounded admin snapshots through the stable OpenTelemetry Metrics API.
primarylease
Package primarylease supplies a locally verifiable, short-lived primary write lease for Meldbase deployments.
Package primarylease supplies a locally verifiable, short-lived primary write lease for Meldbase deployments.
replicationauth
Package replicationauth provides shared identity primitives for trusted server-to-server replication transports.
Package replicationauth provides shared identity primitives for trusted server-to-server replication transports.
replicationhttp
Package replicationhttp transports a verified bootstrap over HTTPS.
Package replicationhttp transports a verified bootstrap over HTTPS.
replicationws
Package replicationws adapts Meldbase's trusted-server replication protocol to WebSocket.
Package replicationws adapts Meldbase's trusted-server replication protocol to WebSocket.
internal
database
Package database implements the Meldbase database API behind the module-root public package.
Package database implements the Meldbase database API behind the module-root public package.
policyrecord
Package policyrecord defines the durable private representation of server query-policy generations.
Package policyrecord defines the durable private representation of server query-policy generations.
qualification
Package qualification contains operational release-evidence runners.
Package qualification contains operational release-evidence runners.
storage
Package storage implements the current Meldbase copy-on-write page format.
Package storage implements the current Meldbase copy-on-write page format.
systemrecord
Package systemrecord defines the private bridge between the root database and higher-level built-in services.
Package systemrecord defines the private bridge between the root database and higher-level built-in services.
Package server exposes Meldbase's authenticated HTTP, WebSocket realtime and data-only RPC transport.
Package server exposes Meldbase's authenticated HTTP, WebSocket realtime and data-only RPC transport.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL