mqlite

package module
v0.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 23, 2026 License: MIT Imports: 20 Imported by: 0

README

mqlite

mqlite

CI Go Reference Go Report Card codecov Release Go

A small, SQLite/Turso-backed online message queue with Azure Service Bus–style semantics — Peek-Lock, retries, DLQ, scheduling, dedup, GroupID ordering, topics — in a single pure-Go binary (no CGO).

Embed it like goqite, or serve it like a broker — the same engine. Start in-process (with same-DB transactional enqueue), and upgrade to a network broker with one line when you outgrow it.

Goal

mqlite aims to be the smallest reliable queue you can drop into a system — and to stay friendly to both humans and AI agents:

  • Lightweight & flexible. One pure-Go binary over a single SQLite file (or a remote Turso database) — no broker cluster, no ZooKeeper, no sidecar required. Embed it in your process or run it as a broker; move the storage from a :memory: test to a replicated Turso DB without changing a line of app code.
  • Reliable under concurrency. At-least-once delivery with fencing tokens, single-broker crash recovery, idempotent send/receive/settle, and O(log n) claims on a deep backlog — built to take high enqueue/dequeue throughput without losing or double-delivering messages.
  • Simple, unambiguous interface. Every operation is one plain HTTP POST with a JSON body — curl-able, trivially scriptable, and easy for an LLM or agent to drive — with exactly one settlement verb per outcome and no aliased calls.
  • Service Bus flavor, not a clone. Peek-Lock, GroupID sessions, DLQ, scheduling and dedup will feel familiar if you know Azure Service Bus — but the API is its own, shaped to be idiomatic Go and unambiguous rather than bug-for-bug compatible.

It targets most everyday queueing needs — background jobs, transactional outbox/event delivery, topic fan-out to subscriptions, and rate-limited pipelines — rather than competing with Kafka-scale streaming.

Why mqlite

  • One file, one binary. Local SQLite (modernc.org/sqlite, pure Go) or remote Turso/libSQL — the same SQL and semantics, selected by a connection string.
  • Service Bus semantics, honestly at-least-once. Peek-Lock + four-way settle (Complete/Abandon/Reject/Defer), visibility timeout with fencing tokens, delivery_count → DLQ, Renew, scheduled/deferred messages.
  • Approximate order by default, strict order opt-in. Plain queues are competing-consumer. Pick a queue-level ordering mode at create time: standard (default — per-GroupID FIFO with cross-group parallelism), group_fifo (same, but a GroupID is required on every message), or strict_fifo (single global head-of-line FIFO across the whole queue). GroupID is an ordering key (= SQS MessageGroupId / ASB SessionId) — not a consumer group; competing consumers just Receive the same queue.
  • curl-able contract. Every RPC is a plain HTTP POST to /mqlite.v1.<Service>/<Method> with a JSON body; the Go SDK is sugar on top.

Layout

Path What
engine/ The queue core (Store + Service Bus semantics). Transport-agnostic.
server/ Connect-style JSON-over-HTTP broker + Bearer-token auth.
wire/ The JSON contract shared by server and client (one source of truth).
. (package mqlite) Native Go SDK: remote Client, Embedded engine, Receiver.
cmd/mqlite/ Single-binary CLI: serve, send, receive, peek, …

Quickstart

Full copy-runnable programs (embedded, transactional outbox, remote consumer, curl): docs/examples.md.

1. Embedded (in-process — no broker, no HTTP, no second process)

mqlite's primary form is a library you embed directly in your Go process, exactly like goqite or using database/sql against SQLite. OpenEmbedded gives you the full queue — Send/Receive/Peek-Lock/DLQ/scheduling/topics — calling the storage engine in-process. There is no broker to run, no network hop, no JSON serialization, no extra daemon: your app and the queue are one binary backed by one SQLite (or Turso) database. You only start an HTTP server if you choose to (see §2) — the embedded path never opens a socket.

Single process, single writer. Embedded mode is one process owning one database. OpenEmbedded takes an exclusive lock on the DB file, so a second process — or a second OpenEmbedded on the same file — fails fast with ErrDBLocked instead of racing it (two writers would corrupt crash recovery and claim ordering). Sharing one file DB across processes is not supported: when you need multiple processes or hosts, run the broker (§2) and connect over HTTP — that single broker is the one writer. (:memory: is private per handle, and a remote Turso DB is serialized server-side, so neither takes the lock.)

ctx := context.Background()

// The whole MQ, in your process. file: local SQLite, or libsql://… for Turso.
eng, err := mqlite.OpenEmbedded(ctx, "file:./mq.db")
if err != nil { log.Fatal(err) }
defer eng.Close()

eng.CreateQueue(ctx, "orders", mqlite.QueueConfig{})

// produce
eng.SendOne(ctx, "orders", mqlite.OutMessage{Body: []byte("hello"), GroupID: "order-42"})

// consume (Peek-Lock): handle, then settle. at-least-once → handler must be idempotent.
msgs, _ := eng.Receive(ctx, "orders", mqlite.RecvOpts{Wait: 5 * time.Second})
for _, m := range msgs {
    if err := handle(m.Body); err != nil {
        _ = m.Abandon(ctx)   // release the lock → redelivered (or DLQ past max)
        continue
    }
    _ = m.Complete(ctx)      // remove it (idempotent under retries)
}

// or hands-off: a Receiver auto-settles (nil→Complete, err→Abandon) — still in-process.
eng.Receiver("orders", mqlite.WithConcurrency(4)).
    Run(ctx, func(ctx context.Context, m *mqlite.Message) error { return handle(m.Body) })

⭐ Transactional outbox — the embedded superpower. Because the queue lives in the same SQLite database as your application tables, you can enqueue a message in the same transaction as your business write. No dual-write, no outbox poller, no lost events: either both commit or neither does.

eng.Tx(ctx, func(tx *engine.EngineTx) error {
    // tx.Context(), not the outer ctx: see the note below.
    if _, err := tx.SQL().ExecContext(tx.Context(), `INSERT INTO orders_tbl(id) VALUES (1)`); err != nil {
        return err // your business write failed — roll back the message too
    }
    _, err := tx.SendOne("orders", engine.OutMessage{Body: []byte("order-created")})
    return err // commit both, or roll back both
})

Keep the callback transaction-bound. On a remote (Turso/libSQL) store the callback may run more than once: a transaction that fails on a retryable connection/busy error is replayed from the start. The SQL rolls back, so your data stays correct — but anything that is not part of the transaction (an HTTP call, a charge, a counter in memory) will have happened twice. Local file and :memory: stores never retry, so the callback runs at most once there; an already-cancelled caller never enters it.

Local cancellation: use tx.Context() for business SQL to protect SQLite from interruption, and check the original ctx.Err() between statements. Raw tx.SQL() calls bypass mqlite's per-statement guards and can still execute after cancellation. The callback must return before the writer is released; cancellation observed before commit rolls back all its writes, even if it returned nil. See the cancellation contract.

Outgrow a single process? The same engine upgrades to a network broker with one call — eng.Serve(ctx, ":6754") — and remote clients speak the same semantics over HTTP (§2–§4). Embedded and broker are not two products; they are one engine.

2. Serve a broker
export MQLITE_DB="file:./mq.db"           # or libsql://<db>.turso.io
export MQLITE_DB_AUTH_TOKEN="<jwt>"        # only for remote Turso
export MQLITE_TOKENS="mqk_dev"             # administrator Bearer tokens
mqlite serve --addr :6754
3. Talk to it with curl
TOKEN=mqk_dev
# discovery: hit the root with no auth — a JSON card (name, version, status, endpoints)
curl http://127.0.0.1:6754/

# create the queue first — sending to a queue that doesn't exist is a 404
curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  --data '{"name":"orders","config":{}}' \
  http://127.0.0.1:6754/mqlite.v1.AdminService/CreateQueue

# send (body is base64)
curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  --data "{\"queue\":\"orders\",\"messages\":[{\"body\":\"$(printf hello | base64)\"}]}" \
  http://127.0.0.1:6754/mqlite.v1.QueueService/Send

# receive (long-poll 5s) → returns lock_token
curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  --data '{"queue":"orders","max_messages":1,"wait_time_ms":5000}' \
  http://127.0.0.1:6754/mqlite.v1.QueueService/Receive

Send response. /Send returns {"seq_numbers":[…]} — one per message, in order. A single message that hits a dedup conflict (same message_id, different body) is rejected with 409; in a multi-message batch the conflicting slot comes back as 0 (skipped, not enqueued) while the rest still commit.

Full endpoint, request/response, and error reference: docs/api-reference.md.

3b. Admin console

The broker bakes a web console into the binary and serves it at /ui (e.g. http://127.0.0.1:6754/ui) — no separate process or Node runtime. Paste your broker URL

  • token and you can list queues with live counts, browse messages, send/receive/settle, edit subscription filters, and redrive a DLQ. It's on by default; set MQLITE_UI=off to run headless. (The same console is also published standalone for hosting elsewhere.)

Starting with v0.3.2, a configured monitor token opens only the read-only overview and metrics views; administration still requires manage.

4. Or the Go SDK (remote)
cli, _ := mqlite.Open(ctx, "http://127.0.0.1:6754", mqlite.WithToken("mqk_dev"))
seq, _ := cli.SendOne(ctx, "orders", mqlite.OutMessage{Body: []byte("hi")})

// hands-off consumer: nil -> Complete, error -> Abandon, auto lock renewal
cli.Receiver("orders", mqlite.WithAutoRenew(), mqlite.WithConcurrency(4)).
    Run(ctx, func(ctx context.Context, m *mqlite.Message) error {
        return process(m.Body) // MUST be idempotent (at-least-once)
    })
5. CLI

The mqlite binary is a first-party client for the common broker operations — the same commands work embedded (against MQLITE_DB) or against a running broker (--endpoint):

mqlite create-queue orders --lock 30s --max-delivery 5 --dedup 10m --ordering strict_fifo
mqlite send orders "hello" --message-id m1 --group order-42 --reply-to replies
mqlite schedule orders "later" --at 30m          # future delivery; cancel <queue> <seq>
mqlite receive orders --wait 5s                  # auto-Complete; --no-ack prints lock tokens
mqlite complete orders 42 <lock-token>           # settle a --no-ack message (also abandon/reject/defer/renew)
mqlite peek orders --state dead_lettered
mqlite status                                    # backend / ping / counts
mqlite observe --output json                     # unified observation; monitor or manage
mqlite list-subscriptions ; mqlite test-filter 'subject == "x"'
mqlite metrics orders --output json              # machine-readable: the same keys as the HTTP API
mqlite redrive orders --max 100                  # DLQ → active
mqlite purge-dlq orders --older-than 24h         # delete dead-lettered (unbounded needs --all)

The data and admin commands take the global flags --endpoint <url> / --token <t> (override the environment) and --output text|json — serve, version and help don't, and vacuum is local-only. Under --output json a message is the HTTP API's message object, key for key. See the CLI reference for the full set.

Connection is read from --endpoint/--token, or from the environment:

Env Meaning
MQLITE_DB DB DSN: file:./mq.db, :memory:, or libsql://<db>.turso.io (embedded/serve)
MQLITE_DB_AUTH_TOKEN auth token for a remote libSQL/Turso DSN
MQLITE_ENDPOINT + MQLITE_TOKEN client mode: talk to a running broker (wins if set)
MQLITE_TOKENS comma-separated administrator Bearer tokens for serve
MQLITE_MONITOR_TOKENS optional, distinct read-only credentials for observe and /metrics; administrator auth must remain enabled
MQLITE_METRICS enable authenticated Prometheus /metrics on the API port (on/true/1; default off); requires administrator auth
MQLITE_SYNC durability level: NORMAL (default) / FULL / OFF / EXTRA (an unknown value is rejected at startup)
MQLITE_DLQ_MAX_AGE · MQLITE_DLQ_MAX_COUNT · MQLITE_DLQ_MAX_BYTES broker DLQ retention (defaults 14d / 1,000,000 per queue, drop-oldest; byte cap off by default; MQLITE_DLQ_RETENTION=off disables)

The DB connection string is only ever read from the environment — it is never compiled into the binary. .env.example lists the variables mqlite reads; the binary does not auto-load a dotenv file, so export them into the process yourself — set -a && . ./.env.local && set +a for a shell, or an Environment=/ compose environment: block for systemd/Docker.

Starting with v0.3.1, create additional keys without restarting the broker using mqlite key create --name producer --permissions send. Keys can grant send, listen, both, or manage (which also grants key issuance and revocation). Configured administrator tokens remain supported. See key commands and the complete access-key guide.

Starting with v0.3.2, mqlite observe and configured MQLITE_MONITOR_TOKENS provide read-only broker observation. See the monitoring guide and Prometheus/Grafana quickstart.

Prometheus scraping is opt-in: set MQLITE_METRICS=on or run mqlite serve --metrics. The API then serves authenticated /metrics, and discovery includes "metrics":"/metrics". With metrics disabled, the route returns 404 and the field is omitted. Give the collector a monitor token. To keep the route off the public Internet, use a private broker or block /metrics in your public ingress; Bearer authentication alone does not make the route private. Prometheus and Grafana can run locally or in an existing platform; no second MQLite VM is required. See the monitoring guide.

6. MCP (drive it from an AI agent)

mqlite-mcp is a Model Context Protocol server that exposes the broker as agent tools (create_queue, send, receive, complete, …) — a thin, dependency-free stdio forwarder, in keeping with mqlite's "friendly to AI agents" goal. Point it at a broker with MQLITE_ENDPOINT + MQLITE_TOKEN. See docs/mcp.md.

Documentation

Use it — pick your interface:

CLI reference every command + flags; works embedded or against a broker
HTTP API reference endpoints, request/response, error codes
MCP server drive mqlite from an AI agent
Examples copy-runnable Go (embedded · outbox · remote) + curl

Operate it: Deployment (Docker/Fly/systemd/Turso) · Production operations (backup/restore/upgrade) · Observability (Prometheus/Grafana) · Retention · Turso

Understand it: Concepts & filters (model + expr filter language) · Internals (how it's built: schema, single-writer, fencing, recovery) · Conformance / TCK · Benchmarks, sizing & tuning · Dependencies

Durability

mqlite stores everything in SQLite/libSQL with WAL journaling. State is durable across a normal process crash: on restart, in-flight (locked) messages are reclaimed and redelivered (at-least-once), and settled work stays settled.

The one knob is SQLite's synchronous level (database-wide — SQLite has no per-queue setting):

Level Guarantee Cost
NORMAL (default) Durable across an app/process crash. A sudden OS or power loss can drop the last few commits not yet checkpointed to the main DB file. Fast — fsync per WAL checkpoint, not per commit.
FULL Every commit is fsync'd before it returns — survives OS/power loss. Slower under high enqueue throughput.

Set it with the embedded option mqlite.WithSynchronous("FULL"), or MQLITE_SYNC=FULL for the serve/CLI path. Remote Turso DSNs ignore this — durability there is the server's responsibility.

What "no message loss" means here: mqlite is honestly at-least-once, not exactly-once. Under NORMAL, a power cut can lose a handful of just-enqueued messages that hadn't reached disk yet; choose FULL if that window is unacceptable. Once a message is durably written it is delivered at least once and never silently dropped.

Retention

Completed messages are deleted on settle and TTL'd messages expire, so a healthy queue's size tracks its in-flight backlog. The one sink that can grow without bound is the dead-letter queue — so serve bounds it by default: a background pass drops dead letters oldest-first once they are older than 14 days or beyond 1,000,000 per queue (MQLITE_DLQ_MAX_AGE / MQLITE_DLQ_MAX_COUNT; an optional per-queue byte cap MQLITE_DLQ_MAX_BYTES is off by default; MQLITE_DLQ_RETENTION=off disables all). Only the DLQ is ever touched — undelivered and in-flight work is never auto-deleted. The embedded library leaves this off unless you opt in with mqlite.WithDLQRetention(...). Freed pages are reused, and a background janitor hands them back to the OS via incremental_vacuum, so the file grows to the peak backlog and then shrinks back gradually as the queue drains (a full VACUUM rewrite stays manual-only, via mqlite vacuum). See docs/retention.md.

Docker

# --pull forces the latest golang:1.27-alpine3.24 base (Go stdlib security patches) —
# use it for release builds so a cached old base layer can't ship known CVEs.
docker build --platform linux/amd64 --pull -t mqlite:dev .
docker run --platform linux/amd64 -p 6754:6754 -e MQLITE_TOKENS=mqk_dev mqlite:dev
# remote Turso instead of the local volume:
docker run --platform linux/amd64 -p 6754:6754 \
  -e MQLITE_DB=libsql://<db>.turso.io -e MQLITE_DB_AUTH_TOKEN=<jwt> \
  -e MQLITE_TOKENS=mqk_dev mqlite:dev

Footprint: ~11 MB static (CGO-free) binary, ~19 MB idle RSS, ~0.4 KB per message on disk — fits Fly's smallest 256 MB machine with room to spare. Measured numbers and Fly.io machine/volume sizing: docs/benchmark.md.

Full copy-runnable recipes — Docker/GHCR, Fly.io (minimal-cost, scale-to-zero), systemd, and Turso — with auth and verification: docs/deployment.md. Wiring /metrics into Prometheus + Grafana (scrape config, PromQL, alerts): docs/observability.md.

Tests

go test ./...                 # hermetic unit + invariant (TCK-style) tests
go test -bench=. ./engine     # embedded throughput benchmarks (msg/s)
go test ./engine -run TestMessageIntegrity      # no-loss + content-hash sweep (default 1k)
MQLITE_INTEGRITY_N=500000 go test ./engine -run TestMessageIntegrity  # large sweep
# live remote round-trip against your own Turso DB:
MQLITE_TEST_DB=libsql://<db>.turso.io MQLITE_TEST_DB_AUTH_TOKEN=<jwt> \
  go test ./engine -run TestTursoIntegration -v
# crash-injection: hard-kill a worker and check recovery (separate Linux CI job; tag-gated):
make crash                    # = go test -race -tags crash_injection -count=1 ./test/crash/

make crash runs with the race detector; go test -race ./... skips this tagged suite. It checks outbox atomicity, survival of producer-acknowledged commits, and orphaned-lock recovery to active or DLQ at the delivery limit. It tests process death, not power loss.

A contiguous 1..N sequence is sent with random bodies (each hashed into a property) and consumed concurrently with redelivery stress; the test asserts every value arrives exactly once (no loss) and every body still matches its hash (no corruption). Local + cloud (Fly) throughput/memory/disk methodology and numbers: docs/benchmark.md. The large sweep also runs weekly in CI.

Status

This source tree targets v0.3.2: default broker port 6754, schema token 5, runtime-managed access keys and unified observability. See CHANGELOG.md for release and upgrade notes. Use matching binaries from tagged releases or pin the image to ghcr.io/mqlitehq/mqlite:0.3.2; see deployment for configuration.

Upgrading from v0.3.1 preserves message and managed-key data. Back up first; process counters reset at restart. A rollback to v0.3.1 also needs compatible scraper/rule configuration because Observe and configured monitor credentials are new in v0.3.2. See upgrade and rollback.

Upgrading from v0.3.0 preserves existing queue data. Back up first and retain a configured administrator token if you need to roll back; v0.3.0 cannot authenticate managed keys.

Upgrading from v0.2.0 requires a new database. There is no in-place schema migration. Account for retained messages in every state, preserve the old binary and a consistent database backup, and follow the upgrade and rollback procedure. Delivery remains at-least-once; consumers must be idempotent.

Hermetic unit + invariant (TCK-style) tests run in CI on every push, and live Turso/libSQL round-trips run in the nightly workflow.

License

MIT © The MQLite Authors

Documentation

Overview

Package mqlite is the native Go SDK for mqlite (design §17.1). The same method set drives two modes:

Open         — a remote client talking to an mqlite broker over HTTP.
OpenEmbedded — the queue engine in-process (like goqite), with same-DB
               transactional enqueue and a one-line upgrade to a broker.

Settlement methods hang off *Message so the lock token never leaks into user code, and receive comes in two tiers: low-level Receive (you settle) and high-level Receiver.Run (handler returns nil -> Complete, error -> Abandon).

Example (Embedded)

Example_embedded runs the whole queue in-process — no broker, no HTTP — against a local SQLite file, and shows the single-process / single-writer guarantee: a second opener of the same DB is rejected with ErrDBLocked (MQLITE-6 / MQLITE-15).

package main

import (
	"context"
	"errors"
	"fmt"
	"log"
	"os"
	"path/filepath"

	"github.com/mqlitehq/mqlite"
)

func main() {
	ctx := context.Background()
	dir, err := os.MkdirTemp("", "mqlite-example-*")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()
	dsn := "file:" + filepath.Join(dir, "mq.db")

	eng, err := mqlite.OpenEmbedded(ctx, dsn)
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = eng.Close() }()

	if err := eng.CreateQueue(ctx, "orders", mqlite.QueueConfig{}); err != nil {
		log.Fatal(err)
	}
	if _, err := eng.SendOne(ctx, "orders", mqlite.OutMessage{Body: []byte("order-42")}); err != nil {
		log.Fatal(err)
	}

	msgs, err := eng.Receive(ctx, "orders", mqlite.RecvOpts{})
	if err != nil {
		log.Fatal(err)
	}
	for _, m := range msgs {
		fmt.Printf("received: %s\n", m.Body)
		_ = m.Complete(ctx) // at-least-once; the handler must be idempotent
	}

	// Embedded mode is single-writer: a second process — or a second OpenEmbedded on
	// the same file — is rejected rather than racing the first.
	if _, err := mqlite.OpenEmbedded(ctx, dsn); errors.Is(err, mqlite.ErrDBLocked) {
		fmt.Println("second open rejected: single writer")
	}

}
Output:
received: order-42
second open rejected: single writer

Index

Examples

Constants

View Source
const (
	OrderStandard   = engine.OrderStandard
	OrderGroupFIFO  = engine.OrderGroupFIFO
	OrderStrictFIFO = engine.OrderStrictFIFO
)
View Source
const (
	Active       = engine.StateActive
	Locked       = engine.StateLocked
	Deferred     = engine.StateDeferred
	Scheduled    = engine.StateScheduled
	DeadLettered = engine.StateDeadLettered
)
View Source
const MaxRenewBatch = engine.MaxRenewBatch

MaxRenewBatch is the most messages one RenewBatch call may renew. Renewal must fit in a single statement so that every Ok is decided against ONE clock reading rather than drifting across a chunked write — see engine.MaxRenewBatch. Ok does not promise the lease is still live when you read it; SettleResult.LockedUntil is the deadline that was committed, and that is what to pace yourself by. Receive hands out at most 256 messages, so a consumer never meets this cap by accident; a caller holding more renews in several calls.

View Source
const TokenPrefix = authkey.Prefix

TokenPrefix is the conventional prefix for a broker auth token, so a token is recognizable at a glance (and greppable in logs/config). It is a convention, not a requirement — any non-empty string in MQLITE_TOKENS is accepted as a token.

Variables

View Source
var (
	ErrLockLost         = engine.ErrLockLost
	ErrUnauthenticated  = engine.ErrUnauthenticated
	ErrPermissionDenied = engine.ErrPermissionDenied
	ErrKeyConflict      = engine.ErrKeyConflict
	ErrOutcomeUnknown   = engine.ErrOutcomeUnknown
	ErrInvalidArgument  = engine.ErrInvalidArgument
	ErrNotFound         = engine.ErrNotFound
	ErrUnsupported      = engine.ErrUnsupported
	ErrQueueNotFound    = engine.ErrQueueNotFound
	ErrDedupConflict    = engine.ErrDedupConflict
	ErrMessageTooLarge  = engine.ErrMessageTooLarge
	ErrNameConflict     = engine.ErrNameConflict
	ErrGroupRequired    = engine.ErrGroupRequired
	// ErrDBLocked is returned by OpenEmbedded when the local file DB is already open
	// in another process: embedded mode is single-process, single-writer (MQLITE-6).
	ErrDBLocked = engine.ErrDBLocked
	// ErrClosed is returned by an operation that starts during or after Close: once Close begins,
	// new work is refused immediately rather than racing a store being torn down (round-8).
	ErrClosed = engine.ErrClosed
)

Re-exported sentinel errors so callers can use errors.Is on either mode.

Functions

func EndpointIdentity added in v0.3.0

func EndpointIdentity(dsn string) (string, error)

EndpointIdentity returns the identity of the broker an endpoint string addresses: it is literally the base URL the client would dial, i.e. the string Client.post concatenates the RPC route onto. Two endpoints reach the same broker exactly when this value is equal.

Use it to answer "are these two endpoints the same broker?" — the question that decides whether an ambient token may be reused for an endpoint the caller overrode.

It deliberately does NOT try to canonicalize a URL. Deciding which parts of a URL are "insignificant" is a losing game — the path, percent-escaping, a query, a fragment, an IPv6 zone id and host case all change (or fail to change) where a reverse proxy actually routes the request, and every component you normalize away is a chance to call two different brokers the same and hand one of them the other's credential. Comparing the dialed string itself cannot make that mistake: if the bytes we send differ, we treat it as a different broker and withhold the token. The failure mode is a warning and an explicit --token, never a leak.

What it does normalize is only what parseDSN already normalizes to build the dial target — an insignificant trailing slash (the reported bug: `http://h:6754/` cost the caller their token), the product default port for the custom schemes, and a credential embedded in an `mqlite://token@host` DSN (which is not part of the target).

func GenerateKeyID added in v0.3.1

func GenerateKeyID() string

GenerateKeyID generates a public access key ID (32 lowercase hex characters). Retain the ID before calling CreateKey so an uncertain result can be reconciled. Like GenerateToken, it panics if the secure random source fails.

func GenerateToken added in v0.1.1

func GenerateToken() string

GenerateToken mints a fresh broker auth token: the "mqk_" prefix plus 256 bits of crypto/rand, hex-encoded ("mqk_" + 64 lowercase hex characters). Use it to seed MQLITE_TOKENS / WithTokens, or let `mqlite serve` call it automatically when no token is configured (secure by default). It panics if the secure random source fails.

Types

type AbandonOpts

type AbandonOpts struct {
	Delay time.Duration // re-hide for this long before redelivery (backoff)
}

AbandonOpts configures a Message.Abandon call.

type AccessKey added in v0.3.1

type AccessKey = wire.AccessKey

AccessKey is public key metadata. It never contains a token or its digest. Permissions are send, listen, or manage (which includes send and listen). All timestamps are UTC epoch milliseconds; zero means no expiry or revocation.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a remote mqlite client (Connect-style JSON over HTTP).

func Open

func Open(ctx context.Context, dsn string, opts ...Option) (*Client, error)

Open builds a client from a connection string: mqlite://<token>@host:port?tls=true

func (*Client) Cancel

func (c *Client) Cancel(ctx context.Context, queue string, seq int64) error

Cancel deletes a not-yet-activated scheduled message.

func (*Client) Close

func (c *Client) Close() error

Close releases client resources (no-op for HTTP).

func (*Client) CompleteBatch added in v0.1.1

func (c *Client) CompleteBatch(ctx context.Context, queue string, msgs ...*Message) ([]SettleResult, error)

CompleteBatch completes many messages in one request (one settlement RPC instead of N), returning a per-message result so a caller can see which sequence numbers settled.

func (*Client) CreateKey added in v0.3.1

func (c *Client) CreateKey(ctx context.Context, opts CreateKeyOptions) (CreateKeyResult, error)

CreateKey creates a managed credential. The caller must have manage permission. It sends one request and never retries a lost response: retain opts.ID to list and revoke a possibly-created key before issuing a replacement.

func (*Client) CreateQueue

func (c *Client) CreateQueue(ctx context.Context, name string, cfg QueueConfig) error

CreateQueue creates or updates a queue.

func (*Client) ListKeys added in v0.3.1

func (c *Client) ListKeys(ctx context.Context, afterID string, limit int) (KeyPage, error)

ListKeys lists managed credentials without secrets in ascending ID order. Limit defaults to 100 and may not exceed 1000. Static administrator tokens are not part of this list. Use ListKeysWithOptions to list newest-created keys first.

func (*Client) ListKeysWithOptions added in v0.3.1

func (c *Client) ListKeysWithOptions(ctx context.Context, opts ListKeysOptions) (KeyPage, error)

ListKeysWithOptions lists managed metadata in the requested sort order. Limit defaults to 100 and may not exceed 1000. No token or digest is returned.

func (*Client) ListQueues

func (c *Client) ListQueues(ctx context.Context) ([]QueueInfo, error)

ListQueues lists queues/subscriptions.

func (*Client) ListSubscriptions added in v0.3.0

func (c *Client) ListSubscriptions(ctx context.Context) ([]SubscriptionInfo, error)

ListSubscriptions returns every subscription with its topic and filter expression.

func (*Client) Message added in v0.3.0

func (c *Client) Message(queue string, seq int64, lockToken string) *Message

CompleteBatch completes many received messages in one round-trip — the fix for the drain N+1 (settling a received batch one HTTP call at a time). Messages should share one queue; a stale lock comes back as Ok=false rather than failing the batch. Message rehydrates a settleable handle for a message you already received but did not settle (e.g. `mqlite receive --no-ack`, then settle in a later invocation). Pass the queue, its SequenceNumber, and the LockToken() from that receive; the returned *Message settles through this Client (Complete/Abandon/Reject/Defer/Renew).

func (*Client) Observe added in v0.3.2

func (c *Client) Observe(ctx context.Context) (Observation, error)

Observe returns canonical queue, storage, maintenance and HTTP measurements.

func (*Client) Peek

func (c *Client) Peek(ctx context.Context, queue string, opts ...PeekOpts) ([]*PeekedMessage, error)

Peek browses without locking.

func (*Client) Purge

func (c *Client) Purge(ctx context.Context, queue string, opts ...PurgeOpts) (int, error)

Purge permanently deletes dead-lettered messages (PurgeOpts scopes it; no opts purges the whole DLQ). Returns count deleted.

func (*Client) Receive

func (c *Client) Receive(ctx context.Context, queue string, opts ...RecvOpts) ([]*Message, error)

Receive claims up to N messages (Peek-Lock by default), with optional long-poll. RecvOpts.Pick fetches previously-deferred messages by seq instead of claiming.

func (*Client) Receiver

func (c *Client) Receiver(queue string, opts ...ReceiverOption) *Receiver

Receiver returns a stateful receive loop bound to this client.

func (*Client) Redrive

func (c *Client) Redrive(ctx context.Context, dlqQueue string, opts ...RedriveOpts) (int, error)

Redrive moves dead-lettered messages back to active.

func (*Client) RenewBatch added in v0.3.0

func (c *Client) RenewBatch(ctx context.Context, queue string, msgs ...*Message) ([]SettleResult, error)

RenewBatch extends the lock lease of many messages in ONE request, returning a per-message result. Renewing a batch with N separate Renew calls costs N round trips — on a slow link that outlasts the very lease it is trying to save (a 64-message batch on a 50ms link needs 3.2s of renewals against a 2s lease), so the locks expire mid-renewal.

SettleResult.LockedUntil is the deadline the broker committed, and it — not Ok — is the fact to pace yourself by. Ok says the lease was live when the broker's STATEMENT finished; the response still has to travel back to you, so on a very short lease it can lapse on the way. Compare LockedUntil against your own clock.

func (*Client) RevokeKey added in v0.3.1

func (c *Client) RevokeKey(ctx context.Context, id string) error

RevokeKey permanently revokes a managed key. Repeating a successful revocation succeeds; an unknown ID returns ErrNotFound. Static tokens cannot be revoked.

func (*Client) Send

func (c *Client) Send(ctx context.Context, queue string, msgs ...OutMessage) ([]int64, error)

Send enqueues one or many messages in one request/transaction (use SendOne for a single seq return or At() scheduling).

func (*Client) SendOne

func (c *Client) SendOne(ctx context.Context, queue string, m OutMessage, opts ...SendOpts) (int64, error)

SendOne enqueues one message and returns its seq. A dedup conflict (same id, different body) surfaces as ErrDedupConflict. SendOpts.At schedules delayed delivery.

func (*Client) Stats

func (c *Client) Stats(ctx context.Context, queue string) (Metrics, error)

Stats returns counters for a queue.

func (*Client) Status added in v0.3.0

func (c *Client) Status(ctx context.Context) (StatusInfo, error)

Status returns a desensitized snapshot of the broker's backend (never a token or DSN).

func (*Client) Subscribe

func (c *Client) Subscribe(ctx context.Context, topic, name string, f *Filter) error

Subscribe registers a subscription under a topic with an optional filter.

func (*Client) TestFilter added in v0.3.0

func (c *Client) TestFilter(ctx context.Context, expr string, sample *OutMessage, enqueuedAtMs, visibleAtMs int64) (FilterTestResult, error)

TestFilter dry-runs a subscription filter expression: it compiles expr and, when sample is non-nil, evaluates it against that message (nothing is enqueued). Read-only.

type CreateKeyOptions added in v0.3.1

type CreateKeyOptions = wire.CreateKeyRequest

CreateKeyOptions describes a new broker-wide credential. ID is required: generate it with GenerateKeyID before calling CreateKey and retain it if the response is lost. A repeated ID returns ErrKeyConflict; an existing secret is never returned.

type CreateKeyResult added in v0.3.1

type CreateKeyResult = wire.CreateKeyResponse

CreateKeyResult contains the credential's metadata and its one-time secret. Store Token securely: neither ListKeys nor a repeated CreateKey can recover it.

type Embedded

type Embedded struct {
	// contains filtered or unexported fields
}

Embedded runs the queue engine in-process (like goqite), exposing the same method set as the remote Client plus same-DB transactional enqueue (Tx) and a one-line upgrade to a network broker (Serve). See design §4.5.

func OpenEmbedded

func OpenEmbedded(ctx context.Context, dbDSN string, opts ...EmbeddedOption) (*Embedded, error)

OpenEmbedded opens the engine on a DB DSN: file:./mq.db | :memory: | libsql://host

func (*Embedded) Cancel

func (e *Embedded) Cancel(ctx context.Context, queue string, seq int64) error

Cancel deletes a not-yet-activated scheduled message.

func (*Embedded) Close

func (e *Embedded) Close() error

Close shuts the engine down safely: it stops the background loops, closes the admission gate (any DB operation that starts after shutdown begins fails fast with ErrClosed), wakes empty long-poll Receives with ErrClosed, waits for already-admitted writes and transactions to finish, and only then closes the DB and releases the file lock. Returns the teardown error, if any.

Because it waits for in-flight work, Close MUST NOT be called from inside an operation it would wait for — in particular, not from within a Tx callback (see Tx). Doing so deadlocks.

func (*Embedded) Compact added in v0.1.1

func (e *Embedded) Compact(ctx context.Context, full bool) error

Compact reclaims free DB pages to the OS: `PRAGMA incremental_vacuum` (no global lock; new local DBs default to auto_vacuum=INCREMENTAL), or a full `VACUUM` when full=true (rewrites the file, global lock — maintenance only). Local-only.

func (*Embedded) CompleteBatch added in v0.1.1

func (e *Embedded) CompleteBatch(ctx context.Context, queue string, msgs ...*Message) ([]SettleResult, error)

CompleteBatch completes many messages in one transaction, returning a per-message result.

func (*Embedded) CreateKey added in v0.3.1

func (e *Embedded) CreateKey(ctx context.Context, opts CreateKeyOptions) (CreateKeyResult, error)

CreateKey creates a managed credential in the embedded database. Embedded mode is trusted local administration and does not require a bearer token.

func (*Embedded) CreateQueue

func (e *Embedded) CreateQueue(ctx context.Context, name string, cfg QueueConfig) error

func (*Embedded) Engine

func (e *Embedded) Engine() *engine.Engine

Engine exposes the underlying engine (advanced/embedded use).

func (*Embedded) ListKeys added in v0.3.1

func (e *Embedded) ListKeys(ctx context.Context, afterID string, limit int) (KeyPage, error)

ListKeys lists managed metadata in the embedded database in ascending ID order.

func (*Embedded) ListKeysWithOptions added in v0.3.1

func (e *Embedded) ListKeysWithOptions(ctx context.Context, opts ListKeysOptions) (KeyPage, error)

ListKeysWithOptions lists managed metadata in the requested sort order in the embedded database, with the same limits and cursor rules as the HTTP client.

func (*Embedded) ListQueues

func (e *Embedded) ListQueues(ctx context.Context) ([]QueueInfo, error)

func (*Embedded) ListSubscriptions added in v0.3.0

func (e *Embedded) ListSubscriptions(ctx context.Context) ([]SubscriptionInfo, error)

ListSubscriptions returns every subscription with its topic and filter expression.

func (*Embedded) Message added in v0.3.0

func (e *Embedded) Message(queue string, seq int64, lockToken string) *Message

CompleteBatch completes many received messages in one transaction (mirrors the remote Client method; for the in-process engine there is no round-trip to save, but it keeps the API symmetric and settles atomically). Message rehydrates a settleable handle for a message you already received but did not settle (the embedded twin of Client.Message). Pass the queue, its SequenceNumber, and the LockToken() from that receive; the returned *Message settles through this engine.

func (*Embedded) Observe added in v0.3.2

func (e *Embedded) Observe(ctx context.Context) (Observation, error)

Observe samples the same canonical observation as the broker. HTTP measurements are explicitly not applicable to an in-process engine.

func (*Embedded) Peek

func (e *Embedded) Peek(ctx context.Context, queue string, opts ...PeekOpts) ([]*PeekedMessage, error)

func (*Embedded) Purge

func (e *Embedded) Purge(ctx context.Context, queue string, opts ...PurgeOpts) (int, error)

Purge permanently deletes dead-lettered messages (PurgeOpts scopes it; no opts purges the whole DLQ). Returns count deleted.

func (*Embedded) Receive

func (e *Embedded) Receive(ctx context.Context, queue string, opts ...RecvOpts) ([]*Message, error)

func (*Embedded) Receiver

func (e *Embedded) Receiver(queue string, opts ...ReceiverOption) *Receiver

Receiver returns a stateful receive loop bound to the embedded engine.

func (*Embedded) Redrive

func (e *Embedded) Redrive(ctx context.Context, dlqQueue string, opts ...RedriveOpts) (int, error)

func (*Embedded) RenewBatch added in v0.3.0

func (e *Embedded) RenewBatch(ctx context.Context, queue string, msgs ...*Message) ([]SettleResult, error)

RenewBatch extends the lock lease of many messages in ONE STATEMENT — deliberately not a transaction (the embedded twin of Client.RenewBatch), returning a per-message result.

SettleResult.LockedUntil is the deadline actually committed, and it — not Ok — is the fact to pace yourself by. Ok says the lease was live when the STATEMENT finished, which is already in the past by the time you read it: nothing is encoded or sent in embedded mode, but your own handling still takes time and a very short lease can lapse inside it. Compare LockedUntil against your own clock rather than trusting Ok to still hold.

func (*Embedded) RevokeKey added in v0.3.1

func (e *Embedded) RevokeKey(ctx context.Context, id string) error

RevokeKey permanently revokes a managed credential in the embedded database.

func (*Embedded) Send

func (e *Embedded) Send(ctx context.Context, queue string, msgs ...OutMessage) ([]int64, error)

Send enqueues one or many messages in one transaction.

func (*Embedded) SendOne

func (e *Embedded) SendOne(ctx context.Context, queue string, m OutMessage, opts ...SendOpts) (int64, error)

SendOne enqueues one message. A dedup conflict surfaces as ErrDedupConflict. SendOpts.At schedules delayed delivery.

func (*Embedded) Serve

func (e *Embedded) Serve(ctx context.Context, addr string, opts ...ServeOption) error

Serve exposes this engine as an HTTP broker until ctx is canceled.

func (*Embedded) Stats

func (e *Embedded) Stats(ctx context.Context, queue string) (Metrics, error)

func (*Embedded) Status added in v0.3.0

func (e *Embedded) Status(ctx context.Context) (StatusInfo, error)

Status returns a desensitized snapshot of the local backend (the embedded twin of Client.Status). Version is empty and UptimeMs/Auth are zero-valued — an in-process engine has no broker build stamp, no process uptime and no auth; queue/subscription counts are live.

func (*Embedded) Subscribe

func (e *Embedded) Subscribe(ctx context.Context, topic, name string, f *Filter) error

func (*Embedded) TestFilter added in v0.3.0

func (e *Embedded) TestFilter(ctx context.Context, expr string, sample *OutMessage, enqueuedAtMs, visibleAtMs int64) (FilterTestResult, error)

TestFilter dry-runs a subscription filter expression against an optional sample message (nothing is enqueued) — the embedded twin of Client.TestFilter.

func (*Embedded) Tx

func (e *Embedded) Tx(ctx context.Context, fn func(*engine.EngineTx) error) error

Tx runs business writes and enqueues in one transaction (§4.5, embedded-only): business success ⇔ message enqueued, with no dual-write window.

On a local store, use tx.Context() for raw tx.SQL() statements and check the original ctx between them. Raw SQL bypasses mqlite's per-statement cancellation guards and can still execute after ctx is cancelled. The callback must return before the writer is released; cancellation observed before commit rolls back the whole transaction, even if the callback returns nil. An already-cancelled caller never enters the callback. See engine.EngineTx.SQL and engine.EngineTx.Context.

ON A REMOTE (Turso/libSQL) STORE, fn MAY RUN MORE THAN ONCE. A transaction that fails on a retryable connection or busy error is replayed from the start — the database work rolls back, so the DATA stays correct, but anything else fn did will have happened twice. Keep fn to transaction-bound work: SQL through the provided handle, and nothing else. A charge, an email, a counter in memory, an HTTP call — none of those roll back with the transaction, and a replay will repeat them (round-4 §5.2). Local file and :memory: stores never retry, so fn runs at most once there.

fn MUST NOT call Close: Close waits for the open transaction to finish before tearing the engine down, and fn cannot finish until it returns, so it would deadlock. Close from outside the callback.

type EmbeddedOption

type EmbeddedOption func(*embeddedConfig)

EmbeddedOption configures OpenEmbedded.

func WithClock

func WithClock(now func() int64) EmbeddedOption

WithClock injects a test clock returning epoch ms.

func WithDBAuthToken

func WithDBAuthToken(tok string) EmbeddedOption

WithDBAuthToken supplies the auth token for a remote libSQL/Turso DSN. The token is passed in by the caller (typically from an env var) — never compiled in.

func WithDLQRetention added in v0.1.1

func WithDLQRetention(maxAge time.Duration, maxCount int, maxBytes int64) EmbeddedOption

WithDLQRetention bounds the dead-letter queue so a long-running broker can't grow without limit (MQLITE-21). A background pass drops dead letters oldest-first when they are older than maxAge, beyond maxCount per queue, or over maxBytes of body per queue; pass 0 to leave a dimension unbounded. ONLY the DLQ is affected — undelivered and in-flight messages are never deleted. Off by default for the embedded library; the `mqlite serve` broker enables a sane default (see cmd/mqlite).

func WithLogger

func WithLogger(lg *slog.Logger) EmbeddedOption

WithLogger routes background-loop failures to lg (nil -> slog.Default()).

func WithMaxMessageBytes

func WithMaxMessageBytes(n int64) EmbeddedOption

WithMaxMessageBytes caps message body size (0 -> 1 MiB default).

func WithSynchronous

func WithSynchronous(mode string) EmbeddedOption

WithSynchronous sets the local SQLite PRAGMA synchronous level — the durability vs throughput knob (MQLITE-7). "NORMAL" (default) is fast and durable across a process crash, but a sudden OS/power loss can drop the last few not-yet- checkpointed commits; "FULL" fsyncs every commit (safer, slower). Database-wide (SQLite has no per-queue sync); ignored for remote Turso DSNs.

func WithoutBackground

func WithoutBackground() EmbeddedOption

WithoutBackground disables the maintenance loops (tests drive them manually).

type Filter

type Filter = engine.Filter

Filter is a subscription filter (equality-AND + subject prefix).

type FilterTestResult added in v0.3.0

type FilterTestResult = engine.FilterTestResult

FilterTestResult mirrors engine.FilterTestResult — the dry-run of a filter expression.

type KeyPage added in v0.3.1

type KeyPage = wire.ListKeysResponse

KeyPage is a page of metadata, including revoked and expired keys. Pass NextAfterID to the next list request with the same sort order; an empty value marks the final page.

type ListKeysOptions added in v0.3.1

type ListKeysOptions = wire.ListKeysRequest

ListKeysOptions selects a metadata page. Sort is "id_asc" (the default) or "created_desc" (creation time descending, then ID descending for ties). Keep the same Sort when passing a page's NextAfterID as AfterID. In created_desc order, AfterID must identify an existing key, including revoked or expired keys.

type Message

type Message struct {
	SequenceNumber int64
	Body           []byte
	MessageID      string
	GroupID        string
	CorrelationID  string
	ReplyTo        string
	Subject        string
	ContentType    string
	Properties     map[string]string
	DeliveryCount  int
	EnqueuedAt     time.Time
	LockedUntil    time.Time
	// contains filtered or unexported fields
}

Message is a delivered message handle. The lock token is held internally so settlement through the Complete/Abandon/… methods can't be misrouted (§17.1); LockToken() exposes it for callers that must settle out of band (e.g. the CLI across processes).

func (*Message) Abandon

func (m *Message) Abandon(ctx context.Context, opts ...AbandonOpts) error

Abandon releases the lock for immediate redelivery (optionally after a delay).

func (*Message) Complete

func (m *Message) Complete(ctx context.Context) error

Complete removes a successfully-processed message.

func (*Message) Defer

func (m *Message) Defer(ctx context.Context) error

Defer sets the message aside, to be retrieved later by SequenceNumber.

func (*Message) LockToken added in v0.3.0

func (m *Message) LockToken() string

LockToken returns the Peek-Lock fencing token for this message. Normal SDK use never needs it — settle through the Complete/Abandon/… methods and the token stays internal. It is exposed for callers that must settle out of band across process boundaries (the `mqlite` CLI: receive with --no-ack, print the token, settle it in a later invocation via Client.Message / Embedded.Message). The same token is already in every Receive HTTP response, so this exposes nothing the wire protocol doesn't.

func (*Message) Reject

func (m *Message) Reject(ctx context.Context, opts ...RejectOpts) error

Reject moves the message to the dead-letter state (optionally with a reason).

func (*Message) Renew

func (m *Message) Renew(ctx context.Context) error

Renew extends the lock lease (for long-running handlers).

type Metrics

type Metrics = engine.Metrics

Metrics mirrors engine.Metrics with the same fields.

type Observation added in v0.3.2

type Observation = wire.ObserveResponse

Observation is the canonical broker and engine observation. Check Collection.State before interpreting queue gauges; process counters reset when the engine restarts.

type Option

type Option func(*config)

Option configures Open.

func WithHTTPClient

func WithHTTPClient(h *http.Client) Option

WithHTTPClient supplies a custom *http.Client (e.g. for TLS config).

func WithToken

func WithToken(tok string) Option

WithToken overrides the token from the connection string.

type OrderingMode

type OrderingMode = engine.OrderingMode

OrderingMode mirrors engine.OrderingMode for queue-level delivery ordering.

type OutMessage

type OutMessage struct {
	Body      []byte
	MessageID string // dedup/idempotency key; empty -> body SHA-256 when dedup on
	// GroupID is an ordering/partition key (= SQS MessageGroupId, ASB SessionId):
	// same GroupID = strict in-order (FIFO per group); empty = own group (max
	// parallelism). NOT a consumer group — competing consumers just Receive the
	// same queue; peek-lock gives each message to exactly one.
	GroupID       string
	CorrelationID string
	ReplyTo       string // = ASB ReplyTo; opaque address the consumer should reply to
	Subject       string // = ASB Label
	ContentType   string
	Properties    map[string]string // custom KV (headers)
	TTL           time.Duration     // 0 -> queue default
}

OutMessage is a message to send. Body is opaque; broker never parses it.

type PeekOpts

type PeekOpts struct {
	From  int64 // start browsing at this seq
	State State // filter by state
	Max   int   // cap results
}

PeekOpts configures a Peek call.

type PeekedMessage

type PeekedMessage struct {
	SequenceNumber int64
	State          State
	Body           []byte
	MessageID      string
	GroupID        string
	CorrelationID  string
	ReplyTo        string
	Subject        string
	ContentType    string
	Properties     map[string]string
	DeliveryCount  int
	EnqueuedAt     time.Time
	VisibleAt      time.Time
	ExpiresAt      time.Time // TTL expiry (zero = no TTL)
	LockedUntil    time.Time
	// Dead-letter metadata, set for messages in the dead_lettered state (triage).
	DeadLetterReason      string
	DeadLetterDescription string
}

PeekedMessage is a read-only browse result.

type PurgeOpts

type PurgeOpts struct {
	Max       int           // cap how many messages are deleted
	OlderThan time.Duration // only delete messages older than this
}

PurgeOpts configures a Purge call.

type QueueConfig

type QueueConfig struct {
	LockDuration       time.Duration
	MaxDeliveryCount   int
	DefaultTTL         time.Duration
	DeadLetterOnExpire *bool
	DedupWindow        time.Duration
	Ordering           OrderingMode // "" -> standard
	// Per-queue DLQ retention overrides (MQLITE-29). For each: 0 -> inherit the
	// broker default; >0 -> this queue's drop-oldest bound; a negative value ->
	// explicitly unbounded (opt out of the default).
	DLQMaxAge   time.Duration
	DLQMaxCount int
	DLQMaxBytes int64
}

QueueConfig configures a queue (durations as time.Duration).

type QueueInfo

type QueueInfo = engine.QueueInfo

QueueInfo mirrors engine.QueueInfo.

type Receiver

type Receiver struct {
	// contains filtered or unexported fields
}

Receiver is a stateful receive loop. handler returning nil auto-Completes; returning error auto-Abandons (§17.1).

func (*Receiver) Close

func (r *Receiver) Close() error

Close releases receiver resources (no-op).

func (*Receiver) Run

func (r *Receiver) Run(ctx context.Context, handler func(context.Context, *Message) error) error

Run pulls and processes messages until ctx is canceled or a permanent error occurs. It returns ctx.Err() on cancellation, or the first permanent receive/settle/renew error.

type ReceiverOption

type ReceiverOption func(*receiverConfig)

ReceiverOption configures a Receiver.

func WithAutoRenew

func WithAutoRenew() ReceiverOption

WithAutoRenew keeps locks alive with a background heartbeat for long handlers.

func WithConcurrency

func WithConcurrency(n int) ReceiverOption

WithConcurrency sets how many messages are processed in parallel.

func WithErrorHandler added in v0.3.0

func WithErrorHandler(fn func(error)) ReceiverOption

WithErrorHandler registers a callback for errors the receive loop would otherwise swallow: a transient receive error being retried, or a per-message settle/renew failure (e.g. a lost lock). It is advisory — for logging or metrics — is called serially (the Receiver guards it, so a plain slice-append or counter is safe), and must not block. A permanent error (bad token, denied permission, missing queue) is passed here first and then also returned from Run (MQLITE-77).

func WithPrefetch

func WithPrefetch(n int) ReceiverOption

WithPrefetch caps how many messages are claimed per receive. It is clamped to the concurrency: the Receiver never claims more than it has idle workers for, so a claimed message always has a worker (and, with WithAutoRenew, a renew heartbeat) ready — it can't sit locked behind a busy worker and let its lock expire (MQLITE-76). Values above the concurrency therefore have no extra effect; use it only to claim in smaller chunks.

type RecvOpts

type RecvOpts struct {
	Max        int           // most messages to claim (0 -> 1)
	Wait       time.Duration // long-poll wait (0 -> don't wait); capped at 20s
	AtMostOnce bool          // receive-and-delete (no lock, no settle)
	Attempt    string        // idempotent-receive key; a retry replays the same batch
	Pick       []int64       // fetch these deferred seqs by seq instead of claiming
}

RecvOpts configures a Receive call.

type RedriveOpts

type RedriveOpts struct {
	To        string        // target queue (empty -> back to source)
	Max       int           // cap how many messages move
	OlderThan time.Duration // only move messages older than this
	Rate      int           // throughput limit per second
}

RedriveOpts configures a Redrive call.

type RejectOpts

type RejectOpts struct {
	Reason string // dead-letter reason
	Detail string // dead-letter description
}

RejectOpts configures a Message.Reject call.

type SendOpts

type SendOpts struct {
	At time.Time // schedule delivery for t (zero -> immediate)
}

SendOpts configures a SendOne call.

type ServeOption

type ServeOption func(*serveConfig)

ServeOption configures Serve.

func WithCORS added in v0.2.0

func WithCORS(origin string) ServeOption

WithCORS sets the Access-Control-Allow-Origin the broker sends, enabling browser apps served from another origin (the standalone console) to call it. "" (the default) leaves CORS off; "*" allows any origin — safe here because every RPC needs a Bearer token and the API uses no cookies, so a cross-origin page gains nothing without the token.

func WithMetrics added in v0.3.2

func WithMetrics(on bool) ServeOption

WithMetrics enables authenticated GET /metrics on the broker's API listener. Off by default. Enabling it requires administrator authentication and adds the /metrics path to discovery. Scrapers can use a read-only monitor token.

func WithMonitorTokens added in v0.3.2

func WithMonitorTokens(tokens ...string) ServeOption

WithMonitorTokens adds configured read-only credentials for Observe and /metrics. Serve rejects blank credentials, overlap with administrators, or disabled auth. These credentials can authenticate during a storage outage without a DB lookup.

func WithReady added in v0.3.0

func WithReady(fn func()) ServeOption

WithReady registers a callback Serve invokes once the listener is bound, before it starts accepting — so a caller announces "ready" only after the bind actually succeeded, never before a bind that then fails (MQLITE-88).

func WithRequestLog added in v0.2.0

func WithRequestLog(lg *slog.Logger) ServeOption

WithRequestLog enables a per-request access log on the broker, emitted through lg (one line per RPC: method, status, latency, at a level chosen by status). nil leaves request logging off. Separate from WithLogger (which routes the engine's background-loop failures) so a caller can colour the broker's live traffic without touching engine logs.

func WithTokenCSV

func WithTokenCSV(csv string) ServeOption

WithTokenCSV sets accepted Bearer tokens from a comma-separated string (env-friendly).

func WithTokens

func WithTokens(tokens ...string) ServeOption

WithTokens sets the accepted Bearer tokens for the broker.

func WithUI added in v0.2.0

func WithUI(on bool) ServeOption

WithUI serves the embedded admin console (the built mqlite-web SPA) at /ui. Off by default for the embedded library; the `mqlite serve` binary turns it on (MQLITE_UI).

func WithVersion added in v0.1.1

func WithVersion(v string) ServeOption

WithVersion sets the version string the broker reports on its open "/" discovery endpoint (typically the CLI/build version).

type SettleResult added in v0.1.1

type SettleResult struct {
	SequenceNumber int64
	Ok             bool
	// LockedUntil is the lease deadline a RenewBatch actually committed (zero for other
	// operations). A renewing caller needs it to know when to renew next — without it, it can
	// only guess, and would either hammer the broker or let the lock it just extended lapse.
	//
	// It is REPORTED, never written back onto the *Message: the caller may be reading those
	// messages concurrently (the CLI renders them while a background renewer runs), and mutating
	// a shared time.Time from another goroutine is a data race.
	LockedUntil time.Time
}

SettleResult is the per-message outcome of a batch settle (Ok=false = the lock was lost/expired for that message — not a fatal error for the batch).

type State

type State = engine.State

State mirrors engine.State for Peek filtering.

type StatusInfo added in v0.3.0

type StatusInfo struct {
	Version       string `json:"version,omitempty"`
	Backend       string `json:"backend"`
	Remote        bool   `json:"remote"`
	Location      string `json:"location"`
	SchemaVersion string `json:"schema_version"`
	PingMs        int64  `json:"ping_ms"`
	DBSizeBytes   int64  `json:"db_size_bytes"` // local on-disk footprint; 0 for memory/remote
	Queues        int    `json:"queues"`
	Subscriptions int    `json:"subscriptions"`
	// Broker-only facts: zero/false in embedded mode, which has no process uptime and no auth.
	UptimeMs int64 `json:"uptime_ms"`
	Auth     bool  `json:"auth"`
}

StatusInfo is a desensitized runtime snapshot of the backend (Client.Status / Embedded.Status): never a connection string or token — Location is a local path or a masked remote host. Version/Queues/Subscriptions come from the broker on the remote path; embedded fills what it can locally (Version is empty embedded).

type SubscriptionInfo added in v0.3.0

type SubscriptionInfo = engine.SubscriptionInfo

SubscriptionInfo mirrors engine.SubscriptionInfo (topic, name, filter expression).

Directories

Path Synopsis
cmd
mqlite command
Command mqlite is the single-binary CLI: run a broker, or produce/consume/ administer queues against a local DB (embedded) or a remote broker (client).
Command mqlite is the single-binary CLI: run a broker, or produce/consume/ administer queues against a local DB (embedded) or a remote broker (client).
mqlite-mcp command
Command mqlite-mcp is a Model Context Protocol (MCP) server that exposes the mqlite broker as agent tools.
Command mqlite-mcp is a Model Context Protocol (MCP) server that exposes the mqlite broker as agent tools.
internal
authkey
Package authkey defines the common format of generated broker credentials.
Package authkey defines the common format of generated broker credentials.
defaults
Package defaults holds mqlite's built-in network defaults in one place so the broker CLI (cmd/mqlite) and the MCP server (cmd/mqlite-mcp) can never drift apart.
Package defaults holds mqlite's built-in network defaults in one place so the broker CLI (cmd/mqlite) and the MCP server (cmd/mqlite-mcp) can never drift apart.
version
Package version holds the single version constant every mqlite binary reports (mqlite CLI/broker and mqlite-mcp).
Package version holds the single version constant every mqlite binary reports (mqlite CLI/broker and mqlite-mcp).
Package server exposes an mqlite Engine as a Connect-style JSON-over-HTTP broker (design §7).
Package server exposes an mqlite Engine as a Connect-style JSON-over-HTTP broker (design §7).
test
bench command
Command mqlite-bench stress-tests the local SQLite engine (embedded, no HTTP) across several high-frequency request patterns, and attributes per-scenario CPU and disk I/O via /proc/self (Linux/Docker).
Command mqlite-bench stress-tests the local SQLite engine (embedded, no HTTP) across several high-frequency request patterns, and attributes per-scenario CPU and disk I/O via /proc/self (Linux/Docker).
bench/httpload command
Command httpload is an HTTP load generator + integrity checker for a *running mqlite broker*.
Command httpload is an HTTP load generator + integrity checker for a *running mqlite broker*.
crash
Package crash holds the crash-injection layer (see crash_test.go).
Package crash holds the crash-injection layer (see crash_test.go).
production/restore command
Command restore is the SDK fixture worker for run.py.
Command restore is the SDK fixture worker for run.py.
production/soak command
Command soak runs bounded mixed workloads against a dedicated candidate broker.
Command soak runs bounded mixed workloads against a dedicated candidate broker.
sdkcheck command
Command sdkcheck is the SDK end-to-end suite (the SDK path).
Command sdkcheck is the SDK end-to-end suite (the SDK path).
Package wire defines the JSON-over-HTTP contract shared by the mqlite broker (server) and the Go SDK client.
Package wire defines the JSON-over-HTTP contract shared by the mqlite broker (server) and the Go SDK client.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL