arbiter

README

arbiter-core

Public Go types and runtime modules shared by Sentio Arbiter control-plane nodes and storage nodes.

The module path is github.com/sentioxyz/arbiter-core; its root Go package is named arbiter so domain types retain concise names such as arbiter.StatementEnvelope.

Packages

Package	Purpose
root	Canonical statement, result-claim, promotion, cleanup, consensus-update, and node types.
authority	Domain-separated promotion, cleanup, consensus-update and snapshot-query-abort signing and validation.
wire	The canonical Go ↔ arbiter-proto conversion and Raft command encoding.
dataplane	Leader-aware Arbiter clients, subscriptions, manifests, and payload stores.
snode	Durable storage-node intake, crash convergence, promotion, and cleanup runtime.
verifier	Replay and byte-side verification runtime shared by storage-node hosts.
conformance	Field and enum compatibility gates against arbiter-proto and Housegate replay types.

The private github.com/sentioxyz/arbiter repository owns the Raft FSM, orchestrator, servers, anchoring backend, and operator binaries. It consumes this module; this module never imports the private control-plane repository.

Build and test

bazel build //...
bazel test //...

Bazel 9.1.0 with Bzlmod is the supported build and dependency contract. arbiter-core consumes Housegate as a first-class Bazel module, so downstream repositories must declare both modules and pin their source revisions:

bazel_dep(name = "arbiter_core")
bazel_dep(
    name = "housegate",
    version = "0.10.0",
)

git_override(
    module_name = "arbiter_core",
    commit = "<arbiter-core commit>",
    remote = "https://github.com/sentioxyz/arbiter-core",
)
git_override(
    module_name = "housegate",
    # Resolved Housegate v0.10.0; source is pinned by the commit below.
    commit = "576848692bb87dc891da95559577115d0fcb9773",
    remote = "https://github.com/housegate/housegate",
)

The go.mod file is retained as Gazelle's dependency manifest and for editor metadata. Housegate now uses the canonical github.com/housegate/housegate module path, so standard Go consumers no longer need a downstream replace directive to resolve it.

The source MODULE.bazel intentionally leaves the arbiter-core module version unset. Until arbiter-core is published in a Bazel registry, downstream repositories select its exact source revision through git_override.

Update both the Go and Bzlmod Housegate pins from a release tag or commit SHA:

bash scripts/update-housegate.sh v0.7.1
bash scripts/update-housegate.sh 4dd088f4fe17d7bf13ba2c2e2311d72d0b97cd54

The script resolves the canonical Go version and full commit, updates go.mod, go.sum, MODULE.bazel, this example, and the Bzlmod lockfile.

Storage-integrity protocol-table construction validates every declared column type against the storage-integrity whitelist before emitting any DDL. A declaration outside that profile fails role construction, so an invalid schema cannot create a partial set of protocol tables.

The protocol-table mode is derived from schema_source: network_state and chain create and reconcile the tables, while clickhouse verifies an existing deployment without creating it. There is no separately configurable mode that can disagree with the authoritative schema source.

hg_promote is created and verified together with hg_unsafe and hg_safe. A verify-only node whose protocol tables were never bootstrapped in create mode, or whose hg_promote table has drifted, now fails at startup instead of discovering the problem during the first promotion.

ClickHouse-backed SNode tests are opt-in. The full DDL acceptance uses two ClickHouse 25.8 nodes sharing one Keeper:

docker network create arbiter-core-ch
docker run -d --rm --name arbiter-core-ch-a --hostname arbiter-core-ch-a \
  --network arbiter-core-ch --network-alias arbiter-core-clickhouse-a \
  -p 9000:9000 \
  -e CLICKHOUSE_SKIP_USER_SETUP=1 \
  -v "$PWD/scripts/ci/clickhouse-shared-keeper-server.xml:/etc/clickhouse-server/config.d/keeper.xml:ro" \
  clickhouse/clickhouse-server:25.8
docker run -d --rm --name arbiter-core-ch-b --hostname arbiter-core-ch-b \
  --network arbiter-core-ch \
  -p 9001:9000 \
  -e CLICKHOUSE_SKIP_USER_SETUP=1 \
  -v "$PWD/scripts/ci/clickhouse-shared-keeper-client.xml:/etc/clickhouse-server/config.d/keeper.xml:ro" \
  clickhouse/clickhouse-server:25.8

ARBITER_CH_INTEGRATION=1 \
  ARBITER_CH_KEEPER=1 \
  ARBITER_CH_REPLICA=1 \
  CH_ADDR=127.0.0.1:9000 \
  CH_REPLICA_ADDR=127.0.0.1:9001 \
  bazel test //dataplane/ddl:ddl_test //snode:snode_test //verifier:verifier_test \
    --test_env=ARBITER_CH_INTEGRATION \
    --test_env=ARBITER_CH_KEEPER \
    --test_env=ARBITER_CH_REPLICA \
    --test_env=CH_ADDR \
    --test_env=CH_REPLICA_ADDR \
    --test_timeout=900

Releases

Run the Cut Release workflow from main. It validates the Bazel module and the ClickHouse-backed SNode and verifier paths, then creates an annotated tag and GitHub Release. Versions follow the same UTC calendar scheme as Arbiter:

• the first cut is v0.0.0;
• another cut on the same UTC day increments patch;
• the first cut on a later UTC day increments minor and resets patch to zero.

Tags are the version ledger; no version file is maintained in the repository.

Compatibility

arbiter-proto is the language-neutral wire contract. During the v0.x series, minor releases of this module may evolve its Go interface; protocol field numbering and canonical signing forms remain guarded by conformance tests.

Consensus-parameter updates use the canonical arbiter.ConsensusParamsUpdate and Raft command slot 18. (*authority.Validator).VerifyConsensusParamsUpdate supports deterministic replay without clock checks; use AuthorizeConsensusParamsUpdate only at live API boundaries. Both fail closed for an empty authority allowlist. See the protocol specification for signed preconditions, address normalization, and the all-voters upgrade gate.

Documentation

Overview

Package arbiter defines the frozen P0 interface seams and canonical command/coordinate types of the Sentio Arbiter (design §3.4). It stays dependency-light so every subpackage (fsm, accumulator, orchestrator, raftnode, server, authority) can import it without cycles.

Index

• Constants
• func StatementIDString(clientAccount string, clientSeq uint64, clientNonce string) string
• type AdmissionCode
• type AnchorRef
• type ByteSideScanBody
• type ByteSideScanMsg
  • func (m ByteSideScanMsg) Body() ByteSideScanBody
• type CandidatePart
• type CleanupAck
• type ConsensusParamsUpdate
• type NodeRegistration
• type NodeRole
• type PartRef
• type PartScan
• type PartitionLtHashSum
• type PromoteSafePartition
• type PromotionAck
• type RCRecord
• type SafePartMapping
• type StatementCoord
• type StatementEnvelope
• type StatementID
  • func (id StatementID) Coord() StatementCoord
  • func (id StatementID) Flat() string
• type StatementKind
• type TablePartition
  • func (p TablePartition) MarshalText() ([]byte, error)
  • func (p *TablePartition) UnmarshalText(b []byte) error
• type UnsafeCleanup

Constants

const (
	// DomainL3Header chains L3 block headers: PrevL3Hash =
	// CanonicalDigest(DomainL3Header, header-with-anchor-excluded) (§5).
	DomainL3Header = "arbiter-l3-header-v1"
	// DomainL3Statements commits the sealed block's envelopes (statement_seq
	// order) into L3BlockHeader.StatementsRoot.
	DomainL3Statements = "arbiter-l3-statements-v1"
	// DomainByteSideScan hashes ByteSideScanBody for scan_hash (§7.2).
	DomainByteSideScan = "arbiter-byte-side-scan-v1"
	// SourceSelectSeedPrefix seeds §5.4 deterministic source selection.
	SourceSelectSeedPrefix = "arbiter-source-select-v1:"
	// VerifierSelectSeedPrefix seeds §7.1 deterministic 3-selection.
	VerifierSelectSeedPrefix = "arbiter-verifier-select-v1:"
)

Frozen CanonicalDigest domains and deterministic-selection seed prefixes (P1a design §3). These are consensus parameters in the P0b §8 sense: compile-time constants, no configuration surface, changed only as a new versioned value with an explicit migration.

const ConsensusAdminProtocolVersion uint32 = 1

ConsensusAdminProtocolVersion advertises support for the ConsensusAdmin capability/read/update RPCs and the replicated parameter-update command. Every voter must support this version before operators enable updates.

Functions

func StatementIDString

func StatementIDString(clientAccount string, clientSeq uint64, clientNonce string) string

StatementIDString renders the canonical flat statement_id used by the replay projection (pkg/replay.Statement.StatementID), RCRecord linkage, and _hg_row_id derivation: "<lowercase account>:<decimal seq>:<nonce>". The accumulator's binary leaf encoding is frozen separately (P0b plan); this string form is the cross-component linking id.

Types

type AdmissionCode

type AdmissionCode int32

AdmissionCode mirrors pb.AdmissionCode (numbers pinned by conformance).

const (
	AdmissionCodeUnspecified        AdmissionCode = 0
	AdmissionCodeAccepted           AdmissionCode = 1
	AdmissionCodeDuplicateClientSeq AdmissionCode = 2
	AdmissionCodeSchemaNotAllowed   AdmissionCode = 3
	AdmissionCodeKindNotAdmitted    AdmissionCode = 4
	AdmissionCodeInvalidSignature   AdmissionCode = 5
	AdmissionCodeInvalidProof       AdmissionCode = 6
	AdmissionCodeMalformed          AdmissionCode = 7
	// AdmissionCodeGapBudgetExceeded: the P0b K=64 open-range budget
	// (arbiter-proto v0.2.0 append).
	AdmissionCodeGapBudgetExceeded AdmissionCode = 8
)

type AnchorRef

type AnchorRef struct {
	L3BlockHash   string `json:"l3_block_hash"`
	StateRoot     string `json:"state_root"`
	L2TxRef       string `json:"l2_tx_ref,omitempty"`
	L2BlockNumber uint64 `json:"l2_block_number,omitempty"`
	DARef         string `json:"da_ref,omitempty"`
}

AnchorRef references the L2 anchor of one L3 block (§5.2).

type ByteSideScanBody

type ByteSideScanBody struct {
	ReplicaID string     `json:"replica_id"`
	BlockSeq  uint64     `json:"block_seq"`
	Parts     []PartScan `json:"parts"`
}

ByteSideScanBody is the canonical hash/sign form of a scan: the message minus its own hash and signature. scan_hash = CanonicalDigest( DomainByteSideScan, msg.Body()); signature = ed25519 over the scan_hash string bytes, hex — the ReplayAttestation convention.

type ByteSideScanMsg

type ByteSideScanMsg struct {
	ReplicaID string     `json:"replica_id"`
	BlockSeq  uint64     `json:"block_seq"`
	Parts     []PartScan `json:"parts"`
	ScanHash  string     `json:"scan_hash"`
	Signature string     `json:"signature"`
}

ByteSideScanMsg mirrors pb.ByteSideScanMsg.

func (ByteSideScanMsg) Body

func (m ByteSideScanMsg) Body() ByteSideScanBody

Body returns the canonical hash/sign form.

type CandidatePart

type CandidatePart struct {
	TableID       string `json:"table_id"`
	PartitionID   string `json:"partition_id"`
	PartName      string `json:"part_name,omitempty"`
	PartRowLtHash string `json:"part_row_lthash"`
	PartPhysHash  string `json:"part_phys_hash,omitempty"`
	RowCount      uint64 `json:"row_count,omitempty"`
	Bytes         uint64 `json:"bytes,omitempty"`
}

CandidatePart is one hg_unsafe part the source claims a statement produced; part_row_lthash is its identity everywhere downstream.

type CleanupAck

type CleanupAck struct {
	NodeID       string `json:"node_id"`
	PromotionSeq uint64 `json:"promotion_seq"`
	TableID      string `json:"table_id"`
	PartitionID  string `json:"partition_id"`
}

CleanupAck acknowledges a scheduled unsafe cleanup.

type ConsensusParamsUpdate

type ConsensusParamsUpdate struct {
	NetworkID            string   `json:"network_id"`
	GenesisSnapshotID    string   `json:"genesis_snapshot_id"`
	ExpectedEpoch        uint64   `json:"expected_epoch"`
	PreviousParamsDigest string   `json:"previous_params_digest"`
	AuthorityAddresses   []string `json:"authority_addresses"`
	MaxWriters           uint64   `json:"max_writers"`
	ExpectedPromotionSeq uint64   `json:"expected_promotion_seq"`
	// ArtifactDispositionCapability is the C1 artifact-disposition kill switch
	// as a governed consensus parameter: 0 keeps every tag-28 command refused,
	// 1 enables the lane. It is absent from the canonical form when zero, so
	// every previously signed update keeps its digest; the FSM refuses 1 -> 0.
	ArtifactDispositionCapability uint32 `json:"artifact_disposition_capability,omitempty"`
}

ConsensusParamsUpdate is the canonical signing form of a complete mutable consensus-parameter transition. Identity and compare-and-swap preconditions are signed along with the target authority set and writer limit.

authority.NormalizeConsensusParamsUpdate canonicalizes the address set before hashing. Protobuf serialization is transport only, never the signed form. The FSM additionally checks identity, epoch, digest, promotion sequence, membership capacity and drained authority work against committed state.

type NodeRegistration

type NodeRegistration struct {
	NodeID        string     `json:"node_id"`
	Roles         []NodeRole `json:"roles"`
	Ed25519Pubkey []byte     `json:"ed25519_pubkey"`
	DialAddr      string     `json:"dial_addr,omitempty"`
}

NodeRegistration enters a data-plane node into FSM membership.

type NodeRole

type NodeRole int32

NodeRole mirrors pb.NodeRole.

const (
	NodeRoleUnspecified NodeRole = 0
	NodeRoleVerifier    NodeRole = 1
	NodeRoleSNode       NodeRole = 2
)

type PartRef

type PartRef struct {
	TableID       string `json:"table_id"`
	PartitionID   string `json:"partition_id"`
	PartRowLtHash string `json:"part_row_lthash"`
	PartName      string `json:"part_name,omitempty"`
}

PartRef identifies a verified part by content commitment (design §8.1). JSON tags mirror the arbiter-proto PartRef field names: these structs are the CANONICAL SIGNING FORM — authority JWS payloads hash them through replay.CanonicalDigest, never re-encoded proto bytes (§4.3).

type PartScan

type PartScan struct {
	TableID              string `json:"table_id"`
	PartitionID          string `json:"partition_id"`
	ClaimedPartRowLtHash string `json:"claimed_part_row_lthash"`
	ScannedPartRowLtHash string `json:"scanned_part_row_lthash"`
	LivePartName         string `json:"live_part_name,omitempty"`
}

PartScan is one scanned part's byte-side result (check 3).

type PartitionLtHashSum

type PartitionLtHashSum struct {
	TableID           string `json:"table_id"`
	PartitionID       string `json:"partition_id"`
	NewPartsLtHashSum string `json:"new_parts_lthash_sum"`
}

PartitionLtHashSum is the source's claimed per-partition new-part LtHash sum — check 2's "claimed" side (§7.3).

type PromoteSafePartition

type PromoteSafePartition struct {
	TableID            string    `json:"table_id"`
	PartitionID        string    `json:"partition_id"`
	PromotionSeq       uint64    `json:"promotion_seq"`
	BaseSafeSnapshotID string    `json:"base_safe_snapshot_id"`
	BasePartitionRoot  string    `json:"base_partition_root"`
	CandidateParts     []PartRef `json:"candidate_parts"`
}

PromoteSafePartition is the canonical signing form of the promotion command (design §8.1); wire form is pb.PromoteSafePartition.

type PromotionAck

type PromotionAck struct {
	NodeID                  string            `json:"node_id"`
	PromotionSeq            uint64            `json:"promotion_seq"`
	TableID                 string            `json:"table_id"`
	PartitionID             string            `json:"partition_id"`
	PostPartitionCommitment string            `json:"post_partition_commitment"`
	Parts                   []SafePartMapping `json:"parts"`
	// SafePartitionParts is the complete active inventory after REPLACE,
	// including previously-safe parts whose physical names may change.
	SafePartitionParts []SafePartMapping `json:"safe_partition_parts,omitempty"`
	Applied            bool              `json:"applied"`
	Detail             string            `json:"detail,omitempty"`
}

PromotionAck reports REPLACE PARTITION completion (§8.3).

type RCRecord

type RCRecord struct {
	StatementID          StatementID          `json:"statement_id"`
	SourceNode           string               `json:"source_node"`
	CandidateParts       []CandidatePart      `json:"candidate_parts"`
	SourceClaimRoot      string               `json:"source_claim_root"`
	PartitionNewPartSums []PartitionLtHashSum `json:"partition_new_part_sums"`
}

RCRecord is the source's result claim (late-bindable by statement_id).

type SafePartMapping

type SafePartMapping struct {
	PartRowLtHash string `json:"part_row_lthash"`
	SafePartName  string `json:"safe_part_name"`
	PartPhysHash  string `json:"part_phys_hash,omitempty"`
}

SafePartMapping records where a promoted part landed in hg_safe.

type StatementCoord

type StatementCoord struct {
	Account   string `json:"account"`
	ClientSeq uint64 `json:"client_seq"`
}

StatementCoord is the statement_id uniqueness coordinate: one statement per (account, client_seq); client_nonce is NOT part of the key (§6.1).

type StatementEnvelope

type StatementEnvelope struct {
	StatementID     StatementID   `json:"statement_id"`
	StatementKind   StatementKind `json:"statement_kind"`
	SQL             string        `json:"sql"`
	SQLHash         string        `json:"sql_hash"`
	SettingsHash    string        `json:"settings_hash,omitempty"`
	PayloadRef      string        `json:"payload_ref,omitempty"`
	PayloadHash     string        `json:"payload_hash,omitempty"`
	PayloadLength   uint64        `json:"payload_length,omitempty"`
	TargetTableID   string        `json:"target_table_id,omitempty"`
	UserJWS         string        `json:"user_jws"`
	EnvelopeVersion uint32        `json:"envelope_version"`
	NetworkID       string        `json:"network_id"`
	KeeperShardID   uint32        `json:"keeper_shard_id"`
	PayloadFormat   string        `json:"payload_format"`
	ClientRevision  uint32        `json:"client_revision"`
	SchemaHash      string        `json:"schema_hash"`
	RowIDProfileID  string        `json:"row_id_profile_id"`
}

StatementEnvelope is the canonical Go form of pb.StatementEnvelopeV2 (the V2 suffix is frozen on the wire only; the Go world drops it).

type StatementID

type StatementID struct {
	ClientAccount string `json:"client_account"`
	ClientSeq     uint64 `json:"client_seq"`
	ClientNonce   string `json:"client_nonce"`
}

StatementID is the structured client-assigned statement identity (uniqueness key = (client_account, client_seq); nonce is entropy, §6.1).

func (StatementID) Coord

func (id StatementID) Coord() StatementCoord

Coord is the accumulator uniqueness coordinate (account normalized).

func (StatementID) Flat

func (id StatementID) Flat() string

Flat renders the canonical flat statement_id string form.

type StatementKind

type StatementKind int32

StatementKind mirrors pb.StatementKind. v1 admits INSERT only.

const (
	StatementKindUnspecified StatementKind = 0
	StatementKindInsert      StatementKind = 1
)

type TablePartition

type TablePartition struct {
	TableID     string `json:"table_id"`
	PartitionID string `json:"partition_id"`
}

TablePartition addresses one partition of one logical table.

func (TablePartition) MarshalText

func (p TablePartition) MarshalText() ([]byte, error)

MarshalText/UnmarshalText let TablePartition key JSON maps in FSM snapshots. NUL is the delimiter — table/partition ids never contain it.

func (*TablePartition) UnmarshalText

func (p *TablePartition) UnmarshalText(b []byte) error

type UnsafeCleanup

type UnsafeCleanup struct {
	TableID      string    `json:"table_id"`
	PartitionID  string    `json:"partition_id"`
	PromotionSeq uint64    `json:"promotion_seq"`
	Parts        []PartRef `json:"parts"`
}

UnsafeCleanup is the canonical signing form of the promoted-unsafe-part cleanup command (design §8.5); wire form is pb.UnsafeCleanup.