shiftlock

package module
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 3, 2026 License: Apache-2.0 Imports: 29 Imported by: 0

README

ShiftLock

CI Go Reference License

ShiftLock is a security-first Go resource fabric: coordinate, protect, supervise, and recover application resources across processes, services, data systems, and deployments — without a hosted control plane.

Graceful shutdown stops an old process. ShiftLock decides who may perform protected work, how ownership moves, and (optionally) how databases, queues, APIs, and workflows stay safe under fencing tokens, policy, and lockdown.

Install

go get github.com/theworker02/shiftlock@latest

Supports the current and previous stable Go releases.

Quick start

be := memory.New()
coord, err := shiftlock.New(shiftlock.Config{
    Service: "billing", InstanceID: "pod-a", Backend: be, LeaseTTL: 15 * time.Second,
})
err = coord.Run(ctx, shiftlock.Worker{
    Name: "billing-reconciler",
    Run: func(ctx context.Context, ownership *shiftlock.Lease) error {
        // fence writes with ownership.FencingToken()
        <-ctx.Done()
        return nil
    },
})
go run ./examples/singleton-worker
go run ./examples/runtime-supervisor
go run ./examples/secure-control-plane
go run ./examples/infrastructure-orchestrator
go run ./examples/object-store-sync
go run ./cmd/shiftlock-inspect rehearse-handoff

Module: github.com/theworker02/shiftlock — clone https://github.com/theworker02/shiftlock.git.

Runtime (Phase 6, opt-in)

rt, err := shiftlock.NewRuntime(shiftlock.RuntimeConfig{
    Config:          shiftlock.Config{Service: "billing", InstanceID: "pod-a", Backend: be},
    SecurityProfile: shiftlock.ProfileStandard,
    EnableSupervisor: true,
    EnableAudit:      true,
})
defer rt.Close()
_ = rt.Supervisor() // task modes, bounded restarts
_ = rt.Features()

Security features are opt-in; shiftlock.New / Coordinator APIs remain unchanged. See docs/migration/phase-5-to-phase-6.md and docs/roadmap-phase-6.md.

Resource fabric & workflows (Phase 7, opt-in)

rt, err := shiftlock.NewRuntime(shiftlock.RuntimeConfig{
    Config:          shiftlock.Config{Service: "billing", InstanceID: "pod-a", Backend: be},
    EnableResources: true,
    EnableWorkflows: true,
    EnableLockdown:  true,
})
defer rt.Close()

_, _ = rt.Resources().Register(resmemory.Worker("production", "billing", "reconciler"), resource.Metadata{})

def, _ := workflow.Define("drain-reconcile").
    Step("drain", func(ctx context.Context, exec *workflow.ExecContext) (workflow.Result, error) {
        return workflow.Result{}, nil
    }).
    Build()
_ = rt.Workflows().Register(def)
_, _ = rt.Workflows().Run(ctx, "drain-reconcile", workflow.RunOptions{})

Local-first durable state:

shiftlock.WithLocalStateDir("/var/lib/shiftlock")(&cfg) // workflows journal + registry path

See docs/roadmap-phase-7.md and docs/audits/phase-7-audit.md. Ownership quick start above is unchanged.

Core API

claim, err := coordinator.Claim(ctx, "billing-reconciler")
lease, err := claim.WaitForOwnership(ctx)
handoff, err := coordinator.PrepareHandoff(ctx)
handoff.Drain(ctx)
handoff.Transfer(ctx, successorGenerationID)
handoff.Commit(ctx) // or Abort

Generation states: joining → standby → preparing → active → draining → transferring → retired|failed.

Backends

Backend Package Notes
Memory backend/memory Tests, fault injection, certification
PostgreSQL backend/postgres Row locks + durable OperationID ops table (Migrate)
Redis backend/redis Lua CAS + Local in-process; AOF recommended
Kubernetes backend/kubernetes Lease objects; no k8s deps on core

Operator tooling

go run ./cmd/shiftlock version
go run ./cmd/shiftlock status
go run ./cmd/shiftlock security scan -production -format text
go run ./cmd/shiftlock redteam run
go run ./cmd/shiftlock audit verify -file audit.ndjson
go run ./cmd/shiftlock snapshot create -out snap.json

go run ./cmd/shiftlock-inspect timeline -journal events.ndjson -claim NAME
go run ./cmd/shiftlock-inspect explain -journal events.ndjson -claim NAME
go run ./cmd/shiftlock-inspect incident create -journal events.ndjson -out incident.tar.gz
go run ./cmd/shiftlock-inspect recovery abort-transfer --claim C --expected-owner G --expected-token N --reason "..." --dry-run
go run ./cmd/shiftlock-inspect readiness-report -format json
go run ./cmd/shiftlock-inspect rehearse-handoff

shiftlock-inspect remains the journal/recovery toolkit and thin-aliases unified shiftlock subcommands when that binary is on PATH.

Recovery never blind force-unlocks: --expected-owner, --expected-token, --reason, and --confirm are required to mutate.

Safety & certification

go test ./backend/memory -run TestCertification
go test ./backend/redis -run TestLocalCertification
go test ./backend/kubernetes -run TestCertification
go test ./lab
go test ./model -count=1
  • Formal model: model/
  • Simulation: internal/simulation/
  • Fault injection: backend/faultinject
  • Chaos lab: lab/ (+ lab/docker-compose.yml)
  • Audits: phase-5 · phase-6 · phase-7
  • Red-team: go test ./security/redteam
  • Scanner: go run ./cmd/shiftlock security scan -production

Integrations

Optional ownership guards (no vendor SDKs on core): integration/{kafka,nats,rabbitmq,sqs,scheduler,httpserver,grpcserver} — see integration/README.md.

Identity providers: identity/{hostname,environment,pod,aws}.
Fencing helpers: fencing/{memory,postgres,redis}.

Documentation

License

Apache License 2.0 — see LICENSE.

Documentation

Overview

Package shiftlock coordinates ownership handoff between process generations during restarts, rolling deployments, and infrastructure replacements.

Only one generation may hold a valid committed fencing-token epoch for a given claim. Stale generations cannot overwrite or release newer ownership.

Index

Constants

View Source
const (
	CodeUnauthorized      = "unauthorized"
	CodeForbidden         = "forbidden"
	CodeReplay            = "replay"
	CodeCapabilityInvalid = "capability_invalid"
	CodeCapabilityExpired = "capability_expired"
	CodeCapabilityRevoked = "capability_revoked"
	CodeEpochOverflow     = "epoch_overflow"
	CodeAuditTamper       = "audit_tamper"
	CodeLockdownActive    = "lockdown_active"
	CodeMaintenanceActive = "maintenance_active"
	CodeQuarantined       = "quarantined"
	CodeGuardDenied       = "guard_denied"
	CodeInvalidArgument   = "invalid_argument"
	CodeDeadlineExceeded  = "deadline_exceeded"
	CodeRateLimited       = "rate_limited"
	CodeExecDenied        = "exec_denied"
)

Stable public error codes (safe to expose; no secrets).

Variables

View Source
var (
	ErrClosed             = errors.New("shiftlock: coordinator closed")
	ErrNotOwner           = errors.New("shiftlock: not claim owner")
	ErrAlreadyOwner       = errors.New("shiftlock: already claim owner")
	ErrClaimHeld          = errors.New("shiftlock: claim held by another generation")
	ErrStaleToken         = errors.New("shiftlock: stale fencing token")
	ErrInvalidState       = errors.New("shiftlock: invalid state transition")
	ErrDraining           = errors.New("shiftlock: draining in progress")
	ErrTransferPending    = errors.New("shiftlock: transfer already pending")
	ErrNoTransfer         = errors.New("shiftlock: no pending transfer")
	ErrTransferFailed     = errors.New("shiftlock: transfer failed")
	ErrHandoffAborted     = errors.New("shiftlock: handoff aborted")
	ErrTimeout            = errors.New("shiftlock: operation timed out")
	ErrNotReady           = errors.New("shiftlock: readiness gates not satisfied")
	ErrPolicy             = errors.New("shiftlock: policy validation failed")
	ErrBackend            = errors.New("shiftlock: backend error")
	ErrGenerationRetired  = errors.New("shiftlock: generation retired")
	ErrGenerationFailed   = errors.New("shiftlock: generation failed")
	ErrClaimNotFound      = errors.New("shiftlock: claim not found")
	ErrGenerationNotFound = errors.New("shiftlock: generation not found")
	ErrConcurrentTransfer = errors.New("shiftlock: concurrent transfer conflict")
	ErrLeaseLost          = errors.New("shiftlock: lease lost")
	ErrCanceled           = errors.New("shiftlock: canceled")
	ErrTokenOverflow      = errors.New("shiftlock: fencing token overflow")
	ErrAmbiguous          = errors.New("shiftlock: ambiguous backend outcome")
	ErrClaimUnavailable   = errors.New("shiftlock: claim unavailable")
	ErrCapability         = errors.New("shiftlock: backend capability mismatch")
	ErrSplitBrain         = errors.New("shiftlock: split-brain detected")

	// Phase 6 security / control-plane sentinels.
	ErrUnauthorized          = errors.New("shiftlock: unauthorized")
	ErrForbidden             = errors.New("shiftlock: forbidden")
	ErrReplay                = errors.New("shiftlock: replay detected")
	ErrCapabilityToken       = errors.New("shiftlock: capability token invalid")
	ErrSecurityEpochOverflow = errors.New("shiftlock: security epoch overflow")
	ErrAuditTamper           = errors.New("shiftlock: audit chain tamper detected")
	ErrLockdown              = errors.New("shiftlock: lockdown active")
	ErrMaintenance           = errors.New("shiftlock: maintenance active")
	ErrQuarantined           = errors.New("shiftlock: generation quarantined")
	ErrGuardDenied           = errors.New("shiftlock: guard denied")
	ErrExecDenied            = errors.New("shiftlock: exec denied")
	ErrRateLimited           = errors.New("shiftlock: rate limited")
	ErrRuntimeClosed         = errors.New("shiftlock: runtime closed")
)

Sentinel errors for stable caller matching via errors.Is.

Functions

func DiagnosticsHandler

func DiagnosticsHandler(c *Coordinator) http.Handler

DiagnosticsHandler returns an http.Handler that serves sanitized coordinator diagnostics as JSON. It does not start a server; mount it on your own mux. No secrets or backend credentials are included.

func IsPrivileged

func IsPrivileged(p Permission) bool

IsPrivileged reports whether p requires explicit authorization.

func ValidateCapabilities

func ValidateCapabilities(cfg Config, caps Capabilities) error

ValidateCapabilities checks policy requirements against backend capabilities. Returns ErrPolicy on unsafe mismatch — never silently degrades.

func WithLocalState

func WithLocalState(ls LocalStateConfig) func(*RuntimeConfig)

WithLocalState returns a helper that enables resources/workflows and sets LocalState.

func WithLocalStateDir

func WithLocalStateDir(dir string) func(*RuntimeConfig)

WithLocalStateDir enables fabric with journal + checkpoints under dir.

Types

type AbortRequest

type AbortRequest struct {
	ClaimName      string
	FromGeneration string
	ToGeneration   string
	ExpectedToken  FencingToken
	OperationID    OperationID
}

AbortRequest is the input to Backend.AbortTransfer.

type AcquireRequest

type AcquireRequest struct {
	ClaimName      string
	GenerationID   string
	TTL            time.Duration
	AllowEmptyOnly bool // if true, only acquire when unowned
	OperationID    OperationID
}

AcquireRequest is the input to Backend.AcquireClaim.

type Backend

type Backend interface {
	// RegisterGeneration records a new generation in joining/standby state.
	RegisterGeneration(ctx context.Context, gen Generation) error

	// AcquireClaim attempts to obtain ownership of claimName for generationID.
	// On success the returned record includes a new or existing fencing token.
	AcquireClaim(ctx context.Context, req AcquireRequest) (*ClaimRecord, error)

	// RenewClaim extends the lease for the current owner. Must fail with
	// ErrStaleToken / ErrNotOwner if the caller no longer owns the claim.
	RenewClaim(ctx context.Context, req RenewRequest) (*ClaimRecord, error)

	// PrepareTransfer reserves ownership transfer to successor without
	// advancing the fencing token yet.
	PrepareTransfer(ctx context.Context, req TransferRequest) (*ClaimRecord, error)

	// CommitTransfer atomically advances the fencing token and assigns
	// ownership to the successor.
	CommitTransfer(ctx context.Context, req CommitRequest) (*ClaimRecord, error)

	// AbortTransfer cancels a pending transfer and restores prior ownership.
	AbortTransfer(ctx context.Context, req AbortRequest) (*ClaimRecord, error)

	// ReleaseClaim releases ownership. Must refuse if fencing token is stale
	// so a partitioned former owner cannot release a newer owner's claim.
	ReleaseClaim(ctx context.Context, req ReleaseRequest) error

	// WatchClaim emits ownership change notifications. The channel is closed
	// when ctx is canceled or Close is called.
	WatchClaim(ctx context.Context, claimName string) (<-chan ClaimEvent, error)

	// UpdateGeneration persists generation state transitions.
	UpdateGeneration(ctx context.Context, gen Generation) error

	// GetClaim returns the current claim record or ErrClaimNotFound.
	GetClaim(ctx context.Context, claimName string) (*ClaimRecord, error)

	// GetGeneration returns a generation or ErrGenerationNotFound.
	GetGeneration(ctx context.Context, generationID string) (*Generation, error)

	// Close releases backend resources.
	Close() error
}

Backend stores generation and claim ownership state. All ownership mutations MUST be atomic under concurrency: at most one generation may hold a committed fencing-token epoch for a claim.

type BarrierFacade

type BarrierFacade struct {
	// contains filtered or unexported fields
}

BarrierFacade creates barriers.

func (*BarrierFacade) New

func (f *BarrierFacade) New(name string, cfg barrier.Config) (*barrier.Barrier, error)

New creates a named barrier.

type Capabilities

type Capabilities struct {
	// AtomicCAS guarantees fencing-token CAS under concurrency.
	AtomicCAS bool `json:"atomic_cas"`

	// IdempotentMutations supports OperationID dedupe on state-changing ops.
	IdempotentMutations bool `json:"idempotent_mutations"`

	// WatchSupported indicates WatchClaim is meaningful (not poll-only stub).
	WatchSupported bool `json:"watch_supported"`

	// DurableStorage indicates ownership survives process restart.
	DurableStorage bool `json:"durable_storage"`

	// ExpireBeforeMutate clears expired leases before prepare/commit/release.
	ExpireBeforeMutate bool `json:"expire_before_mutate"`

	// RenewDuringReserved allows heartbeats while transfer is reserved.
	RenewDuringReserved bool `json:"renew_during_reserved"`

	// GlobalExclusive can guarantee single-owner across all clients of the store.
	GlobalExclusive bool `json:"global_exclusive"`

	// MaxFencingToken is the highest token the backend will issue (0 = MaxUint64-1).
	MaxFencingToken FencingToken `json:"max_fencing_token,omitempty"`
}

Capabilities describes backend safety features. The coordinator validates Config.Policy against backend capabilities and refuses silent degradation.

func DefaultMemoryCapabilities

func DefaultMemoryCapabilities() Capabilities

DefaultMemoryCapabilities are the in-process memory backend capabilities.

type Capabler

type Capabler interface {
	Capabilities() Capabilities
}

Capabler is optionally implemented by backends that advertise capabilities.

type Claim

type Claim struct {
	// contains filtered or unexported fields
}

Claim is a named ownership unit coordinated across generations.

func (*Claim) DrainGroup

func (c *Claim) DrainGroup() *DrainGroup

DrainGroup returns the claim's drain group for in-flight work tracking.

func (*Claim) FencingToken

func (c *Claim) FencingToken() FencingToken

FencingToken returns the current known fencing token.

func (*Claim) Name

func (c *Claim) Name() string

Name returns the claim name.

func (*Claim) Ownership

func (c *Claim) Ownership() Ownership

Ownership returns a snapshot of current ownership.

func (*Claim) Release

func (c *Claim) Release(ctx context.Context) error

Release voluntarily releases ownership if this generation holds it.

func (*Claim) TryAcquire

func (c *Claim) TryAcquire(ctx context.Context) (*Lease, error)

TryAcquire attempts a single non-blocking acquire. Returns ErrClaimHeld if another generation owns the claim or a local acquire is already in flight.

func (*Claim) WaitForOwnership

func (c *Claim) WaitForOwnership(ctx context.Context) (*Lease, error)

WaitForOwnership blocks until this generation owns the claim or ctx ends.

type ClaimEvent

type ClaimEvent struct {
	Claim  ClaimRecord
	Time   time.Time
	Reason TransitionReason
}

ClaimEvent is emitted by WatchClaim.

type ClaimPhase

type ClaimPhase string

ClaimPhase describes the ownership status of a named claim.

const (
	// ClaimUnowned means no generation currently owns the claim.
	ClaimUnowned ClaimPhase = "unowned"
	// ClaimOwned means a generation holds a committed fencing token.
	ClaimOwned ClaimPhase = "owned"
	// ClaimReserved means a transfer is prepared but not yet committed.
	ClaimReserved ClaimPhase = "reserved"
	// ClaimDraining means the owner is draining before transfer or release.
	ClaimDraining ClaimPhase = "draining"
)

type ClaimRecord

type ClaimRecord struct {
	Name             string
	OwnerGeneration  string
	FencingToken     FencingToken
	Phase            ClaimPhase
	AcquiredAt       time.Time
	ExpiresAt        time.Time
	PreviousOwner    string
	PendingSuccessor string
	DrainStatus      string
	TransferStatus   string
	LastHeartbeat    time.Time
	Reason           TransitionReason
	Version          uint64 // opaque CAS version for backends that need it
}

ClaimRecord is the durable claim state stored by a backend.

func (*ClaimRecord) ToOwnership

func (r *ClaimRecord) ToOwnership() Ownership

ToOwnership converts a claim record to a public Ownership snapshot.

type Clock

type Clock interface {
	Now() time.Time
	After(d time.Duration) <-chan time.Time
	NewTicker(d time.Duration) Ticker
	Since(t time.Time) time.Duration
}

Clock provides time for deterministic tests.

type CommitRequest

type CommitRequest struct {
	ClaimName      string
	FromGeneration string
	ToGeneration   string
	ExpectedToken  FencingToken
	TTL            time.Duration
	OperationID    OperationID
}

CommitRequest is the input to Backend.CommitTransfer.

type Config

type Config struct {
	// Service is the logical service name (required).
	Service string

	// InstanceID uniquely identifies this process instance (required).
	InstanceID string

	// Backend stores ownership state (required).
	Backend Backend

	// GenerationID overrides the auto-generated generation id.
	// If empty, Service/InstanceID/timestamp is used.
	GenerationID string

	// LeaseTTL is how long a claim lease remains valid without renewal.
	// Default: 15s.
	LeaseTTL time.Duration

	// RenewInterval is how often the owner renews claims.
	// Default: LeaseTTL / 3.
	RenewInterval time.Duration

	// AcquireInterval is the retry interval while waiting for ownership.
	// Default: 500ms.
	AcquireInterval time.Duration

	// TransferTimeout bounds how long a reserved transfer may stay pending.
	// Default: 30s. Expired transfers are aborted.
	TransferTimeout time.Duration

	// DrainTimeout bounds graceful drain before forced transfer.
	// Default: 30s.
	DrainTimeout time.Duration

	// ReadinessTimeout bounds readiness gate evaluation.
	// Default: 30s.
	ReadinessTimeout time.Duration

	// WatchBuffer is the per-claim event channel buffer size.
	// Default: 16.
	WatchBuffer int

	// EventBuffer is the async observer buffer size.
	// Default: 64.
	EventBuffer int

	// Policy validates configuration and runtime constraints.
	Policy Policy

	// Clock overrides the wall clock (tests).
	Clock Clock

	// Hooks are synchronous callbacks invoked inline on events.
	Hooks []Hook

	// Observers receive events asynchronously.
	Observers []Observer
}

Config configures a Coordinator.

type Coordinator

type Coordinator struct {
	// contains filtered or unexported fields
}

Coordinator manages a process generation and its claims.

func New

func New(cfg Config) (*Coordinator, error)

New creates and registers a Coordinator generation.

func (*Coordinator) Capabilities

func (c *Coordinator) Capabilities() Capabilities

Capabilities returns negotiated backend capabilities.

func (*Coordinator) Claim

func (c *Coordinator) Claim(ctx context.Context, name string) (*Claim, error)

Claim obtains a handle for a named ownership unit.

func (*Coordinator) Close

func (c *Coordinator) Close() error

Close stops all internal goroutines and releases resources. Ownership is not forcibly released (successor may take over via expiry); use PrepareHandoff for graceful transfer.

func (*Coordinator) Config

func (c *Coordinator) Config() Config

Config returns a copy of the effective configuration (backend omitted from safety? keep full).

func (*Coordinator) DetectSplitBrain

func (c *Coordinator) DetectSplitBrain(ctx context.Context, claimName string) (*SplitBrainReport, error)

DetectSplitBrain compares local claim view to backend.

func (*Coordinator) Diagnostics

func (c *Coordinator) Diagnostics() Diagnostics

Diagnostics returns a sanitized snapshot for HTTP/CLI inspection.

func (*Coordinator) EventDropped

func (c *Coordinator) EventDropped() uint64

EventDropped returns count of async events dropped due to full buffer.

func (*Coordinator) Generation

func (c *Coordinator) Generation() Generation

Generation returns the current generation snapshot.

func (*Coordinator) Health

func (c *Coordinator) Health(_ context.Context) HealthReport

Health returns a multi-axis health snapshot.

func (*Coordinator) LastHeartbeat

func (c *Coordinator) LastHeartbeat() time.Time

LastHeartbeat returns the last successful backend renew/heartbeat time.

func (*Coordinator) PlanRecovery

func (c *Coordinator) PlanRecovery(ctx context.Context, claim string) (*RecoveryPlan, error)

PlanRecovery inspects claim state and proposes recovery steps.

func (*Coordinator) PrepareHandoff

func (c *Coordinator) PrepareHandoff(ctx context.Context) (*Handoff, error)

PrepareHandoff starts a graceful ownership handoff as the current owner.

func (*Coordinator) Run

func (c *Coordinator) Run(ctx context.Context, worker Worker) error

Run acquires ownership for worker.Name and invokes worker.Run. It returns when Run returns or ownership is lost.

type DegradationPolicy

type DegradationPolicy string

DegradationPolicy controls behavior when ownership cannot be proven.

const (
	// DegradeFailClosed stops protected work immediately (default).
	DegradeFailClosed DegradationPolicy = "fail_closed"
	// DegradeContinueUntilMargin continues until lease expiry minus renew margin.
	DegradeContinueUntilMargin DegradationPolicy = "continue_until_lease_margin"
	// DegradeFinishCurrentOnly finishes in-flight DrainGroup ops then stops.
	DegradeFinishCurrentOnly DegradationPolicy = "finish_current_work_only"
	// DegradeImmediateStop cancels lease contexts immediately.
	DegradeImmediateStop DegradationPolicy = "immediate_stop"
)

type Diagnostics

type Diagnostics struct {
	Service       string       `json:"service"`
	InstanceID    string       `json:"instance_id"`
	Generation    Generation   `json:"generation"`
	LastHeartbeat time.Time    `json:"last_heartbeat,omitempty"`
	EventDropped  uint64       `json:"event_dropped"`
	Capabilities  Capabilities `json:"capabilities"`
	Claims        []Ownership  `json:"claims"`
}

Diagnostics is a sanitized coordinator snapshot (no secrets).

type DrainGroup

type DrainGroup struct {
	// contains filtered or unexported fields
}

DrainGroup tracks in-flight work that must complete before ownership transfer or shutdown. Begin/BeginNamed return a release function.

func NewDrainGroup

func NewDrainGroup(maxNamed int) *DrainGroup

NewDrainGroup creates an empty drain group. maxNamed bounds concurrent named operations (0 = unlimited).

func (*DrainGroup) Active

func (g *DrainGroup) Active() int

Active returns the number of in-flight operations.

func (*DrainGroup) Begin

func (g *DrainGroup) Begin() (func(), error)

Begin starts an anonymous in-flight operation. Returns ErrDraining if drain has started, ErrClosed if closed.

func (*DrainGroup) BeginNamed

func (g *DrainGroup) BeginNamed(name string) (func(), error)

BeginNamed starts a named in-flight operation.

func (*DrainGroup) Close

func (g *DrainGroup) Close()

Close permanently rejects new work.

func (*DrainGroup) Draining

func (g *DrainGroup) Draining() bool

Draining reports whether drain has started.

func (*DrainGroup) SetDeadline

func (g *DrainGroup) SetDeadline(ch <-chan struct{})

SetDeadline cancels Wait when the channel closes.

func (*DrainGroup) StartDrain

func (g *DrainGroup) StartDrain()

StartDrain marks the group as draining. New Begin calls fail with ErrDraining.

func (*DrainGroup) Wait

func (g *DrainGroup) Wait(ctx context.Context) error

Wait blocks until all in-flight ops complete or ctx/deadline expires.

type ElectionFacade

type ElectionFacade struct {
	// contains filtered or unexported fields
}

ElectionFacade exposes Join.

func (*ElectionFacade) Join

func (f *ElectionFacade) Join(ctx context.Context, name string) (*election.Election, error)

Join starts participating in a named election.

type Error

type Error struct {
	Op       string
	Claim    string
	Gen      string
	Token    FencingToken
	Reason   TransitionReason
	Err      error
	Message  string
	Code     string
	Category ErrorCategory
}

Error is a typed shiftlock error with optional cause and context.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Is

func (e *Error) Is(target error) bool

func (*Error) PublicMessage

func (e *Error) PublicMessage() string

PublicMessage returns a caller-safe message (no internal paths/secrets).

func (*Error) Unwrap

func (e *Error) Unwrap() error

type ErrorCategory

type ErrorCategory string

ErrorCategory groups errors for operators and stable API docs.

const (
	CategoryGeneral       ErrorCategory = "general"
	CategorySecurity      ErrorCategory = "security"
	CategoryAuthorization ErrorCategory = "authorization"
	CategoryLockdown      ErrorCategory = "lockdown"
	CategoryMaintenance   ErrorCategory = "maintenance"
	CategoryCapability    ErrorCategory = "capability"
	CategoryAudit         ErrorCategory = "audit"
	CategoryQuarantine    ErrorCategory = "quarantine"
	CategoryPolicy        ErrorCategory = "policy"
	CategoryBackend       ErrorCategory = "backend"
	CategoryOwnership     ErrorCategory = "ownership"
)

type Event

type Event struct {
	Type       EventType         `json:"type"`
	Time       time.Time         `json:"time"`
	Service    string            `json:"service"`
	Generation string            `json:"generation,omitempty"`
	InstanceID string            `json:"instance_id,omitempty"`
	Claim      string            `json:"claim,omitempty"`
	Token      FencingToken      `json:"fencing_token,omitempty"`
	FromState  GenerationState   `json:"from_state,omitempty"`
	ToState    GenerationState   `json:"to_state,omitempty"`
	Reason     TransitionReason  `json:"reason,omitempty"`
	Err        string            `json:"error,omitempty"`
	Attrs      map[string]string `json:"attrs,omitempty"`
}

Event is a structured coordinator notification.

type EventFilter

type EventFilter func(Event) bool

EventFilter decides whether an event is delivered.

type EventType

type EventType string

EventType classifies coordinator lifecycle and ownership events.

const (
	EventGenerationRegistered EventType = "generation.registered"
	EventGenerationState      EventType = "generation.state"
	EventClaimAcquired        EventType = "claim.acquired"
	EventClaimRenewed         EventType = "claim.renewed"
	EventClaimLost            EventType = "claim.lost"
	EventClaimReleased        EventType = "claim.released"
	EventClaimExpired         EventType = "claim.expired"
	EventDrainStarted         EventType = "drain.started"
	EventDrainCompleted       EventType = "drain.completed"
	EventDrainFailed          EventType = "drain.failed"
	EventTransferPrepared     EventType = "transfer.prepared"
	EventTransferCommitted    EventType = "transfer.committed"
	EventTransferAborted      EventType = "transfer.aborted"
	EventHandoffStarted       EventType = "handoff.started"
	EventHandoffCompleted     EventType = "handoff.completed"
	EventHandoffFailed        EventType = "handoff.failed"
	EventReadinessStarted     EventType = "readiness.started"
	EventReadinessPassed      EventType = "readiness.passed"
	EventReadinessFailed      EventType = "readiness.failed"
	EventLeaseRevoked         EventType = "lease.revoked"
	EventBackendHeartbeat     EventType = "backend.heartbeat"
	EventError                EventType = "error"
	EventClosed               EventType = "closed"
)

type FeatureFlags

type FeatureFlags struct {
	Supervisor   bool `json:"supervisor"`
	Commands     bool `json:"commands"`
	Maintenance  bool `json:"maintenance"`
	Lockdown     bool `json:"lockdown"`
	Capabilities bool `json:"capabilities"`
	Guard        bool `json:"guard"`
	Audit        bool `json:"audit"`
	AntiReplay   bool `json:"anti_replay"`
	Exec         bool `json:"exec"`
	Resources    bool `json:"resources"`
	Workflows    bool `json:"workflows"`
}

FeatureFlags reports which Runtime subsystems are active.

type FencedClaims

type FencedClaims struct{ RT *Runtime }

FencedClaims adapts Coordinator claims for election/syncprim.

func (FencedClaims) Acquire

func (f FencedClaims) Acquire(ctx context.Context, claim string) (uint64, error)

Acquire obtains a claim lease token.

func (FencedClaims) Release

func (f FencedClaims) Release(ctx context.Context, claim string, _ uint64) error

Release releases a claim.

func (FencedClaims) Renew

func (f FencedClaims) Renew(_ context.Context, claim string, token uint64) error

Renew checks continued ownership (coordinator renew loop handles heartbeats).

type FencingToken

type FencingToken uint64

FencingToken is a monotonically increasing ownership epoch. Callers MUST reject work associated with a token older than the newest accepted token for a given claim.

const MaxSafeFencingToken FencingToken = FencingToken(^uint64(0) - 1)

MaxSafeFencingToken is the last token that may be issued before terminal overflow.

func (FencingToken) Less

func (t FencingToken) Less(other FencingToken) bool

Less reports whether t is strictly older than other.

func (FencingToken) Zero

func (t FencingToken) Zero() bool

Zero reports whether the token is unset.

type Gate

type Gate struct {
	// Name identifies the gate in reports.
	Name string
	// Check performs the readiness probe. Return nil on success.
	Check func(ctx context.Context) error
	// Required means failure blocks readiness (default true).
	Required bool
	// Optional is the inverse of Required for convenience.
	Optional bool
	// Timeout bounds a single Check invocation (0 = use group default).
	Timeout time.Duration
	// Retries is the number of additional attempts after the first failure.
	Retries int
	// RetryDelay waits between retries.
	RetryDelay time.Duration
	// StableSuccesses requires N consecutive successes (default 1).
	StableSuccesses int
}

Gate is a readiness check that must pass before becoming active.

type GateMode

type GateMode int

GateMode controls concurrency of gate evaluation.

const (
	// GateSequential evaluates gates one at a time in order.
	GateSequential GateMode = iota
	// GateParallel evaluates all gates concurrently.
	GateParallel
)

type GateReport

type GateReport struct {
	Name     string        `json:"name"`
	Passed   bool          `json:"passed"`
	Required bool          `json:"required"`
	Attempts int           `json:"attempts"`
	Duration time.Duration `json:"duration"`
	Err      string        `json:"error,omitempty"`
}

GateReport is the result of evaluating a single gate.

type Generation

type Generation struct {
	ID         string           `json:"id"`
	Service    string           `json:"service"`
	InstanceID string           `json:"instance_id"`
	State      GenerationState  `json:"state"`
	StartedAt  time.Time        `json:"started_at"`
	UpdatedAt  time.Time        `json:"updated_at"`
	Reason     TransitionReason `json:"reason,omitempty"`
}

Generation identifies a process generation participating in ownership.

func (Generation) Clone

func (g Generation) Clone() Generation

Clone returns a shallow copy.

type GenerationState

type GenerationState string

GenerationState is the lifecycle state of a process generation.

const (
	// StateJoining is the initial state after registration.
	StateJoining GenerationState = "joining"
	// StateStandby means the generation is registered and waiting.
	StateStandby GenerationState = "standby"
	// StatePreparing means the generation is running readiness gates.
	StatePreparing GenerationState = "preparing"
	// StateActive means the generation holds committed ownership.
	StateActive GenerationState = "active"
	// StateDraining means the generation is finishing in-flight work.
	StateDraining GenerationState = "draining"
	// StateTransferring means ownership transfer is reserved but not committed.
	StateTransferring GenerationState = "transferring"
	// StateRetired means the generation has permanently released ownership.
	StateRetired GenerationState = "retired"
	// StateFailed means the generation failed and must not reclaim work.
	StateFailed GenerationState = "failed"
)

func (GenerationState) Terminal

func (s GenerationState) Terminal() bool

Terminal reports whether the generation can no longer become active.

func (GenerationState) Valid

func (s GenerationState) Valid() bool

Valid reports whether s is a known generation state.

type Handoff

type Handoff struct {
	// contains filtered or unexported fields
}

Handoff coordinates graceful ownership transfer from this generation to a successor. Sequence: Drain → Transfer → Commit (or Abort).

Normal sequence:

  1. Successor registers and passes readiness
  2. Current owner Drain() finishes in-flight work
  3. Transfer(successorID) reserves the claim
  4. Commit() advances fencing token; successor becomes active
  5. Previous owner observes stale token and retires

Rollback: Abort() before Commit restores prior ownership without advancing the fencing token.

func (*Handoff) Abort

func (h *Handoff) Abort(ctx context.Context) error

Abort cancels a pending transfer and restores prior ownership.

func (*Handoff) Commit

func (h *Handoff) Commit(ctx context.Context) error

Commit advances fencing tokens and assigns ownership to the successor.

func (*Handoff) Drain

func (h *Handoff) Drain(ctx context.Context) error

Drain stops accepting new work and waits for in-flight ops on all claims.

func (*Handoff) Status

func (h *Handoff) Status() HandoffStatus

Status returns the current handoff status.

func (*Handoff) Transfer

func (h *Handoff) Transfer(ctx context.Context, successorGenerationID string) error

Transfer reserves ownership transfer to successorGenerationID for all owned claims. Drain must complete first unless force is implied by empty claims.

type HandoffStatus

type HandoffStatus string

HandoffStatus tracks handoff progress.

const (
	HandoffPending      HandoffStatus = "pending"
	HandoffDraining     HandoffStatus = "draining"
	HandoffTransferring HandoffStatus = "transferring"
	HandoffCommitted    HandoffStatus = "committed"
	HandoffAborted      HandoffStatus = "aborted"
	HandoffFailed       HandoffStatus = "failed"
)

type HealthReport

type HealthReport struct {
	Time        time.Time         `json:"time"`
	Process     HealthStatus      `json:"process"`
	Backend     HealthStatus      `json:"backend"`
	Coordinator HealthStatus      `json:"coordinator"`
	Readiness   HealthStatus      `json:"readiness"`
	Ownership   HealthStatus      `json:"ownership"`
	Degradation DegradationPolicy `json:"degradation_policy"`
	Details     map[string]string `json:"details,omitempty"`
}

HealthReport separates process/backend/coordinator/readiness/ownership health.

type HealthStatus

type HealthStatus string

HealthStatus is a coarse health signal.

const (
	HealthOK       HealthStatus = "ok"
	HealthDegraded HealthStatus = "degraded"
	HealthFailing  HealthStatus = "failing"
	HealthUnknown  HealthStatus = "unknown"
)

type Hook

type Hook func(Event)

Hook is a synchronous callback invoked inline (must not block long).

type Lease

type Lease struct {
	// contains filtered or unexported fields
}

Lease is a live ownership grant. Canceling lease.Context() means ownership was lost or the coordinator is shutting down.

func (*Lease) Context

func (l *Lease) Context() context.Context

Context is canceled when ownership is lost, the claim is closed, or the coordinator shuts down.

func (*Lease) FencingToken

func (l *Lease) FencingToken() FencingToken

FencingToken returns the fencing token for this lease epoch.

func (*Lease) Ownership

func (l *Lease) Ownership() Ownership

Ownership returns a snapshot of the current ownership record.

func (*Lease) Valid

func (l *Lease) Valid() bool

Valid reports whether the lease context is still active.

type LocalStateConfig

type LocalStateConfig struct {
	Mode         LocalStateMode
	WorkflowPath string // used when Mode == LocalStateFile
	Dir          string // used when Mode == LocalStateDir (or with WithLocalStateDir)
	MaxInstances int
}

LocalStateConfig is opt-in local-first fabric state.

type LocalStateMode

type LocalStateMode string

LocalStateMode selects where Phase 7 local-first durable state is kept.

const (
	// LocalStateMemory keeps resource/workflow durable state in-process.
	LocalStateMemory LocalStateMode = "memory"
	// LocalStateFile keeps workflow checkpoints in a JSON file.
	LocalStateFile LocalStateMode = "file"
	// LocalStateDir keeps a local-first layout under Dir:
	//   Dir/workflows/checkpoints.journal  — journal-backed workflow store
	//   Dir/registry/events.ndjson         — resource registry event journal
	LocalStateDir LocalStateMode = "dir"
)

type Observer

type Observer interface {
	OnEvent(Event)
}

Observer receives events asynchronously.

type ObserverFunc

type ObserverFunc func(Event)

ObserverFunc adapts a function to Observer.

func (ObserverFunc) OnEvent

func (f ObserverFunc) OnEvent(e Event)

type OperationID

type OperationID string

OperationID is a stable client-generated identifier for idempotent mutations. Retries with the same OperationID must return the prior successful result without advancing the fencing token or emitting duplicate history.

func (OperationID) Empty

func (id OperationID) Empty() bool

Empty reports whether the ID is unset (legacy non-idempotent path).

type Ownership

type Ownership struct {
	ClaimName        string           `json:"claim_name"`
	OwnerGeneration  string           `json:"owner_generation,omitempty"`
	FencingToken     FencingToken     `json:"fencing_token"`
	Phase            ClaimPhase       `json:"phase"`
	AcquiredAt       time.Time        `json:"acquired_at,omitempty"`
	ExpiresAt        time.Time        `json:"expires_at,omitempty"`
	PreviousOwner    string           `json:"previous_owner,omitempty"`
	PendingSuccessor string           `json:"pending_successor,omitempty"`
	DrainStatus      string           `json:"drain_status,omitempty"`
	TransferStatus   string           `json:"transfer_status,omitempty"`
	LastHeartbeat    time.Time        `json:"last_heartbeat,omitempty"`
	Reason           TransitionReason `json:"reason,omitempty"`
}

Ownership describes committed or reserved ownership of a claim.

func (Ownership) Clone

func (o Ownership) Clone() Ownership

Clone returns a shallow copy.

func (Ownership) Controls

func (o Ownership) Controls(genID string) bool

Controls reports whether genID still controls the claim for renewals (owned or reserved as current owner with a non-zero token).

func (Ownership) OwnedBy

func (o Ownership) OwnedBy(genID string) bool

OwnedBy reports whether genID is the committed owner (phase owned). During a reserved transfer this returns false so workers stop accepting new protected work. Use Controls for heartbeat/renew eligibility.

type Permission

type Permission string

Permission is a stable authorization unit. Privileged permissions are denied unless explicitly granted via capability / guard policy.

const (
	PermClaimAcquire      Permission = "claim.acquire"
	PermClaimRelease      Permission = "claim.release"
	PermClaimForceRelease Permission = "claim.force_release"
	PermTaskStart         Permission = "task.start"
	PermTaskStop          Permission = "task.stop"
	PermTaskRestart       Permission = "task.restart"
	PermMaintenanceEnter  Permission = "maintenance.enter"
	PermMaintenanceExit   Permission = "maintenance.exit"
	PermLockdownEnter     Permission = "lockdown.enter"
	PermLockdownExit      Permission = "lockdown.exit"
	PermCommandInvoke     Permission = "command.invoke"
	PermCommandRegister   Permission = "command.register"
	PermCapabilityIssue   Permission = "capability.issue"
	PermCapabilityRevoke  Permission = "capability.revoke"
	PermElectionJoin      Permission = "election.join"
	PermElectionResign    Permission = "election.resign"
	PermAuditRead         Permission = "audit.read"
	PermExecRun           Permission = "exec.run"
	PermQuorumVote        Permission = "quorum.vote"
	PermRecoveryAct       Permission = "recovery.act"
)

type Policy

type Policy struct {
	// MinLeaseTTL rejects configs with shorter leases.
	MinLeaseTTL time.Duration

	// MaxLeaseTTL rejects configs with longer leases.
	MaxLeaseTTL time.Duration

	// RequireRenewBelowTTL requires RenewInterval < LeaseTTL.
	RequireRenewBelowTTL bool

	// MaxConcurrentClaims limits claims per coordinator (0 = unlimited).
	MaxConcurrentClaims int

	// AllowForceRelease permits ReleaseClaim without being active owner.
	// Default false — required for fencing safety.
	AllowForceRelease bool

	// RejectStaleRelease ensures backends refuse stale-token releases.
	// Always enforced by memory/postgres/redis backends.
	RejectStaleRelease bool

	// RequireDurable refuses backends without durable storage.
	RequireDurable bool

	// RequireIdempotent requires OperationID-capable backends.
	RequireIdempotent bool

	// RequireGlobalExclusive requires the backend to guarantee cross-client exclusivity.
	RequireGlobalExclusive bool

	// FailClosedOnAmbiguous stops workers when backend outcomes are ambiguous (default true).
	FailClosedOnAmbiguous bool

	// AllowLocalDegradation permits non-durable backends (default true for tests/dev).
	AllowLocalDegradation bool
}

Policy constrains coordinator behavior for safety.

func (Policy) Validate

func (p Policy) Validate(cfg Config) error

Validate checks config against policy.

type Principal

type Principal struct {
	ID         string            `json:"id"`
	Kind       PrincipalKind     `json:"kind"`
	Generation string            `json:"generation,omitempty"`
	Attrs      map[string]string `json:"attrs,omitempty"`
}

Principal identifies an actor for authorization and audit.

type PrincipalKind

type PrincipalKind string

PrincipalKind classifies principals.

const (
	PrincipalService    PrincipalKind = "service"
	PrincipalOperator   PrincipalKind = "operator"
	PrincipalGeneration PrincipalKind = "generation"
	PrincipalSystem     PrincipalKind = "system"
	PrincipalCapability PrincipalKind = "capability"
)

type QuorumFacade

type QuorumFacade struct {
	// contains filtered or unexported fields
}

QuorumFacade creates quorum barriers.

func (*QuorumFacade) New

func (f *QuorumFacade) New(name string, maxParticipants int) (*barrier.Barrier, error)

New creates a quorum barrier (majority of max).

type Readiness

type Readiness struct {
	Gates   []Gate
	Mode    GateMode
	Timeout time.Duration
	Clock   Clock
}

Readiness evaluates a set of gates.

func (Readiness) Run

Run evaluates all gates and returns a report.

type ReadinessReport

type ReadinessReport struct {
	Passed   bool          `json:"passed"`
	Duration time.Duration `json:"duration"`
	Gates    []GateReport  `json:"gates"`
}

ReadinessReport summarizes gate evaluation.

type RecoveryFacade

type RecoveryFacade struct {
	// contains filtered or unexported fields
}

RecoveryFacade wraps recovery planning.

func (*RecoveryFacade) Plan

func (f *RecoveryFacade) Plan(ctx context.Context, claim string) (*RecoveryPlan, error)

Plan inspects claim state for operator recovery.

type RecoveryPlan

type RecoveryPlan struct {
	Claim           string       `json:"claim"`
	Situation       string       `json:"situation"`
	Recommended     []string     `json:"recommended_actions"`
	RequiresConfirm bool         `json:"requires_confirm"`
	ExpectedToken   FencingToken `json:"expected_token,omitempty"`
	ExpectedOwner   string       `json:"expected_owner,omitempty"`
}

RecoveryPlan describes safe operator next steps (no auto-destructive actions).

type ReleaseRequest

type ReleaseRequest struct {
	ClaimName    string
	GenerationID string
	Token        FencingToken
	OperationID  OperationID
}

ReleaseRequest is the input to Backend.ReleaseClaim.

type RenewRequest

type RenewRequest struct {
	ClaimName    string
	GenerationID string
	Token        FencingToken
	TTL          time.Duration
	OperationID  OperationID
}

RenewRequest is the input to Backend.RenewClaim.

type ReplayCache

type ReplayCache struct {
	// contains filtered or unexported fields
}

ReplayCache is a bounded anti-replay store for request IDs and nonces. Expired entries are evicted on access; when full, oldest entries are dropped.

func NewReplayCache

func NewReplayCache(maxEntries int, clock Clock) *ReplayCache

NewReplayCache creates a bounded cache. maxEntries must be > 0.

func (*ReplayCache) CheckAndStore

func (c *ReplayCache) CheckAndStore(key string, ttl time.Duration) error

CheckAndStore returns ErrReplay if key was seen and not expired; otherwise stores it until expiry.

func (*ReplayCache) Len

func (c *ReplayCache) Len() int

Len returns the number of live entries (approximate after lazy eviction).

func (*ReplayCache) Seen

func (c *ReplayCache) Seen(key string) bool

Seen reports whether key is present and unexpired without storing.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime composes Coordinator with optional security and control-plane services.

func NewRuntime

func NewRuntime(cfg RuntimeConfig) (*Runtime, error)

NewRuntime constructs a Runtime. defer runtime.Close() stops all subsystems.

func (*Runtime) AllowElection

func (r *Runtime) AllowElection(_ string) error

AllowElection implements election.Gate.

func (*Runtime) AllowTask

func (r *Runtime) AllowTask(spec supervise.Spec) error

AllowTask implements supervise.Gate.

func (*Runtime) Audit

func (r *Runtime) Audit() *audit.Store

Audit returns the audit store (may be nil).

func (*Runtime) AuditCommand

func (r *Runtime) AuditCommand(actor, name, decision, outcome string)

AuditCommand implements command.Auditor.

func (*Runtime) AuthorizeCommand

func (r *Runtime) AuthorizeCommand(actor, name, permission string) error

AuthorizeCommand implements command.Authorizer.

func (*Runtime) Barrier

func (r *Runtime) Barrier() *BarrierFacade

Barrier returns the barrier façade.

func (*Runtime) Capabilities

func (r *Runtime) Capabilities() *capability.Authority

Capabilities returns the capability authority (may be nil).

func (*Runtime) Claims

func (r *Runtime) Claims() *Coordinator

Claims is a convenience alias for the coordinator (claim access).

func (*Runtime) Close

func (r *Runtime) Close() error

Close stops all subsystems then the coordinator.

func (*Runtime) Commands

func (r *Runtime) Commands() *command.Registry

Commands returns the command registry (may be nil).

func (*Runtime) Coordinator

func (r *Runtime) Coordinator() *Coordinator

Coordinator returns the underlying Phase 5 coordinator.

func (*Runtime) Election

func (r *Runtime) Election() *ElectionFacade

Election returns the election façade.

func (*Runtime) EnterQuarantine

func (r *Runtime) EnterQuarantine(reason string)

EnterQuarantine marks the generation unable to acquire claims / vote / issue caps.

func (*Runtime) ExecGuard

func (r *Runtime) ExecGuard() *execguard.Guard

ExecGuard returns the exec allowlist guard.

func (*Runtime) Failover

func (r *Runtime) Failover() *failover.Manager

Failover returns the failover manager (may be nil if resources not enabled).

func (*Runtime) Features

func (r *Runtime) Features() FeatureFlags

Features returns active subsystem flags.

func (*Runtime) Guard

func (r *Runtime) Guard() *guard.Engine

Guard returns the policy engine (may be nil).

func (*Runtime) Health

func (r *Runtime) Health(ctx context.Context) health.Report

Health returns an extended health graph report.

func (*Runtime) IssueCapability

func (r *Runtime) IssueCapability(req capability.Request) (capability.Token, error)

IssueCapability issues a capability when the authority is enabled and not quarantined.

func (*Runtime) LocalStateDir

func (r *Runtime) LocalStateDir() string

LocalStateDir returns the configured local-first state directory (may be empty).

func (*Runtime) Lockdown

func (r *Runtime) Lockdown() *lockdown.Manager

Lockdown returns the lockdown manager (may be nil).

func (*Runtime) Maintenance

func (r *Runtime) Maintenance() *maintenance.Manager

Maintenance returns the maintenance manager (may be nil).

func (*Runtime) Migrations

func (r *Runtime) Migrations() *migration.Coordinator

Migrations returns the migration coordinator (may be nil if resources not enabled).

func (*Runtime) Quarantined

func (r *Runtime) Quarantined() bool

Quarantined reports generation quarantine.

func (*Runtime) Quorum

func (r *Runtime) Quorum() *QuorumFacade

Quorum returns a quorum helper façade (barrier policy-based).

func (*Runtime) Recovery

func (r *Runtime) Recovery() *RecoveryFacade

Recovery returns recovery planning façade.

func (*Runtime) Replay

func (r *Runtime) Replay() *ReplayCache

Replay returns the root anti-replay cache (may be nil).

func (*Runtime) Resources

func (r *Runtime) Resources() *resource.Registry

Resources returns the Runtime-owned resource registry (may be nil if not enabled).

func (*Runtime) Security

func (r *Runtime) Security() SecuritySettings

Security returns inspectable expanded settings.

func (*Runtime) SecurityEpoch

func (r *Runtime) SecurityEpoch() SecurityEpoch

SecurityEpoch returns the current epoch.

func (*Runtime) Supervisor

func (r *Runtime) Supervisor() *supervise.Supervisor

Supervisor returns the task supervisor (may be nil).

func (*Runtime) Sync

func (r *Runtime) Sync() *syncpkg.Engine

Sync returns the sync engine façade (may be nil if resources not enabled).

func (*Runtime) TriggerSplitBrainLockdown

func (r *Runtime) TriggerSplitBrainLockdown(claim string)

TriggerSplitBrainLockdown is invoked after DetectSplitBrain when profiles request it.

func (*Runtime) VerifyAudit

func (r *Runtime) VerifyAudit() error

VerifyAudit runs chain verification and may auto-lockdown / quarantine.

func (*Runtime) Workflows

func (r *Runtime) Workflows() *workflow.Engine

Workflows returns the workflow engine (may be nil if not enabled).

type RuntimeConfig

type RuntimeConfig struct {
	Config

	// SecurityProfile expands to inspectable SecuritySettings.
	SecurityProfile SecurityProfile

	// SecurityOverrides overlays duration/numeric fields from the profile.
	SecurityOverrides *SecuritySettings

	// ApplySecurityOverridesBooleans when true copies boolean fields from SecurityOverrides.
	ApplySecurityOverridesBooleans bool

	EnableSupervisor   bool
	EnableCommands     bool
	EnableMaintenance  bool
	EnableLockdown     bool
	EnableCapabilities bool
	EnableGuard        bool
	EnableAudit        bool

	// Phase 7 fabric (opt-in; Coordinator APIs unchanged).
	EnableResources bool
	EnableWorkflows bool
	MaxResources    int
	LocalState      *LocalStateConfig

	// AuditStore overrides the default in-memory audit store.
	AuditStore *audit.Store

	MaintenancePath      string
	LockdownPath         string
	LockdownEvidencePath string

	// Capability options applied when EnableCapabilities is set.
	CapabilityOptions []capability.Option

	SecurityEpoch SecurityEpoch
}

RuntimeConfig configures an opt-in security-aware Runtime. Existing Coordinator Config remains valid; security subsystems are additive.

type SecurityEpoch

type SecurityEpoch uint64

SecurityEpoch is a monotonic authorization epoch. It must never decrease or wrap silently; overflow is a terminal error.

const MaxSecurityEpoch SecurityEpoch = SecurityEpoch(^uint64(0) - 1)

MaxSecurityEpoch is the last valid epoch before terminal overflow.

func (SecurityEpoch) Next

func (e SecurityEpoch) Next() (SecurityEpoch, error)

Next returns epoch+1 or ErrSecurityEpochOverflow.

type SecurityProfile

type SecurityProfile string

SecurityProfile selects a named secure-defaults bundle.

const (
	ProfileDevelopment     SecurityProfile = "development"
	ProfileTesting         SecurityProfile = "testing"
	ProfileStandard        SecurityProfile = "standard"
	ProfileHardened        SecurityProfile = "hardened"
	ProfileMaximumSecurity SecurityProfile = "maximum-security"
)

type SecuritySettings

type SecuritySettings struct {
	Profile SecurityProfile `json:"profile"`

	// DenyPrivilegedByDefault rejects privileged ops without an allow decision.
	DenyPrivilegedByDefault bool `json:"deny_privileged_by_default"`

	// RequireCapabilityForPrivileged requires a verified capability token.
	RequireCapabilityForPrivileged bool `json:"require_capability_for_privileged"`

	// AuditEnabled turns on hash-chained audit recording.
	AuditEnabled bool `json:"audit_enabled"`

	// AuditFailClosed refuses privileged ops if audit append fails.
	AuditFailClosed bool `json:"audit_fail_closed"`

	// AntiReplayEnabled enables bounded nonce/request-id cache checks.
	AntiReplayEnabled bool `json:"anti_replay_enabled"`

	// AntiReplayMaxEntries bounds the replay cache (required > 0 when enabled).
	AntiReplayMaxEntries int `json:"anti_replay_max_entries"`

	// CapabilityMaxTTL caps issued capability lifetime.
	CapabilityMaxTTL time.Duration `json:"capability_max_ttl"`

	// CapabilityDefaultTTL is used when Issue omits TTL.
	CapabilityDefaultTTL time.Duration `json:"capability_default_ttl"`

	// RequireSingleUseCapabilities forces single-use on issue when true.
	RequireSingleUseCapabilities bool `json:"require_single_use_capabilities"`

	// LockdownOnAuditTamper auto-enters lockdown when audit verify fails.
	LockdownOnAuditTamper bool `json:"lockdown_on_audit_tamper"`

	// LockdownOnSplitBrain auto-enters lockdown on split-brain detection.
	LockdownOnSplitBrain bool `json:"lockdown_on_split_brain"`

	// LockdownOnCommandFlood auto-enters lockdown after unauthorized flood.
	LockdownOnCommandFlood bool `json:"lockdown_on_command_flood"`

	// CommandFloodThreshold is unauthorized denials per window.
	CommandFloodThreshold int `json:"command_flood_threshold"`

	// CommandFloodWindow is the flood detection window.
	CommandFloodWindow time.Duration `json:"command_flood_window"`

	// AllowExec enables execguard (still allowlist-only). Default false.
	AllowExec bool `json:"allow_exec"`

	// QuarantineOnTamper marks the generation quarantined on security tamper.
	QuarantineOnTamper bool `json:"quarantine_on_tamper"`

	// MaxCommandBodyBytes limits command payloads.
	MaxCommandBodyBytes int `json:"max_command_body_bytes"`

	// DefaultCommandDeadline bounds command execution.
	DefaultCommandDeadline time.Duration `json:"default_command_deadline"`
}

SecuritySettings are the expanded, inspectable security controls. Zero-value fields after ExpandSecurityProfile mean "use profile default".

func ExpandSecurityProfile

func ExpandSecurityProfile(p SecurityProfile) SecuritySettings

ExpandSecurityProfile returns explicit settings for a profile.

func MergeSecurity

func MergeSecurity(base SecuritySettings, overlay SecuritySettings) SecuritySettings

MergeSecurity overlays non-zero overrides onto base (booleans use pointer-less explicit OverrideSecurity for RuntimeConfig).

type SplitBrainReport

type SplitBrainReport struct {
	Claim        string       `json:"claim"`
	LocalToken   FencingToken `json:"local_token"`
	BackendToken FencingToken `json:"backend_token"`
	LocalOwner   string       `json:"local_owner"`
	BackendOwner string       `json:"backend_owner"`
	DetectedAt   time.Time    `json:"detected_at"`
	ActionTaken  string       `json:"action_taken,omitempty"`
}

SplitBrainReport describes conflicting ownership observations.

type Ticker

type Ticker interface {
	C() <-chan time.Time
	Stop()
}

Ticker is a cancelable periodic timer.

type TokenValidator

type TokenValidator struct {
	// contains filtered or unexported fields
}

TokenValidator rejects stale fencing tokens. Accept is safe for concurrent use.

func NewTokenValidator

func NewTokenValidator() *TokenValidator

NewTokenValidator returns a validator that initially accepts any non-zero token.

func (*TokenValidator) Accept

func (v *TokenValidator) Accept(token FencingToken) bool

Accept returns true if token is newer than or equal to the current epoch, and atomically advances the stored epoch when token is newer. A zero token is always rejected.

Invariant: once Accept returns true for token T, any later call with token < T returns false. Tokens never decrease.

func (*TokenValidator) Current

func (v *TokenValidator) Current() FencingToken

Current returns the newest accepted token.

type TransferRequest

type TransferRequest struct {
	ClaimName      string
	FromGeneration string
	ToGeneration   string
	Token          FencingToken
	TTL            time.Duration
	OperationID    OperationID
}

TransferRequest is the input to Backend.PrepareTransfer.

type TransitionReason

type TransitionReason string

TransitionReason explains why a generation or claim changed state.

const (
	ReasonRegistered        TransitionReason = "registered"
	ReasonReadinessPassed   TransitionReason = "readiness_passed"
	ReasonReadinessFailed   TransitionReason = "readiness_failed"
	ReasonAcquired          TransitionReason = "acquired"
	ReasonRenewed           TransitionReason = "renewed"
	ReasonDrainStarted      TransitionReason = "drain_started"
	ReasonDrainComplete     TransitionReason = "drain_complete"
	ReasonTransferPrepared  TransitionReason = "transfer_prepared"
	ReasonTransferCommitted TransitionReason = "transfer_committed"
	ReasonTransferAborted   TransitionReason = "transfer_aborted"
	ReasonReleased          TransitionReason = "released"
	ReasonExpired           TransitionReason = "expired"
	ReasonFencedOut         TransitionReason = "fenced_out"
	ReasonRetired           TransitionReason = "retired"
	ReasonFailed            TransitionReason = "failed"
	ReasonClosed            TransitionReason = "closed"
	ReasonRollback          TransitionReason = "rollback"
	ReasonTimeout           TransitionReason = "timeout"
	ReasonPartition         TransitionReason = "partition"
)

type Worker

type Worker struct {
	// Name is the claim name to own before Run is invoked.
	Name string
	// Run executes work. ownership.Context() is canceled when the lease is lost.
	Run func(ctx context.Context, ownership *Lease) error
	// Readiness optional gates evaluated before acquiring ownership.
	Readiness *Readiness
}

Worker runs under exclusive ownership of a named claim.

Directories

Path Synopsis
Package audit provides a tamper-evident, hash-chained audit log.
Package audit provides a tamper-evident, hash-chained audit log.
backend
kubernetes
Package kubernetes implements a ShiftLock backend using Kubernetes Lease objects.
Package kubernetes implements a ShiftLock backend using Kubernetes Lease objects.
Package barrier coordinates participants with epoch isolation and bounds.
Package barrier coordinates participants with epoch isolation and bounds.
Package budget provides operation budgets with stop/pause/degrade behaviors.
Package budget provides operation budgets with stop/pause/degrade behaviors.
Package capability implements narrow, expiring, optionally signed authorization tokens.
Package capability implements narrow, expiring, optionally signed authorization tokens.
cmd
shiftlock command
Command shiftlock is the unified ShiftLock operator CLI.
Command shiftlock is the unified ShiftLock operator CLI.
shiftlock-agent command
Command shiftlock-agent is a local control-plane agent skeleton.
Command shiftlock-agent is a local control-plane agent skeleton.
shiftlock-inspect command
Thin alias: prefer `shiftlock` for new workflows.
Thin alias: prefer `shiftlock` for new workflows.
shiftlock-loadtest command
Command shiftlock-loadtest is a DESTRUCTIVE load generator for ShiftLock backends.
Command shiftlock-loadtest is a DESTRUCTIVE load generator for ShiftLock backends.
Package configlock protects runtime configuration with signed bundles and an explicit draft→…→active lifecycle.
Package configlock protects runtime configuration with signed bundles and an explicit draft→…→active lifecycle.
control
command
Package command registers and invokes audited, rate-limited control commands.
Package command registers and invokes audited, rate-limited control commands.
execguard
Package execguard runs only allowlisted absolute executables with exact argument patterns.
Package execguard runs only allowlisted absolute executables with exact argument patterns.
lockdown
Package lockdown provides fail-closed service lockdown stronger than maintenance.
Package lockdown provides fail-closed service lockdown stronger than maintenance.
maintenance
Package maintenance manages scoped, durable, auto-expiring maintenance windows.
Package maintenance manages scoped, durable, auto-expiring maintenance windows.
snapshot
Package snapshot creates sanitized runtime snapshots for inspection and diff.
Package snapshot creates sanitized runtime snapshots for inspection and diff.
Package election provides leader election built on claim + fencing semantics.
Package election provides leader election built on claim + fencing semantics.
examples
capability-authorization command
Example capability-authorization issues, verifies, delegates, and rejects widening.
Example capability-authorization issues, verifies, delegates, and rejects widening.
developer-tool command
Command developer-tool demonstrates local-first ShiftLock usage with a filesystem resource and in-memory coordination (no cloud required).
Command developer-tool demonstrates local-first ShiftLock usage with a filesystem resource and in-memory coordination (no cloud required).
ecommerce-platform command
Command ecommerce-platform stubs an e-commerce resource fabric using in-memory adapters (queue, cache, HTTP-shaped services) and a small workflow.
Command ecommerce-platform stubs an e-commerce resource fabric using in-memory adapters (queue, cache, HTTP-shaped services) and a small workflow.
edge-sync-agent command
Command edge-sync-agent stubs offline buffering and reconnect sync using the sync package memory stores and conflict policies.
Command edge-sync-agent stubs offline buffering and reconnect sync using the sync package memory stores and conflict policies.
emergency-lockdown command
Example emergency-lockdown demonstrates fail-closed lockdown enter/unlock rules.
Example emergency-lockdown demonstrates fail-closed lockdown enter/unlock rules.
infrastructure-orchestrator command
Command infrastructure-orchestrator runs a deployment-style workflow across fake resources (config, workers, object store) with parallel fan-out and compensation on failure.
Command infrastructure-orchestrator runs a deployment-style workflow across fake resources (config, workers, object store) with parallel fan-out and compensation on failure.
kubernetes-worker command
Command kubernetes-worker demonstrates the Kubernetes Lease backend with an in-memory LeaseClient stand-in.
Command kubernetes-worker demonstrates the Kubernetes Lease backend with an in-memory LeaseClient stand-in.
leader-election command
Example leader-election demonstrates election Join using a memory fenced lock adapter.
Example leader-election demonstrates election Join using a memory fenced lock adapter.
maintenance-mode command
Example maintenance-mode enters and exits a durable maintenance window.
Example maintenance-mode enters and exits a durable maintenance window.
media-processing-pipeline command
Command media-processing-pipeline demos semaphore + budget + memory queue coordination for a bounded media job pipeline.
Command media-processing-pipeline demos semaphore + budget + memory queue coordination for a bounded media job pipeline.
object-store-sync command
Command object-store-sync demos syncing record metadata through memory object stores (S3-shaped Put/Get without cloud SDKs).
Command object-store-sync demos syncing record metadata through memory object stores (S3-shaped Put/Get without cloud SDKs).
order-platform command
Command order-platform demonstrates stale-write rejection during rolling handoff.
Command order-platform demonstrates stale-write rejection during rolling handoff.
quorum-deployment command
Example quorum-deployment waits for a quorum barrier before proceeding.
Example quorum-deployment waits for a quorum barrier before proceeding.
rolling-handoff command
Command rolling-handoff demonstrates Drain → Transfer → Commit between two generations.
Command rolling-handoff demonstrates Drain → Transfer → Commit between two generations.
secure-control-plane command
Example secure-control-plane demonstrates a multi-step Phase 6 control plane: ownership, supervised worker, signed config, capability auth, forged-cap lockdown, audit verify, snapshot diff, and unlock recovery.
Example secure-control-plane demonstrates a multi-step Phase 6 control plane: ownership, supervised worker, signed config, capability auth, forged-cap lockdown, audit verify, snapshot diff, and unlock recovery.
singleton-worker command
Command singleton-worker demonstrates exclusive ownership of a named claim using the in-memory backend.
Command singleton-worker demonstrates exclusive ownership of a named claim using the in-memory backend.
Package failover provides primary/standby resource groups with manual and health-based failover policy skeletons.
Package failover provides primary/standby resource groups with manual and health-based failover policy skeletons.
fencing
Package guard provides a deterministic authorization policy engine.
Package guard provides a deterministic authorization policy engine.
Package health provides an extended health status graph for Runtime.
Package health provides an extended health status graph for Runtime.
Package identity documents ShiftLock instance identity providers.
Package identity documents ShiftLock instance identity providers.
aws
pod
integration
grpcserver
Package grpcserver integrates ShiftLock ownership with gRPC unary interceptors.
Package grpcserver integrates ShiftLock ownership with gRPC unary interceptors.
httpserver
Package httpserver integrates ShiftLock ownership with HTTP leadership gates.
Package httpserver integrates ShiftLock ownership with HTTP leadership gates.
kafka
Package kafka integrates ShiftLock ownership with Kafka consumer groups.
Package kafka integrates ShiftLock ownership with Kafka consumer groups.
nats
Package nats integrates ShiftLock ownership with NATS queue groups.
Package nats integrates ShiftLock ownership with NATS queue groups.
rabbitmq
Package rabbitmq integrates ShiftLock ownership with RabbitMQ consumers.
Package rabbitmq integrates ShiftLock ownership with RabbitMQ consumers.
scheduler
Package scheduler integrates ShiftLock ownership with singleton schedulers.
Package scheduler integrates ShiftLock ownership with singleton schedulers.
sqs
Package sqs integrates ShiftLock ownership with AWS SQS consumers.
Package sqs integrates ShiftLock ownership with AWS SQS consumers.
internal
Package migration coordinates multi-resource data migration lifecycles.
Package migration coordinates multi-resource data migration lifecycles.
dualwrite
Package dualwrite provides an app-supplied dual-write migration helper.
Package dualwrite provides an app-supplied dual-write migration helper.
observe
Package promotion provides a skeleton for environment promotion workflows (e.g.
Package promotion provides a skeleton for environment promotion workflows (e.g.
Package reconcile provides bounded reconciliation controllers.
Package reconcile provides bounded reconciliation controllers.
recovery
playbook
Package playbook provides versioned recovery playbooks with validate and dry-run.
Package playbook provides versioned recovery playbooks with validate and dry-run.
Package resource provides ShiftLock's resource fabric foundation: typed resource identities, capability declarations, a Runtime-owned registry, dependency graphs, bundles, health, and monotonic resource epochs.
Package resource provides ShiftLock's resource fabric foundation: typed resource identities, capability declarations, a Runtime-owned registry, dependency graphs, bundles, health, and monotonic resource epochs.
cache
Package cache provides shared helpers for cache resource adapters.
Package cache provides shared helpers for cache resource adapters.
cache/memory
Package memory provides an in-process cache resource for tests and demos.
Package memory provides an in-process cache resource for tests and demos.
cache/redis
Package redis is a thin cache resource adapter.
Package redis is a thin cache resource adapter.
database/postgres
Package postgres is a thin database resource adapter.
Package postgres is a thin database resource adapter.
memory
Package memory provides in-process resource adapters for tests and local-first demos.
Package memory provides in-process resource adapters for tests and local-first demos.
queue
Package queue provides a generic queue resource plus an in-memory adapter for demos and tests.
Package queue provides a generic queue resource plus an in-memory adapter for demos and tests.
ratelimit
Package ratelimit provides rate-limit resources (token bucket and concurrency).
Package ratelimit provides rate-limit resources (token bucket and concurrency).
service/http
Package httpresource provides HTTP service resource guardrails.
Package httpresource provides HTTP service resource guardrails.
storage/filesystem
Package filesystem provides a hardened directory resource adapter.
Package filesystem provides a hardened directory resource adapter.
storage/object
Package object defines an object-storage resource abstraction.
Package object defines an object-storage resource abstraction.
Package secrets provides opaque secret references and redaction helpers.
Package secrets provides opaque secret references and redaction helpers.
security
antireplay
Package antireplay provides a bounded request-ID / nonce cache with expiry.
Package antireplay provides a bounded request-ID / nonce cache with expiry.
attestation
Package attestation describes runtime identity evidence and trust levels.
Package attestation describes runtime identity evidence and trust levels.
redteam
Package redteam provides scenario definitions and runnable prevention tests for ShiftLock security controls.
Package redteam provides scenario definitions and runnable prevention tests for ShiftLock security controls.
scanner
Package scanner finds unsafe ShiftLock configurations and emits findings in text, JSON, or SARIF form.
Package scanner finds unsafe ShiftLock configurations and emits findings in text, JSON, or SARIF form.
signing
Package signing provides Ed25519 helpers for ShiftLock high-value records.
Package signing provides Ed25519 helpers for ShiftLock high-value records.
Package stress holds longer-running stress helpers.
Package stress holds longer-running stress helpers.
Package supervise runs ownership-aware tasks with bounded restart policies.
Package supervise runs ownership-aware tasks with bounded restart policies.
Package sync provides source/target synchronization with conflict policies.
Package sync provides source/target synchronization with conflict policies.
Package syncprim provides Semaphore and Once built on fenced claim primitives.
Package syncprim provides Semaphore and Once built on fenced claim primitives.
Package workflow provides ShiftLock's workflow foundation: definitions, step/compensate graphs, durable checkpoints, dry-run, and Runtime-soft integration with guard/audit/lockdown without importing the root module.
Package workflow provides ShiftLock's workflow foundation: definitions, step/compensate graphs, durable checkpoints, dry-run, and Runtime-soft integration with guard/audit/lockdown without importing the root module.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL