redlease

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 11, 2026 License: MIT Imports: 12 Imported by: 0

README

redlease

CI Go Reference

Lease-based leader election on Redis, with fencing tokens.

go get github.com/nijatsdev/redlease

v1 is stable: the API follows semantic versioning, so no breaking changes within v1.x.

redlease elects one long-lived leader among instances and keeps it elected — for a singleton background job, a scheduler, a cron-like task, or the single writer in a one-writer-many-readers system. It manages the whole leadership lifecycle: acquire, renew, step down, release, and fail over.

It is not a general-purpose mutex. The litmus test:

Is the lock held for the duration of a role or the duration of an operation? Role (be the leader, own the schedule, be the one writer) → redlease. Operation (guard this critical section, update this counter safely) → a distributed mutex.

One instance holds a Redis lock with a TTL and runs your work while it is leader. If it cannot renew the lock, it steps down so another instance takes over. What redlease adds on top:

Every leadership term is assigned a strictly increasing fencing token, and writes routed through the Fence* helpers reject any token below the newest elected term or applied write. A paused, GC-stalled, or clock-skewed leader that still believes it holds the lock cannot overwrite newer state — its writes are refused at Redis from the moment a successor is elected, even before that successor writes anything.


Why fencing

Lease-based leader election has an unavoidable window. The lock has a TTL; if the leader pauses (GC, CPU starvation) or is partitioned, the lock can expire and a second instance can be elected while the first still thinks it is leader. For a few seconds, two leaders exist. This is inherent to any lease-based lock — shortening the TTL only shrinks the window, it never closes it.

Fencing makes that window safe. Each term gets a token; the protected resource only accepts writes whose token is at least the highest it has already seen. The stale leader's writes carry an old token and are rejected. This is the mitigation Martin Kleppmann describes in How to do distributed locking.

Do you actually need it?

Fencing matters only when a stale leader's write to shared state would be harmful:

Your leader… Need fencing?
writes a value that must not regress (sequence number, counter, monotonic state) Yes
does no writes (runs a cron, sends notifications) No — at most you want idempotency/dedup
writes self-healing last-writer-wins state that the next correct write repairs No — a plain lock is enough

If you are in the "No" rows, a simpler lock will do. redlease is for the first row.


Usage

rc := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})

e, err := redlease.New(rc, redlease.Config{Name: "report-builder", TTL: 5 * time.Second})
if err != nil {
    log.Fatal(err)
}

// Run blocks until ctx is cancelled. The callback runs only while leader; its
// context is cancelled the instant leadership is lost. The Fencer carries this
// term's fencing token — use it for every fenced write, or pass it down to the
// code that writes shared state.
e.Run(ctx, func(leaderCtx context.Context, f redlease.Fencer) {
    applied, err := f.HSet(leaderCtx, "jobs:report", "status", "running")
    if err != nil {
        // Redis error.
    }
    if !applied {
        // A newer leader has taken over; this term is stale. Stop working.
        return
    }
})

All instances that should contend for the same leadership must share the same Config.Name.

Fenced writes

A Fencer binds the term's token to the elector that minted it, so you carry one value instead of threading a token and a client separately. Each fenced write checks the token and performs the write atomically in a single Lua script, so there is no window in which the token could go stale between the check and the write. All methods share one high-water mark, so a token advanced by any of them fences every later, lower-token write:

f.HSet(ctx, hashKey, field, value) // fenced HSET
f.Set(ctx, key, value)             // fenced SET

// Escape hatch for any other Redis write (ZADD, XADD, multi-key, ...).
// KEYS[1]/ARGV[1] are reserved for the fence; address yours from index 2.
f.Eval(ctx,
    "redis.call('zadd', KEYS[2], ARGV[2], ARGV[3])",
    []string{"board"}, "100", "alice")

All three return (applied bool, err error): applied == false means your token is stale — a newer leader has taken over — and you should stop writing. f.Token() returns the raw token for fencing a resource that isn't Redis (see below).

The same writes are available on the elector itself — e.FenceHSet(ctx, token, ...), e.FenceSet, e.FenceEval — taking the token explicitly. Use those in the token-driven style, where you hold a token from e.Token() rather than a Fencer from the callback.

Observability

redlease emits no logs of its own — a library should not impose a log format, level, or destination on its caller. Instead it exposes leadership transitions through an optional Observer; wire them to your own logger, metrics, or tracing. Every field is optional and a nil one is simply not called.

e, _ := redlease.New(rc, redlease.Config{
    Name: "report-builder",
    Observer: redlease.Observer{
        OnElected:     func(token int64) { slog.Info("leader elected", "fence", token) },
        OnSteppedDown: func()            { slog.Info("leader stepped down") },
        OnFollower:    func()            { slog.Info("running as follower") },
        OnError:       func(err error)   { slog.Warn("election redis error", "err", err) },
    },
})

OnFollower fires when an acquire attempt finds the lock held by another instance — once per transition into the follower role, not on every retry. It lets a follower learn its initial role at startup without waiting to win.

Role transitions drive behavior. Transient Redis errors during acquire, renewal, or release are handled internally — retried, backed off, or stepped down from — and the consequence the caller cares about, losing leadership, surfaces through OnSteppedDown. OnError additionally exposes those handled errors for logs and metrics, so you can see trouble (a flaky network, a slow Redis) before it costs leadership; it never requires action. Callbacks run on the Run goroutine and must not block.

Checking leadership outside the callback

The Fencer reaches your LeaderFunc directly, but sometimes another goroutine — an HTTP handler, say — needs to act as the leader too. Fencer(), Token(), and IsLeader() expose the current state from any goroutine:

if f, ok := e.Fencer(); ok {
    // We are the leader. The fenced write is safe even if leadership changes
    // right now: a stale token is rejected at write time.
    f.HSet(ctx, "state", "key", "value")
}

Prefer Token() for anything that writes. IsLeader() exists for display/metrics, but it is advisory — leadership can be lost the instant after it returns, so never gate a correctness-sensitive write on if e.IsLeader() { write() }. That is the split-brain race fencing exists to prevent; carry the token through a Fence* helper instead.

If all your leader work happens this way — outside the callback — pass nil for the LeaderFunc. Run then just keeps the instance elected (acquire, renew, release) while your other goroutines act via Token():

go e.Run(ctx, nil) // hold leadership; do the work elsewhere via e.Token()

The callback style and this token-driven style are interchangeable — pick whichever fits your app.

A term ends when leadership is lost, the LeaderFunc returns, ctx is cancelled, or you call e.Resign(). Resign ends the current term voluntarily — the lock is released and Run re-contends — which is the step-down lever for the token-driven style. Note that returning from a LeaderFunc also ends the term: a leader that wants to stay elected must block until its context is cancelled.


How it works

  • Acquire — a single Lua script does SET name:leader <id> NX PX <ttl-ms> and, on success, INCR name:fence to mint the token and stamps it into name:fence:applied. Doing it all atomically guarantees every term's token is strictly greater than any prior term's. A lock left over from a previous term of the same instance (a release that never reached Redis, a restart with a fixed InstanceID) is taken over immediately — with a refreshed TTL and a fresh token, since it is a new term — instead of waiting out the TTL. The TTL is set in milliseconds so sub-second values are honored exactly. ⚠️ The takeover is why an explicit InstanceID must be unique among live processes: replicas sharing an ID seize the lock from each other and both run as leader, silently. Derive it from something unique per replica (pod/host name), never the service name.
  • Contend — followers poll on AcquireInterval with slight downward jitter, so contenders that started together spread out instead of hitting Redis in the same instant. When the acquire round trip errors (Redis unreachable) the retry delay doubles per consecutive error, capped at the TTL, and resets once Redis answers again; losing the race to another instance never backs off, so failover speed is unaffected.
  • Hold — the leader renews the lock on RenewInterval via an ownership-checked script (it only extends a lock whose value is still its own id). A renewal that returns 0 means the lock was lost; a transient Redis error is tolerated until TTL would have lapsed, then the leader steps down.
  • Release — on graceful step-down the leader deletes the lock (ownership-checked, so it never deletes a successor's lock), letting the next instance take over without waiting for the TTL.
  • Fence — each fenced write runs a Lua script that checks the token against the high-water mark in name:fence:applied, performing the write only when the token is current. The mark advances on every applied write and on every election, so a deposed leader's tokens are dead from the moment a successor is elected.

Fencing writes that don't go to Redis

Fencing is not a Redis concept — it applies to any shared resource a stale leader could corrupt (a Postgres row, an S3 object). The catch is that the fence must be enforced at the resource itself, atomically with the write, because that is the only place the check and the write can happen together.

This package enforces the fence for Redis writes, because Redis is the resource it can reach into (via Lua). If your leader writes elsewhere, redlease still gives you the universal half — the monotonic token — but you must enforce it at your resource. For example, in Postgres:

UPDATE state SET value = $1, fence = $2 WHERE key = $3 AND fence <= $2;
-- rows affected == 0  ->  your token was stale; you were fenced out

So: use the Fence* helpers when you write to Redis; use the token with a conditional write (a WHERE fence <= token, a compare-and-swap, an If-Match precondition) when you write anywhere else.


Correctness boundary

Read this before using it for anything that matters.

The fencing token is generated and stored in Redis. On a single Redis instance, this gives a strict, monotonic guarantee: tokens never go backward, and the fence is sound.

That strictness assumes the fence keys survive, which puts two requirements on the server:

  • Persistence. A crash-restart recovers whatever the persistence layer kept. Default RDB snapshotting loses recent writes, so name:fence can come back lower — or missing, restarting tokens at 1 — and silently invert the fence against a still-live older term. AOF narrows the window (appendfsync everysec to ~1s); only appendfsync always makes the counter truly durable.
  • Eviction. The fence keys have no TTL, but under maxmemory-policy allkeys-lru/lfu/random they are eviction candidates like any key, and eviction resets the counter mid-flight. Run noeviction (or a volatile-* policy, which never evicts keys without a TTL).

On a replicated Redis deployment (Sentinel or Cluster), Redis replication is asynchronous. A primary can acknowledge the acquire — and the token INCR — before it has propagated to a replica, and a failover to that replica can lose it. In that window the monotonicity the fence depends on can be violated, and two leaders could in principle obtain non-ordered tokens. This is the same limitation that affects every Redis-based lock, fencing or not.

On Redis Cluster there is also a separate, non-negotiable requirement: the lock, fence, and applied keys all derive from Config.Name, and a single Lua script touches more than one of them, so they must hash to the same slot. Wrap Name in a hash tag — e.g. "{report-builder}" — so every derived key shares it. Without one they scatter across slots and the acquire script fails with CROSSSLOT — retried forever at the backoff cadence and surfaced only through Observer.OnError, so wire OnError up before first deploying against Cluster.

The same constraint extends to fenced writes: each Fence* call runs one script against name:fence:applied and your target keys, so every key you write through the fence must carry the same hash tag — on Cluster, your fenced application state has to live in the elector's slot. If pinning your data to one slot doesn't fit, enforce the fence at the resource yourself with the raw token (f.Token()), exactly as you would for a non-Redis store (see above).

So redlease is the right tool when:

  • you run Redis single-instance, or
  • a brief, rare token regression on Redis failover is acceptable for your workload.

If you need a fencing guarantee that survives failover, source the token from a linearizable store and apply it against your resource. redlease deliberately does not pretend Redis can provide that.


License

MIT

Documentation

Overview

Package redlease implements lease-based leader election on Redis with fencing tokens.

One instance holds a Redis lock with a TTL and runs the caller's work while it is leader; if it cannot renew, it steps down so another instance takes over. Unlike a plain Redis lock, every leadership term is assigned a strictly increasing fencing token. Writes routed through the Elector's Fence* helpers are rejected when they carry a token below the newest elected term or applied write — electing a term fences out its predecessors immediately — so a paused or clock-skewed leader that still believes it holds the lock cannot overwrite newer state.

When you need this

Fencing matters only when a stale leader's write to shared state would be harmful — for example a value that must not regress, a sequence number, or a counter. If the leader does no writes, or writes self-healing last-writer-wins state, a plain lock is enough and you do not need this package.

Correctness boundary

The fence token is generated and stored in Redis. On a single Redis instance this gives a strict, monotonic guarantee — provided the fence keys survive: crash-restart durability requires AOF appendfsync always (RDB and AOF everysec lose recent writes), and eviction must not touch the keys (run noeviction or a volatile-* policy; the fence keys carry no TTL). On a replicated deployment (Sentinel or Cluster), Redis replication is asynchronous: a primary can acknowledge the acquire — and the token increment — before it reaches a replica, and a failover to that replica can lose them. In that window the monotonicity the fence relies on can be violated. For strict correctness across failover, source the fencing token from a linearizable store (etcd, ZooKeeper) instead of Redis. This package is the right tool when you run Redis single-instance, or when a brief, rare token regression on failover is acceptable.

On Redis Cluster, also wrap Config.Name in a hash tag (e.g. "{name}") so the lock, fence, and applied keys hash to the same slot; otherwise the acquire script fails with CROSSSLOT. The same constraint extends to fenced writes: each Fence* script touches the applied key and your target keys in one script, so every key written through the fence must carry the same hash tag as Name — on Cluster, fenced application state must live in the elector's slot. If that does not fit your data layout, enforce the fence at the resource yourself using the raw token (see Elector.Token).

Example

This example runs a periodic job on exactly one instance at a time. Several instances call Run with the same Config.Name; only the elected leader executes the job. Each write the leader makes carries its leadership term's fencing token, so a stale leader that has not yet noticed it lost the lock cannot overwrite a newer leader's state.

package main

import (
	"context"
	"log"
	"time"

	goredis "github.com/redis/go-redis/v9"

	"github.com/nijatsdev/redlease"
)

func main() {
	rc := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})

	e, err := redlease.New(rc, redlease.Config{
		Name: "report-builder",
		TTL:  5 * time.Second,
	})
	if err != nil {
		log.Fatal(err)
	}

	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Run blocks until ctx is cancelled. The callback runs only while this
	// instance is the leader; its context is cancelled the moment leadership
	// is lost. The Fencer carries this term's fencing token — use it for every
	// write to shared state.
	e.Run(ctx, func(leaderCtx context.Context, f redlease.Fencer) {
		ticker := time.NewTicker(time.Second)
		defer ticker.Stop()

		for {
			select {
			case <-leaderCtx.Done():
				return
			case <-ticker.C:
				// Record progress under the fencing token. If a newer leader
				// has taken over, applied is false: this term is stale and
				// must stop working.
				applied, err := f.HSet(leaderCtx, "jobs:report", "status", "running")
				if err != nil {
					log.Printf("write failed: %v", err)
					continue
				}

				if !applied {
					log.Printf("fenced out by a newer leader; stopping")
					return
				}
			}
		}
	})
}
Example (DatabaseFencing)

This example fences a write to a resource other than Redis — here a SQL database. redlease enforces the fence for you only on Redis writes; for any other store it gives you the term's fencing token and you enforce it at the resource, atomically with your write. In SQL that means a conditional UPDATE that only applies when the row's stored fence is not newer than your token.

The table is assumed to hold a fence column alongside the value:

CREATE TABLE state (id text PRIMARY KEY, value text, fence bigint NOT NULL DEFAULT 0);

sql.Open needs a registered driver; a real program imports one, e.g.

import _ "github.com/jackc/pgx/v5/stdlib" // database/sql driver "pgx"

The LeaderFunc here performs one write and returns, which ends the term (see LeaderFunc); a leader with ongoing work loops until leaderCtx is done, as in the package example above.

package main

import (
	"context"
	"database/sql"
	"log"
	"time"

	goredis "github.com/redis/go-redis/v9"

	"github.com/nijatsdev/redlease"
)

func main() {
	rc := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})

	db, err := sql.Open("pgx", "postgres://localhost/app")
	if err != nil {
		log.Fatal(err)
	}

	defer func() { _ = db.Close() }()

	e, err := redlease.New(rc, redlease.Config{Name: "report-builder", TTL: 5 * time.Second})
	if err != nil {
		log.Print(err)
		return
	}

	ctx := context.Background()

	e.Run(ctx, func(leaderCtx context.Context, f redlease.Fencer) {
		// Take the term's token from the Fencer and enforce it yourself in SQL.
		// The UPDATE applies only when our token is at least the stored fence,
		// then advances it; a stale leader's lower token matches no row.
		const q = `UPDATE state
		             SET value = $1, fence = $2
		           WHERE id = $3 AND fence <= $2`

		res, err := db.ExecContext(leaderCtx, q, "running", f.Token(), "report")
		if err != nil {
			log.Printf("write failed: %v", err)
			return
		}

		n, err := res.RowsAffected()
		if err != nil {
			log.Printf("rows affected: %v", err)
			return
		}

		if n == 0 {
			// No row updated: a newer leader has advanced the fence past our
			// token — we are stale and must stop working. (Zero rows is also
			// what a missing row yields; this example assumes the row is
			// seeded. An INSERT ... ON CONFLICT upsert with the same fence
			// condition covers the first write too.)
			log.Printf("fenced out by a newer leader; stopping")
			return
		}
	})
}

Index

Examples

Constants

View Source
const DefaultTTL = 5 * time.Second

DefaultTTL is the lock TTL used when Config.TTL is unset. Override via Config.

Variables

View Source
var ErrTooManyEvalBodies = errors.New("redlease: too many distinct FenceEval bodies; keep bodies constant and pass variable data through writeKeys and args")

ErrTooManyEvalBodies is returned (wrapped) by Elector.FenceEval once more than maxEvalScripts distinct Lua bodies have been used. It signals a caller bug — variable data interpolated into the body — not a transient Redis error: check for it with errors.Is, fix the body to be constant (pass variable data through writeKeys and args), and do not retry.

Functions

This section is empty.

Types

type Config

type Config struct {
	// Name identifies the lock. All instances contending for the same leadership
	// must use the same Name; different Names are independent locks.
	//
	// The lock, fence, and applied keys all derive from Name, and a single Lua
	// script touches more than one of them. On Redis Cluster they must therefore
	// hash to the same slot: wrap Name in a hash tag, e.g. "{report-builder}",
	// so all derived keys share it. Without one they scatter across slots and the
	// acquire script fails with CROSSSLOT. Keys written through the Fence*
	// helpers must carry the same hash tag too; see the package documentation.
	Name string

	// TTL is how long the lock lives without renewal. A leader that cannot renew
	// within TTL loses leadership. Shorter TTL means faster failover but less
	// tolerance for pauses. Must be at least 100ms so a renewal can complete
	// before expiry. Defaults to DefaultTTL.
	//
	// Redis applies the TTL at millisecond granularity (PX), so a fractional
	// remainder is truncated.
	TTL time.Duration

	// RenewInterval is how often the leader renews the lock. Must be at most
	// TTL/2, so at least two renewal attempts fit before expiry, and at least
	// 1ms (it doubles as each renewal's timeout). Note that at exactly TTL/2
	// the retry after a dropped renewal lands right at expiry; keep it strictly
	// below TTL/2 (the TTL/3 default) for real retry headroom.
	RenewInterval time.Duration

	// AcquireInterval is how often a follower retries acquiring the lock. Must be
	// less than TTL, so a follower polls at least once per lock lifetime, and at
	// least 1ms, like RenewInterval. Defaults to TTL/2.
	//
	// The interval governs the ordinary case of losing the race to another
	// instance. When the acquire round trip errors instead (Redis unreachable),
	// the retry delay doubles per consecutive error, capped at TTL, and resets
	// once Redis answers again.
	AcquireInterval time.Duration

	// InstanceID uniquely identifies this instance; it is the lock's value and
	// guards renewal and release so an instance never affects another's lock. If
	// empty, a hostname + random suffix is generated.
	//
	// The ID must be unique among live processes, not merely stable: a lock
	// found holding this instance's own ID is treated as its leftover and taken
	// over, so two live replicas sharing an ID seize the lock from each other
	// and both run as leader — permanently and silently. Derive a fixed ID from
	// something unique per replica (pod or host name), never a value shared
	// across replicas like the service name.
	InstanceID string

	// Observer receives lifecycle events. All fields are optional; the library
	// emits no output of its own, so callers wire these to their own logger,
	// metrics, or tracing as they see fit.
	Observer Observer
}

Config configures an Elector. Only Name is required (the client is passed separately to New); zero-valued timing fields fall back to the Default* constants.

type Elector

type Elector struct {
	// contains filtered or unexported fields
}

Elector runs leader election for a single lock and mints fencing tokens.

func New

func New(client Redis, cfg Config) (*Elector, error)

New returns an Elector from cfg. It returns an error if required fields are missing or timing parameters are inconsistent: TTL below 100ms, RenewInterval greater than TTL/2, AcquireInterval not less than TTL, or either interval below 1ms.

func (*Elector) FenceEval

func (e *Elector) FenceEval(ctx context.Context, token int64, body string, writeKeys []string, args ...any) (applied bool, err error)

FenceEval fences an arbitrary Redis write supplied as a Lua body, for writes the typed helpers do not cover (ZADD, XADD, multi-key updates, and so on). The body runs atomically only when token is current; the fence returns 1 when applied and 0 when fenced out.

The body must not contain its own return statement: FenceEval appends the fence's epilogue, which advances the token high-water mark and returns 1. A return inside body skips that epilogue, so the result reports applied == false and the mark does not advance even on a write that ran. Just perform the write and let FenceEval supply the return.

Within body, the protected token check is already done. Address your own keys and arguments starting at index 2 — KEYS[1] and ARGV[1] are reserved for the fence (the high-water key and the token). Pass your keys in writeKeys and your arguments in args; they appear as KEYS[2..] and ARGV[2..] in that order. On Redis Cluster, every key in writeKeys must share the elector's hash slot; see Elector.FenceHSet.

Each distinct body compiles once and is cached on the Elector for EVALSHA reuse, capped at 256 bodies; past the cap FenceEval returns an error wrapping ErrTooManyEvalBodies. Keep bodies constant and pass variable data through writeKeys and args — interpolating values into the body creates a new cache entry (and a new script for Redis) per call.

Example — fence a ZADD:

applied, err := e.FenceEval(ctx, token,
    "redis.call('zadd', KEYS[2], ARGV[2], ARGV[3])",
    []string{"myset"}, "1.0", "member")

func (*Elector) FenceHSet

func (e *Elector) FenceHSet(ctx context.Context, token int64, hashKey, field, value string) (applied bool, err error)

FenceHSet performs a fenced HSET of field=value into hashKey: the write is applied only if token is at least the fence high-water mark — the highest token minted by an election or carried by an applied fenced write. It returns true when applied and false when token is stale — a newer term has been elected or has written — so the caller can stop emitting derived events. token must be the value passed to the LeaderFunc for the current term. A non-nil error means the write could not be attempted.

On Redis Cluster, hashKey must hash to the same slot as the elector's keys: the check and the write run in one script against the applied key and hashKey together, so give hashKey the same hash tag as Config.Name or the call fails with CROSSSLOT. The same holds for every Fence* helper's keys.

func (*Elector) FenceSet

func (e *Elector) FenceSet(ctx context.Context, token int64, key, value string) (applied bool, err error)

FenceSet performs a fenced SET of key=value, applied only if token is current. Semantics match Elector.FenceHSet.

func (*Elector) Fencer added in v0.3.0

func (e *Elector) Fencer() (f Fencer, ok bool)

Fencer returns a Fencer bound to the current leadership term and true while this instance is the leader, or an inert Fencer and false otherwise — inert meaning it carries token 0, the "not leader" sentinel every fenced write rejects, so even a caller that ignores ok cannot slip a write through. It is the token-driven-style counterpart to the Fencer a LeaderFunc receives: code outside the callback can take a Fencer and pass it down to its writers.

Like Elector.Token, the returned Fencer is a snapshot; leadership can change immediately after. That is safe because the fence is enforced at write time — a stale token is rejected by the Fencer's methods.

func (*Elector) InstanceID

func (e *Elector) InstanceID() string

InstanceID returns this elector's instance identity.

func (*Elector) IsLeader

func (e *Elector) IsLeader() bool

IsLeader reports whether this instance currently holds leadership. It is advisory only: leadership can be lost the instant after it returns, so never gate a correctness-sensitive write on it. For writes, carry the token from Elector.Token through a Fence* helper, which rejects a stale token at write time.

func (*Elector) Resign added in v1.0.0

func (e *Elector) Resign()

Resign voluntarily ends the current leadership term, if any: the LeaderFunc's context is cancelled, the lock is released, and Run re-contends after AcquireInterval — so this instance may well win again unless another takes the lock first. It is a no-op when this instance is not leading, and safe to call from any goroutine. It is the step-down lever for the token-driven style (Run with a nil LeaderFunc), which otherwise could stop leading only by cancelling Run's context entirely.

func (*Elector) Run

func (e *Elector) Run(ctx context.Context, fn LeaderFunc)

Run contends for leadership until ctx is cancelled. Each time this instance wins, it invokes fn (a LeaderFunc) with a Fencer for the term, then steps down and re-contends when the term ends — leadership lost, fn returned, Elector.Resign called, or ctx cancelled. Run blocks until ctx is cancelled and fn (if running) has returned; on the final shutdown step-down the OnSteppedDown observer still fires.

fn may be nil. Then Run just keeps this instance elected — acquiring, renewing, and releasing the lock — while the caller does its leader work from another goroutine, gated on Elector.Token. This is the token-driven style; the callback style and this one are interchangeable, pick whichever fits.

Run must not be called more than once concurrently on the same Elector; the two calls would corrupt shared leadership state. A concurrent call panics. Use one Elector per Run. Sequential calls (a new Run after a previous one returned) are fine.

func (*Elector) Token

func (e *Elector) Token() (token int64, ok bool)

Token returns the current leadership term's fencing token and true while this instance is the leader, or 0 and false otherwise. It is safe to call from any goroutine, so code outside the LeaderFunc can perform fenced writes:

if token, ok := e.Token(); ok {
    e.FenceHSet(ctx, token, "state", "key", "value")
}

The returned token is a snapshot; leadership can change immediately after. That is safe because the fence is enforced at write time — a stale token is rejected by the Fence* helpers — so unlike Elector.IsLeader there is no time-of-check-to-time-of-use hazard in acting on it.

type Fencer added in v0.3.0

type Fencer struct {
	// contains filtered or unexported fields
}

Fencer binds a leadership term's fencing token to the Elector that minted it, so a single value carries everything a fenced write needs. It is what a LeaderFunc receives, and what you pass down into the code that writes shared state.

A Fencer is a small value; copy it freely. Its methods are the bound equivalents of the Elector's Fence* helpers: each applies its write only when the bound token is at least the fence high-water mark, and returns applied == false when the token is stale (a newer term has been elected or has written), so the caller can stop. A Fencer is valid only for its term: its writes are fenced out once a successor is elected. Between a voluntary step-down and the next election, writes under the old token may still apply (there is no newer state to protect); treat the term context's cancellation, not a failing write, as the signal to stop working.

func NewFencer added in v0.3.0

func NewFencer(e *Elector, token int64) Fencer

NewFencer binds an explicit token to e, for callers that already hold a token — for example one persisted from a prior leadership term, or supplied in a test. Most code should instead take the Fencer a LeaderFunc receives or call Elector.Fencer. The token is not validated against the elector's current term (the fence is enforced at write time), but it must originate from this package's electors or stay within their magnitude: the fence comparison is exact only up to 2^53, and stamping an arbitrarily large token would permanently fence out every legitimate writer.

e must be non-nil — the Fencer's methods call through it. Likewise a hand-declared zero Fencer{} panics on use; obtain Fencers from the elector.

func (Fencer) Eval added in v0.3.0

func (f Fencer) Eval(ctx context.Context, body string, writeKeys []string, args ...any) (applied bool, err error)

Eval fences an arbitrary Lua write. See Elector.FenceEval.

func (Fencer) HSet added in v0.3.0

func (f Fencer) HSet(ctx context.Context, hashKey, field, value string) (applied bool, err error)

HSet performs a fenced HSET of field=value into hashKey. See Elector.FenceHSet for the full semantics.

func (Fencer) Set added in v0.3.0

func (f Fencer) Set(ctx context.Context, key, value string) (applied bool, err error)

Set performs a fenced SET of key=value. See Elector.FenceSet.

func (Fencer) Token added in v0.3.0

func (f Fencer) Token() int64

Token returns the fencing token this Fencer carries — the value the elector assigned to this leadership term.

type LeaderFunc

type LeaderFunc func(ctx context.Context, f Fencer)

LeaderFunc is the work run while an instance is leader. It receives a context cancelled when leadership is lost or Run's context is cancelled, and a Fencer bound to this leadership term — use it for every fenced write, or pass it down to the code that writes shared state. LeaderFunc must return promptly once its context is cancelled; Run does not release the lock or re-contend until it does.

Returning also ends the term: holding the lock with no work running would only block other instances, so Run releases it and re-contends. A LeaderFunc that wants to stay leader must block until its context is cancelled.

A panic in a LeaderFunc is not recovered: it crashes the process like any other goroutine panic, and the lock is left to expire at its TTL rather than being released, so failover waits out the TTL. Recover inside your LeaderFunc if you want a panicking term to step down gracefully instead.

type Observer

type Observer struct {
	// OnElected fires when this instance wins a leadership term, with that
	// term's fencing token. It fires before the LeaderFunc is invoked.
	OnElected func(token int64)

	// OnSteppedDown fires when this instance loses or relinquishes leadership,
	// after the LeaderFunc has returned.
	OnSteppedDown func()

	// OnFollower fires when this instance is, for now, a follower: an acquire
	// attempt found the lock held by another instance. It fires once per
	// transition into the follower role — on the first lost attempt, and again
	// only after an intervening leadership term — not on every retry. A consumer
	// can use it to learn its initial role at startup without waiting to win.
	OnFollower func()

	// OnError fires when a Redis round trip fails during acquire, renewal, or
	// release. The elector has already handled the failure — retried, backed
	// off, or stepped down — so the callback exists purely for visibility: wire
	// it to your logger or metrics to see trouble before it costs leadership.
	// It is not called for the context cancellation that ends Run.
	//
	// Deterministic failures (a missing Cluster hash tag surfacing as
	// CROSSSLOT, auth errors) are retried forever at the backoff cadence, and
	// OnError is their only signal — wire it up before first deploying.
	OnError func(err error)
}

Observer is a set of optional callbacks invoked on leadership transitions. A nil field is simply not called. Callbacks run on the Run goroutine and must not block — a slow OnElected eats TTL budget before the first renewal and can cost the term. A panicking callback is not recovered (see LeaderFunc); OnElected in particular fires while the lock is held, so its panic leaves the lock to expire at the TTL.

Role transitions are the primary events, mirroring the design of k8s.io/client-go/tools/leaderelection: transient Redis errors during acquire, renewal, and release are handled internally — retried, backed off, or stepped down from — and the consequence the caller cares about, losing leadership, surfaces through OnSteppedDown. OnError additionally exposes those handled errors for logging and metrics; it never requires action.

type Redis

type Redis interface {
	goredis.Scripter
}

Redis is the subset of the go-redis client this package needs: the script runner used by all lock and fence operations. *goredis.Client and *goredis.ClusterClient both satisfy it, as does any compatible wrapper.

Configure the client with sane I/O timeouts: the elector bounds every round trip itself, but a call abandoned past its bound stays blocked inside a timeout-less client, pinning a pooled connection — during a long server hang those accumulate and can starve a shared client.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL