redlease

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 24, 2026 License: MIT Imports: 10 Imported by: 0

README

redlease

CI Go Reference Go Report Card

Lease-based leader election on Redis, with fencing tokens.

go get github.com/nijatsdev/redlease

Pre-1.0: the API may change between v0.x releases.

redlease elects one long-lived leader among instances and keeps it elected — for running a singleton background job, a scheduler, a cron-like task, or the single writer in a one-writer-many-readers system. It manages the whole leadership lifecycle: acquire, renew, step down, release, and fail over.

It is not a general-purpose mutex. For short-lived mutual exclusion around a critical section, use redsync or redislock instead. The litmus test:

Is the lock held for the duration of a role or the duration of an operation? Role (be the leader, own the schedule, be the one writer) → redlease. Operation (guard this critical section, update this counter safely) → a mutex.

One instance holds a Redis lock with a TTL and runs your work while it is leader. If it cannot renew the lock, it steps down so another instance takes over. That much is what every Redis-lock library does. What redlease adds is the part most of them omit:

Every leadership term is assigned a strictly increasing fencing token, and writes routed through the Fence* helpers reject any token older than the latest applied. A paused, GC-stalled, or clock-skewed leader that still believes it holds the lock cannot overwrite newer state — its writes are refused at Redis.

Why fencing

Lock-based leader election has an unavoidable window. The lock has a TTL; if the leader pauses (GC, CPU starvation) or is partitioned, the lock can expire and a second instance can be elected while the first still thinks it is leader. For a few seconds, two leaders exist. This is inherent to any lease-based lock — shortening the TTL only shrinks the window, it never closes it.

Fencing makes that window safe. Each term gets a token; the protected resource only accepts writes whose token is at least the highest it has already seen. The stale leader's writes carry an old token and are rejected. This is the mitigation Martin Kleppmann describes in How to do distributed locking — and the reason a plain Redis lock is not safe for correctness-sensitive writes.

Do you actually need it?

Fencing matters only when a stale leader's write to shared state would be harmful:

Your leader… Need fencing?
writes a value that must not regress (sequence number, counter, monotonic state) Yes
does no writes (runs a cron, sends notifications) No — at most you want idempotency/dedup
writes self-healing last-writer-wins state that the next correct write repairs No — a plain lock is enough

If you are in the "No" rows, you do not need this library; a simpler lock will do. redlease is for the first row.

Usage

rc := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})

e, err := redlease.New(rc, redlease.Config{Name: "report-builder", TTL: 5 * time.Second})
if err != nil {
    log.Fatal(err)
}

// Run blocks until ctx is cancelled. The callback runs only while leader; its
// context is cancelled the instant leadership is lost.
e.Run(ctx, func(leaderCtx context.Context, token int64) {
    applied, err := e.FenceHSet(leaderCtx, token, "jobs:report", "status", "running")
    if err != nil {
        // Redis error.
    }
    if !applied {
        // A newer leader has taken over; this token is stale. Stop working.
        return
    }
})

All instances that should contend for the same leadership must share the same Config.Name. The token passed to your work function is the term's fencing token — stamp every fenced write with it.

Fenced writes

Each fenced write checks the token and performs the write atomically in a single Lua script, so there is no window in which the token could go stale between the check and the write. All helpers share one high-water mark, so a token advanced by any of them fences every later, lower-token write:

e.FenceHSet(ctx, token, hashKey, field, value) // fenced HSET
e.FenceSet(ctx, token, key, value)             // fenced SET

// Escape hatch for any other Redis write (ZADD, XADD, multi-key, ...).
// KEYS[1]/ARGV[1] are reserved for the fence; address yours from index 2.
e.FenceEval(ctx, token,
    "redis.call('zadd', KEYS[2], ARGV[2], ARGV[3])",
    []string{"board"}, "100", "alice")

All three return (applied bool, err error): applied == false means your token is stale — a newer leader has taken over — and you should stop writing.

Observability

redlease emits no logs of its own — a library should not impose a log format, level, or destination on its caller. Instead it exposes leadership transitions through an optional Observer; wire them to your own logger, metrics, or tracing. Both fields are optional and a nil one is simply not called.

e, _ := redlease.New(rc, redlease.Config{
    Name: "report-builder",
    Observer: redlease.Observer{
        OnElected:     func(token int64) { slog.Info("leader elected", "fence", token) },
        OnSteppedDown: func()            { slog.Info("leader stepped down") },
    },
})

Only leadership transitions are reported, following the design of client-go's leaderelection. Transient Redis errors during acquire or renewal are handled internally; the consequence the caller cares about — losing leadership — surfaces through OnSteppedDown. Monitor Redis health through your Redis client, not through the Observer. Callbacks run on the Run goroutine and must not block.

Checking leadership outside the callback

The fencing token reaches your LeaderFunc directly, but sometimes another goroutine — an HTTP handler, say — needs to act as the leader too. Token() and IsLeader() expose the current state from any goroutine:

if token, ok := e.Token(); ok {
    // We are the leader. The fenced write is safe even if leadership changes
    // right now: a stale token is rejected at write time.
    e.FenceHSet(ctx, token, "state", "key", "value")
}

Prefer Token() for anything that writes. IsLeader() exists for display/metrics, but it is advisory — leadership can be lost the instant after it returns, so never gate a correctness-sensitive write on if e.IsLeader() { write() }. That is the split-brain race fencing exists to prevent; carry the token through a Fence* helper instead.

If all your leader work happens this way — outside the callback — pass nil for the LeaderFunc. Run then just keeps the instance elected (acquire, renew, release) while your other goroutines act via Token():

go e.Run(ctx, nil) // hold leadership; do the work elsewhere via e.Token()

The callback style and this token-driven style are interchangeable — pick whichever fits your app.

How it works

  • Acquire — a single Lua script does SET name:leader <id> NX PX <ttl-ms> and, on success, INCR name:fence to mint the token. Doing both atomically guarantees every term's token is strictly greater than any prior term's. The TTL is set in milliseconds so sub-second values are honored exactly.
  • Hold — the leader renews the lock on RenewInterval via an ownership-checked script (it only extends a lock whose value is still its own id). A renewal that returns 0 means the lock was lost; a transient Redis error is tolerated until TTL would have lapsed, then the leader steps down.
  • Release — on graceful step-down the leader deletes the lock (ownership-checked, so it never deletes a successor's lock), letting the next instance take over without waiting for the TTL.
  • Fence — each fenced write runs a Lua script that stores the highest applied token in name:fence:applied and rejects any write carrying a lower token, performing the write only when the token is current.

Fencing writes that don't go to Redis

Fencing is not a Redis concept — it applies to any shared resource a stale leader could corrupt (a Postgres row, an S3 object, a Kafka topic). The catch is that the fence must be enforced at the resource itself, atomically with the write, because that is the only place the check and the write can happen together.

This package enforces the fence for Redis writes, because Redis is the resource it can reach into (via Lua). If your leader writes elsewhere, redlease still gives you the universal half — the monotonic token — but you must enforce it at your resource. For example, in Postgres:

UPDATE state SET value = $1, fence = $2 WHERE key = $3 AND fence <= $2;
-- rows affected == 0  ->  your token was stale; you were fenced out

So: use the Fence* helpers when you write to Redis; use the token with a conditional write (a WHERE fence <= token, a compare-and-swap, an If-Match precondition) when you write anywhere else.

Correctness boundary

Read this before using it for anything that matters.

The fencing token is generated and stored in Redis. On a single Redis instance, this gives a strict, monotonic guarantee: tokens never go backward, and the fence is sound.

On a replicated Redis deployment (Sentinel or Cluster), Redis replication is asynchronous. A primary can acknowledge the acquire — and the token INCR — before it has propagated to a replica, and a failover to that replica can lose it. In that window the monotonicity the fence depends on can be violated, and two leaders could in principle obtain non-ordered tokens. This is the same limitation that affects every Redis-based lock, fencing or not.

So redlease is the right tool when:

  • you run Redis single-instance, or
  • a brief, rare token regression on Redis failover is acceptable for your workload.

If you need a fencing guarantee that survives failover, source the token from a linearizable store — etcd or ZooKeeper — and apply it against your resource. redlease deliberately does not pretend Redis can provide that.

Election, not a mutex

redlease and a distributed mutex (redsync, redislock) solve different problems with different shapes. Reaching for the wrong one is the classic split-brain footgun — holding a mutex across long-running work with no renewal lets the lock expire mid-work.

Mutex (redsync, redislock) Election (redlease)
Mental model grab lock → short critical section → release be leader → run long-lived work → step down
You call Lock() … Unlock() manually Run(ctx, fn) — lifecycle managed for you
Lock held for a brief operation the whole leadership term (often hours)
Renewal you add it yourself, or the lock expires built in
Loser of the race blocks / retries to acquire now becomes a follower, re-contends in the background
Fencing none core feature

If your lock guards an operation, use a mutex. If it confers a role, use redlease.

When to use something else

You want… Use
election + fencing, Redis you already run, single-instance or tolerant of rare failover regressions redlease
a plain distributed mutex for short critical sections go-redsync/redsync
consensus-grade election and you run on Kubernetes k8s.io/client-go/tools/leaderelection (etcd-backed Lease)
consensus-grade election off Kubernetes etcd concurrency.Election
leadership as part of your own replicated state machine a Raft library (hashicorp/raft, etcd/raft)

redlease fills one specific gap: the Redis-lock libraries that don't fence, for users whose leader writes shared state where a stale write would do harm.

License

MIT

Documentation

Overview

Package redlease implements lease-based leader election on Redis with fencing tokens.

One instance holds a Redis lock with a TTL and runs the caller's work while it is leader; if it cannot renew, it steps down so another instance takes over. Unlike a plain Redis lock, every leadership term is assigned a strictly increasing fencing token. Writes routed through the Elector's Fence* helpers are rejected when they carry a token older than the latest applied, so a paused or clock-skewed leader that still believes it holds the lock cannot overwrite newer state.

When you need this

Fencing matters only when a stale leader's write to shared state would be harmful — for example a value that must not regress, a sequence number, or a counter. If the leader does no writes, or writes self-healing last-writer-wins state, a plain lock is enough and you do not need this package.

Correctness boundary

The fence token is generated and stored in Redis. On a single Redis instance this gives a strict, monotonic guarantee. On a replicated deployment (Sentinel or Cluster), Redis replication is asynchronous: a primary can acknowledge the acquire — and the token increment — before it reaches a replica, and a failover to that replica can lose them. In that window the monotonicity the fence relies on can be violated. For strict correctness across failover, source the fencing token from a linearizable store (etcd, ZooKeeper) instead of Redis. This package is the right tool when you run Redis single-instance, or when a brief, rare token regression on failover is acceptable.

Example

This example runs a periodic job on exactly one instance at a time. Several instances call Run with the same Config.Name; only the elected leader executes the job. Each write the leader makes carries its leadership term's fencing token, so a stale leader that has not yet noticed it lost the lock cannot overwrite a newer leader's state. 

package main

import (
	"context"
	"log"
	"time"

	goredis "github.com/redis/go-redis/v9"

	"github.com/nijatsdev/redlease"
)

func main() {
	rc := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})

	e, err := redlease.New(rc, redlease.Config{
		Name: "report-builder",
		TTL:  5 * time.Second,
	})
	if err != nil {
		log.Fatal(err)
	}

	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Run blocks until ctx is cancelled. The callback runs only while this
	// instance is the leader; its context is cancelled the moment leadership
	// is lost.
	e.Run(ctx, func(leaderCtx context.Context, token int64) {
		ticker := time.NewTicker(time.Second)
		defer ticker.Stop()

		for {
			select {
			case <-leaderCtx.Done():
				return
			case <-ticker.C:
				// Record progress under the fencing token. If a newer leader
				// has taken over, applied is false: this term is stale and
				// must stop working.
				applied, err := e.FenceHSet(leaderCtx, token, "jobs:report", "status", "running")
				if err != nil {
					log.Printf("write failed: %v", err)
					continue
				}

				if !applied {
					log.Printf("fenced out by a newer leader; stopping")
					return
				}
			}
		}
	})
}
Example (DatabaseFencing)

This example fences a write to a resource other than Redis — here a SQL database. redlease enforces the fence for you only on Redis writes; for any other store it gives you the term's fencing token and you enforce it at the resource, atomically with your write. In SQL that means a conditional UPDATE that only applies when the row's stored fence is not newer than your token.

The table is assumed to hold a fence column alongside the value: 

CREATE TABLE state (id text PRIMARY KEY, value text, fence bigint NOT NULL DEFAULT 0);

package main

import (
	"context"
	"database/sql"
	"log"
	"time"

	goredis "github.com/redis/go-redis/v9"

	"github.com/nijatsdev/redlease"
)

func main() {
	rc := goredis.NewClient(&goredis.Options{Addr: "localhost:6379"})

	db, err := sql.Open("postgres", "postgres://localhost/app")
	if err != nil {
		log.Fatal(err)
	}

	defer func() { _ = db.Close() }()

	e, err := redlease.New(rc, redlease.Config{Name: "report-builder", TTL: 5 * time.Second})
	if err != nil {
		log.Print(err)
		return
	}

	ctx := context.Background()

	e.Run(ctx, func(leaderCtx context.Context, token int64) {
		// The UPDATE applies only when our token is at least the stored fence,
		// then advances it. A stale leader's lower token matches no row, so its
		// write is rejected at the database — the same guarantee FenceHSet gives
		// for Redis, enforced here in SQL.
		const q = `UPDATE state
		             SET value = $1, fence = $2
		           WHERE id = $3 AND fence <= $2`

		res, err := db.ExecContext(leaderCtx, q, "running", token, "report")
		if err != nil {
			log.Printf("write failed: %v", err)
			return
		}

		if n, _ := res.RowsAffected(); n == 0 {
			// No row updated: a newer leader has advanced the fence past our
			// token. We are stale and must stop working.
			log.Printf("fenced out by a newer leader; stopping")
			return
		}
	})
}

Index

Examples

Constants

View Source 
const (
	DefaultTTL             = 5 * time.Second
	DefaultRenewInterval   = 2 * time.Second
	DefaultAcquireInterval = 2 * time.Second
)

Default timing parameters. Override via Config.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config 

type Config struct {
	// Name identifies the lock. All instances contending for the same leadership
	// must use the same Name; different Names are independent locks.
	Name string

	// TTL is how long the lock lives without renewal. A leader that cannot renew
	// within TTL loses leadership. Shorter TTL means faster failover but less
	// tolerance for pauses. Defaults to DefaultTTL.
	TTL time.Duration

	// RenewInterval is how often the leader renews the lock. Must be well below
	// TTL. Defaults to DefaultRenewInterval.
	RenewInterval time.Duration

	// AcquireInterval is how often a follower retries acquiring the lock.
	// Defaults to DefaultAcquireInterval.
	AcquireInterval time.Duration

	// InstanceID uniquely identifies this instance; it is the lock's value and
	// guards renewal and release so an instance never affects another's lock. If
	// empty, a hostname + random suffix is generated.
	InstanceID string

	// Observer receives lifecycle events. All fields are optional; the library
	// emits no output of its own, so callers wire these to their own logger,
	// metrics, or tracing as they see fit.
	Observer Observer
}

Config configures an Elector. Only Name is required (the client is passed separately to New); zero-valued timing fields fall back to the Default* constants.

type Elector 

type Elector struct {
	// contains filtered or unexported fields
}

Elector runs leader election for a single lock and mints fencing tokens.

func New 

func New(client Redis, cfg Config) (*Elector, error)

New returns an Elector from cfg. It returns an error if required fields are missing or timing parameters are inconsistent (e.g. RenewInterval >= TTL).

func (*Elector) FenceEval 

func (e *Elector) FenceEval(ctx context.Context, token int64, body string, writeKeys []string, args ...any) (applied bool, err error)

FenceEval fences an arbitrary Redis write supplied as a Lua body, for writes the typed helpers do not cover (ZADD, XADD, multi-key updates, and so on). The body runs atomically only when token is current; it must not return a value (the fence returns 1 when applied, 0 when fenced out).

Within body, the protected token check is already done. Address your own keys and arguments starting at index 2 — KEYS[1] and ARGV[1] are reserved for the fence (the high-water key and the token). Pass your keys in writeKeys and your arguments in args; they appear as KEYS[2..] and ARGV[2..] in that order.

Example — fence a ZADD: 

applied, err := e.FenceEval(ctx, token,
    "redis.call('zadd', KEYS[2], ARGV[2], ARGV[3])",
    []string{"myset"}, "1.0", "member")

func (*Elector) FenceHSet 

func (e *Elector) FenceHSet(ctx context.Context, token int64, hashKey, field, value string) (applied bool, err error)

FenceHSet performs a fenced HSET of field=value into hashKey: the write is applied only if token is at least the highest token already applied through this elector. It returns true when applied and false when token is stale — a newer leadership term has since written — so the caller can stop emitting derived events. token must be the value passed to the LeaderFunc for the current term. A non-nil error means the write could not be attempted.

func (*Elector) FenceSet 

func (e *Elector) FenceSet(ctx context.Context, token int64, key, value string) (applied bool, err error)

FenceSet performs a fenced SET of key=value, applied only if token is current. Semantics match Elector.FenceHSet.

func (*Elector) InstanceID 

func (e *Elector) InstanceID() string

InstanceID returns this elector's instance identity.

func (*Elector) IsLeader 

func (e *Elector) IsLeader() bool

IsLeader reports whether this instance currently holds leadership. It is advisory only: leadership can be lost the instant after it returns, so never gate a correctness-sensitive write on it. For writes, carry the token from Elector.Token through a Fence* helper, which rejects a stale token at write time.

func (*Elector) Run 

func (e *Elector) Run(ctx context.Context, fn LeaderFunc)

Run contends for leadership until ctx is cancelled. Each time this instance wins, it invokes fn (a LeaderFunc) with the term's fencing token, then steps down and re-contends when leadership is lost. Run blocks until ctx is cancelled and fn (if running) has returned; on the final shutdown step-down the OnSteppedDown observer still fires.

fn may be nil. Then Run just keeps this instance elected — acquiring, renewing, and releasing the lock — while the caller does its leader work from another goroutine, gated on Elector.Token. This is the token-driven style; the callback style and this one are interchangeable, pick whichever fits.

func (*Elector) Token 

func (e *Elector) Token() (token int64, ok bool)

Token returns the current leadership term's fencing token and true while this instance is the leader, or 0 and false otherwise. It is safe to call from any goroutine, so code outside the LeaderFunc can perform fenced writes: 

if token, ok := e.Token(); ok {
    e.FenceHSet(ctx, token, "state", "key", "value")
}

The returned token is a snapshot; leadership can change immediately after. That is safe because the fence is enforced at write time — a stale token is rejected by the Fence* helpers — so unlike Elector.IsLeader there is no time-of-check-to-time-of-use hazard in acting on it.

type LeaderFunc 

type LeaderFunc func(ctx context.Context, token int64)

LeaderFunc is the work run while an instance is leader. It receives a context cancelled when leadership is lost or Run's context is cancelled, and the fencing token for this leadership term. Stamp every fenced write with token. LeaderFunc must return promptly once its context is cancelled; Run does not release the lock or re-contend until it does.

type Observer 

type Observer struct {
	// OnElected fires when this instance wins a leadership term, with that
	// term's fencing token. It fires before the LeaderFunc is invoked.
	OnElected func(token int64)

	// OnSteppedDown fires when this instance loses or relinquishes leadership,
	// after the LeaderFunc has returned.
	OnSteppedDown func()
}

Observer is a set of optional callbacks invoked on leadership transitions. A nil field is simply not called. Callbacks run on the Run goroutine and must not block; offload slow work (I/O, network) to a goroutine of your own.

Only leadership transitions are reported, mirroring the design of k8s.io/client-go/tools/leaderelection. Transient Redis errors during acquire or renewal are handled internally; their only consequence the caller cares about — losing leadership — surfaces through OnSteppedDown. Monitor Redis health through your Redis client, not through this Observer.

type Redis 

type Redis interface {
	goredis.Scripter
}

Redis is the subset of the go-redis client this package needs: the script runner used by all lock and fence operations. *goredis.Client and *goredis.ClusterClient both satisfy it, as does any compatible wrapper.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.