distributed_lock

README

distributed_lock

Distributed locks in golang - when you have multiple golang processes (on one computer or many across networks) and you need to guarantee - no matter what - that your code is executing all by itself. Comes with a built in watchdog system for lock expiration.

The Basics

Import the package without an alias:

import "github.com/hlfshell/distributed_lock"

Assuming you have a function that you want to run without risking multiple executions:

// Create a lock for the context of "reports:daily".
// It'll expire within a minute if not refreshed by the lock.
// By default, the renewal timer is  expiration / 3, 20 seconds here.
// It'll time out if we don't acquire the lock within 10 seconds.
lock, err := distributed_lock.NewLock("reports:daily",
    distributed_lock.WithExpiration(time.Minute),
    distributed_lock.WithAcquireTimeout(10*time.Second),
)
if err != nil {
    return err
}

// Wait up to ten seconds for ownership. The watchdog starts on success.
if err := lock.Lock(); err != nil {
    return fmt.Errorf("lock daily report: %w", err)
}
defer lock.Release()

// If we've hit this point, we have the lock - do your thing!
return generateReport()

The watchdog refreshes the lock automatically while your function runs. If the process dies, other workers can acquire the lock, assuming the prior lock died.

A lock has three methods:

Method	Behavior
Lock() error	Wait for ownership and start automatic renewal.
Release() error	End ownership, stop renewal, and clean up. Safe to defer.
Locked() bool	Check whether or not the lock is indeed locked

Options and defaults

Options belong on NewLock, and apply each time that object calls Lock(). Managers can also supply defaults for the locks they create.

Option	Definition	Default
WithExpiration(duration)	How long ownership lasts without a successful renewal	One minute; minimum 10ms
WithRenewalInterval(duration)	How often the watchdog renews ownership	Expiration / 3; zero restores this default
WithAcquireTimeout(duration)	Maximum time Lock() waits for ownership	Zero: wait indefinitely
WithNoWait()	Make one attempt instead of waiting for a held name	Disabled
WithReleaseTimeout(duration)	Maximum time Release() waits for cleanup	Five seconds; must be positive
WithOnExpiration(func(error))	Callback when ownership unexpectedly ends	None

An acquisition timeout returns context.DeadlineExceeded; WithNoWait returns ErrLocked when another object owns the name. Backend failures return errors. The expiration is refreshed automatically, so it does not limit total execution time while renewal succeeds. Choose an expiration comfortably above backend latency.

Unexpected expiration

You can react without polling Locked() by setting a callback at construction:

lock, err := distributed_lock.NewLock("reports:daily",
    distributed_lock.WithOnExpiration(func(err error) {
        // Signal your application to stop the protected work.
        // errors.Is(err, distributed_lock.ErrLost) is true.
    }),
)

The callback runs once per unexpected ownership loss, asynchronously outside internal locks. It does not run on normal release or manager closure. It must return and must not panic. Renewal failures are retried within the last confirmed validity window; late responses never restore lost ownership.

Lock types

Type	Who shares ownership?	What is different?
In memory	Goroutines using the same memory backend	No external service; state lasts for this process.
Singleton	Callers using the process-wide default	A convenient accessor to a chosen backend; defaults to memory.
PostgreSQL / MySQL	Processes or machines using the same database and table	Dedicated lock table with explicit migrations.
SQLite	Processes using the same local database file	Persistent local storage without a separate database server.
Redis	Processes or machines using the same Redis authority and prefix	Expiring keys and atomic owner checks; no table migrations.

All types use the same construction options, watchdog, three lock methods, and expiration callback. The snippets below belong inside a function returning error, with a caller-provided ctx. Construct the lock once, call Lock() to begin protected work, and defer Release().

In-memory locks

Use this when only goroutines in one process need to coordinate. A memory manager creates named locks sharing one backend. Managers from separate NewMemory calls are independent:

manager, err := distributed_lock.NewMemory()
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

To share one lock namespace between separately constructed managers, give both the same Memory instance:

storage := &distributed_lock.Memory{}
first, err := distributed_lock.New(storage)
if err != nil { return err }
defer first.Close(context.Background())
second, err := distributed_lock.New(storage)
if err != nil { return err }
defer second.Close(context.Background())

lock, err := first.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

contender, err := second.NewLock("reports:daily", distributed_lock.WithNoWait())
if err != nil { return err }
// err is distributed_lock.ErrLocked because first still owns this name.
err = contender.Lock()

Singleton locks

Use package-level construction when you do not want to pass a manager around:

lock, err := distributed_lock.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

Each NewLock returns a separate, initially unlocked object. By default the objects share one in-memory backend within the process. The singleton itself does not coordinate separate processes. Configure it once, before first use, to select a manager with custom defaults or a database backend:

manager, err := distributed_lock.NewMemory(distributed_lock.WithExpiration(2*time.Minute))
if err != nil { return err }
defer manager.Close(context.Background())
if err := distributed_lock.SetDefault(manager); err != nil { return err }

// Anywhere else in this process:
lock, err := distributed_lock.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

To coordinate separate processes, configure a manager from one of the database examples below instead. Default() returns the configured manager. A second SetDefault call, or one after first use, returns an error. Keep the configured manager alive for the application's lifetime.

Database locks

Each backend is a separate Go module inside this repository. Import only the backend you need; the root package has no third-party dependencies. SQL backends use a dedicated lock table, with configurable table and column names. Database connections stay caller-owned. Managers hold no SQL transaction during your work.

PostgreSQL

Use PostgreSQL to coordinate processes across machines through an existing PostgreSQL database. Open uses pgx through database/sql and accepts a URL or DSN. Import github.com/hlfshell/distributed_lock/postgres alongside the root package:

db, err := postgres.Open(ctx, "postgres://user:password@localhost:5432/app?sslmode=disable")
if err != nil { return err }
defer db.Close()

store, err := postgres.New(db, postgres.Config{Table: "job_locks"})
if err != nil { return err }
if err := store.Migrate(ctx); err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

The connection string above targets local development; configure the connection's TLS settings for your deployment. PostgreSQL migrations serialize concurrent table creation. Select a schema through the connection's search path.

MySQL

Use MySQL for the same cross-machine coordination when your application already uses MySQL. Tables use InnoDB. Open accepts a go-sql-driver/mysql DSN. Import github.com/hlfshell/distributed_lock/mysql:

db, err := mysql.Open(ctx, "user:password@tcp(localhost:3306)/app?timeout=5s")
if err != nil { return err }
defer db.Close()

store, err := mysql.New(db, mysql.Config{
    Table: "job_locks",
    KeyColumn: "resource",
    OwnerColumn: "holder",
    ExpiryColumn: "deadline",
})
if err != nil { return err }
if err := store.Migrate(ctx); err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

All clients must use the same table mapping. The column customization shown here also works with PostgreSQL and SQLite.

SQLite

Use SQLite for persistent coordination between processes on the same machine without running a database server. It uses the pure-Go modernc.org/sqlite driver. Import github.com/hlfshell/distributed_lock/sqlite:

db, err := sqlite.Open(ctx,
    "file:/absolute/path/locks.db?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)")
if err != nil { return err }
defer db.Close()

store, err := sqlite.New(db, sqlite.Config{})
if err != nil { return err }
if err := store.Migrate(ctx); err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

Use the same absolute file path and table mapping in every process. Configure the busy timeout on every connection. SQLite coordinates through its local file. Use a local filesystem for SQLite, not a network filesystem.

Redis

Use Redis for cross-machine coordination through expiring keys. It requires no migrations. Import the backend and its client package:

import (
    "github.com/hlfshell/distributed_lock/redis"
    goredis "github.com/redis/go-redis/v9"
)

client, err := redis.Open(ctx, &goredis.Options{Addr: "localhost:6379"})
if err != nil { return err }
defer client.Close()

store, err := redis.New(client, redis.Config{Prefix: "myapp:locks:"})
if err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

The default prefix is distributed_lock:. There are no migrations. Acquisition uses atomic SET NX PX; Lua scripts compare the owner before renewing or deleting. Use a namespace that cannot be evicted or flushed. This is not Redlock and does not guarantee safety across failover that loses acknowledged writes.

redis.Open enables context timeouts and disables command retries. When supplying an existing client to redis.New, configure it to obey operation contexts and use bounded network timeouts. Automatic failover to storage missing a live lock breaks the backend contract even if the client reconnects successfully.

SQL configuration and migrations

Database locks use their own table. The default is distributed_locks, with columns named lock_key, owner_token, and expires_at. Change these with Table, KeyColumn, OwnerColumn, and ExpiryColumn. All processes sharing a lock need to use the same table and column names.

Names can contain letters, numbers, and underscores, up to 63 characters. Start each name with a letter or underscore. For PostgreSQL, set the schema through your connection's search path.

Call Migrate(ctx) during setup to create the table. It's safe to call more than once, including from multiple processes at the same time. Creating a lock manager doesn't run migrations for you. To get the SQL and run it yourself, use MigrationSQL().

Migrations only create the table. They don't change an existing table's columns or provide a way to drop it. Expiration uses the database's clock, stored as microseconds since the Unix epoch. Transactions finish before your function runs.

License

MIT.

Documentation

Index

• Variables
• func SetDefault(locker *Manager) error
• type Backend
• type Lock
  • func NewLock(key string, options ...Option) (*Lock, error)
  • func (l *Lock) Lock() error
  • func (l *Lock) Locked() bool
  • func (l *Lock) Release() error
• type Locker
• type Manager
  • func Default() *Manager
  • func New(backend Backend, options ...Option) (*Manager, error)
  • func NewMemory(options ...Option) (*Manager, error)
  • func (m *Manager) Close(ctx context.Context) error
  • func (m *Manager) NewLock(key string, options ...Option) (*Lock, error)
• type Memory
  • func (m *Memory) Acquire(ctx context.Context, key, owner string, ttl time.Duration) (bool, error)
  • func (m *Memory) Release(ctx context.Context, key, owner string) error
  • func (m *Memory) Renew(ctx context.Context, key, owner string, ttl time.Duration) (bool, error)
• type Option
  • func WithAcquireTimeout(d time.Duration) Option
  • func WithExpiration(d time.Duration) Option
  • func WithNoWait() Option
  • func WithOnExpiration(fn func(error)) Option
  • func WithReleaseTimeout(d time.Duration) Option
  • func WithRenewalInterval(d time.Duration) Option

Variables

var (
	// ErrLocked means another owner currently holds the requested key.
	ErrLocked = errors.New("lock is held")
	// ErrLost means ownership expired or could not be confirmed before its deadline.
	ErrLost = errors.New("lock ownership lost")
	// ErrClosed means the manager has been closed.
	ErrClosed = errors.New("lock manager closed")
	// ErrInvalid means configuration or a request is invalid.
	ErrInvalid = errors.New("invalid lock configuration")
	// ErrReleased denotes an explicitly released lock.
	ErrReleased = errors.New("lock released")
	// ErrAlreadyLocked means Lock was called on an object that already owns its name.
	ErrAlreadyLocked = errors.New("this lock is already held")
	// ErrBusy means this object's Lock or Release call is still in progress.
	ErrBusy = errors.New("lock operation already in progress")
)

Functions

func SetDefault

func SetDefault(locker *Manager) error

SetDefault configures the singleton once, before Default or package NewLock.

Types

type Backend

type Backend interface {
	Acquire(ctx context.Context, key, owner string, ttl time.Duration) (bool, error)
	Renew(ctx context.Context, key, owner string, ttl time.Duration) (bool, error)
	Release(ctx context.Context, key, owner string) error
}

Backend implements atomic lock storage. Calls must obey ctx. Acquire returns false only for contention. Renew must reject expired owners; Release must never remove another owner's record. Successful calls establish at least ttl of storage validity measured from the start of the operation. Backend clock jumps and loss of committed storage are outside this contract.

type Lock

type Lock struct {
	// contains filtered or unexported fields
}

Lock is a reusable named lock. Construction does not acquire ownership. Do not copy a Lock. Create separate objects for independent contenders.

func NewLock

func NewLock(key string, options ...Option) (*Lock, error)

NewLock constructs a named lock using the process-wide default manager. Options are validated now and reused on every Lock call.

func (*Lock) Lock

func (l *Lock) Lock() error

Lock waits for ownership, then starts automatic renewal. WithAcquireTimeout bounds waiting; Release cancels an in-progress call. Repeated Lock on an owned object returns ErrAlreadyLocked. After release or expiration it may be reused.

func (*Lock) Locked

func (l *Lock) Locked() bool

Locked reports whether this object still has confirmed, unexpired ownership. It performs no network I/O and does not acquire or renew anything. A false result also covers another object's ownership of the same name. A true result is a point-in-time observation, not a guarantee about subsequent work.

func (*Lock) Release

func (l *Lock) Release() error

Release stops renewal and removes only this object's current ownership. It is safe before Lock, after loss, or repeatedly. It also cancels a pending Lock. Cleanup errors can be retried; WithReleaseTimeout bounds each call.

type Locker

type Locker interface {
	Lock() error
	Locked() bool
	Release() error
}

Locker is a configured named lock. Lock and Release may perform I/O; Locked reports whether this object currently owns the lock.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager coordinates waiting and watchdogs over a Backend. Close does not close the backend or caller-owned database connections.

func Default

func Default() *Manager

Default returns the process-wide locker, initialized to memory on first use.

func New

func New(backend Backend, options ...Option) (*Manager, error)

New constructs a manager with optional acquisition defaults.

func NewMemory

func NewMemory(options ...Option) (*Manager, error)

NewMemory constructs an independent in-memory manager.

func (*Manager) Close

func (m *Manager) Close(ctx context.Context) error

Close stops waiters and all watchdogs, and attempts to release outstanding locks. Repeated calls are safe; failed releases remain bounded by storage expiration.

func (*Manager) NewLock

func (m *Manager) NewLock(key string, options ...Option) (*Lock, error)

NewLock constructs an initially unlocked object backed by this manager. All objects created by a manager coordinate on the same backend namespace.

type Memory

type Memory struct {
	// contains filtered or unexported fields
}

Memory stores locks shared by managers using this instance. Its zero value is usable. Storage is process-local and disappears when the process terminates.

func (*Memory) Acquire

func (m *Memory) Acquire(ctx context.Context, key, owner string, ttl time.Duration) (bool, error)

Acquire atomically claims an absent or expired record.

func (*Memory) Release

func (m *Memory) Release(ctx context.Context, key, owner string) error

Release removes only the matching owner. Missing records are already released.

func (*Memory) Renew

func (m *Memory) Renew(ctx context.Context, key, owner string, ttl time.Duration) (bool, error)

Renew extends an unexpired record only for its current owner.

type Option

type Option func(*settings)

Option configures a lock at construction, or a manager default.

func WithAcquireTimeout

func WithAcquireTimeout(d time.Duration) Option

WithAcquireTimeout bounds Lock waiting; zero waits until Release or manager closure.

func WithExpiration

func WithExpiration(d time.Duration) Option

WithExpiration sets how long ownership lasts without renewal (default one minute, minimum 10 milliseconds).

func WithNoWait

func WithNoWait() Option

WithNoWait makes Lock attempt once and return ErrLocked if the name is held.

func WithOnExpiration

func WithOnExpiration(fn func(error)) Option

WithOnExpiration installs a callback for unexpected loss only. It runs once, asynchronously outside locks for each unexpected ownership loss. The error satisfies errors.Is(err, ErrLost). The callback must return and must not panic.

func WithReleaseTimeout

func WithReleaseTimeout(d time.Duration) Option

WithReleaseTimeout bounds Release, including cancellation of a pending Lock. The default is five seconds; the duration must be positive.

func WithRenewalInterval

func WithRenewalInterval(d time.Duration) Option

WithRenewalInterval sets the watchdog period; zero selects expiration/3.