idem

package module

v0.1.0 Latest Latest Go to latest Published: Oct 2, 2026 License: MIT Imports: 10 Imported by: 0

README ¶

idem

Idempotency keys for Go. idem runs an operation at most once per key and replays the stored result to every retry, so a client that times out can send the same request again without charging a card twice.

It works for HTTP handlers, gRPC methods and any plain function, and keeps its records in memory, Redis or PostgreSQL. The core module has no third-party dependencies.

engine := idem.New(memstore.New())

mux.Handle("POST /orders", idemhttp.Middleware(engine)(createOrder))

$ curl -i -X POST localhost:8080/orders -H 'Idempotency-Key: demo-1' -d '{"sku":"book"}'
HTTP/1.1 201 Created
{"id":"ord_0001", ...}

$ curl -i -X POST localhost:8080/orders -H 'Idempotency-Key: demo-1' -d '{"sku":"book"}'
HTTP/1.1 201 Created
Idempotent-Replayed: true
{"id":"ord_0001", ...}

What it adds

There are other idempotency middlewares for Go. This one exists for four things:

Any function, not only requests. idem.Do protects a queue consumer or a scheduled job with the same engine and store that protect your handlers.
Leases with heartbeats instead of fixed lock timeouts. A key is held on a short lease (30 seconds by default) that is renewed while the work runs. A crashed process blocks retries for one lease at most, and a slow one is not cut off.
Lease tokens. Every claim carries a random token, and the store rejects writes from a token that no longer holds the key. A worker that stalled cannot overwrite the result of the worker that took over.
Transactional completion on PostgreSQL. pgstore.DoTx commits the idempotency record in the same transaction as your own writes, so the two can never disagree.

The core module imports only the standard library. Redis and PostgreSQL are reached through the client you already have.

Install

go get github.com/udaykishore-resu/idem

Requires Go 1.22 or later. The gRPC interceptor is a separate module, so that only gRPC services pull in gRPC:

go get github.com/udaykishore-resu/idem/idemgrpc

Packages

Package	Purpose
`idem`	The engine, `Do`, and the `Store` interface
`idemhttp`	`net/http` middleware for the `Idempotency-Key` header
`idemgrpc`	Unary server interceptor (separate module)
`memstore`	In-memory store for tests and single instances
`redisstore`	Redis store, through any client that can run a script
`pgstore`	PostgreSQL store through `database/sql`, with `DoTx`
`storetest`	Conformance suite for your own `Store`

How a key behaves

absent ──Acquire──▶ in progress ──Complete──▶ completed ──ttl──▶ absent
                        │
                        └──Release, or the lease runs out──▶ absent

sequenceDiagram
    participant C as Client
    participant S as Service
    participant St as Store
    C->>S: POST /orders (Idempotency-Key: k)
    S->>St: Acquire(k)
    St-->>S: acquired
    S->>S: run the handler
    Note over C,S: the reply is lost
    C->>S: POST /orders (Idempotency-Key: k), retry
    S->>St: Acquire(k)
    St-->>S: in progress
    S-->>C: 409 Conflict (or wait, with WithWait)
    S->>St: Complete(k, response)
    C->>S: POST /orders (Idempotency-Key: k), retry
    S->>St: Acquire(k)
    St-->>S: completed + stored response
    S-->>C: 201 Created, Idempotent-Replayed: true

Usage

Any function

receipt, replayed, err := idem.Do(ctx, engine, msg.ID,
    func(ctx context.Context) (Receipt, error) {
        return charge(ctx, msg)
    })

Do returns the stored result without calling the function if the key has already completed. If the function returns an error or panics, nothing is stored and the key is released, so the retry runs it again. The context passed to the function is cancelled if the lease is lost.

For results that are not a single value, Engine.Begin hands you the claim directly:

attempt, stored, err := engine.Begin(ctx, key)
if err != nil {
    return err // in progress elsewhere, wrong fingerprint, or a store error
}
if attempt == nil {
    return use(stored) // already completed
}
defer attempt.Release(ctx) // does nothing once Complete has succeeded

result, err := work(attempt.Context())
if err != nil {
    return err
}
return attempt.Complete(ctx, result)

HTTP

idempotent := idemhttp.Middleware(engine,
    idemhttp.Scope(func(r *http.Request) string { return auth.TenantID(r.Context()) }),
    idemhttp.RequireKey(),
)
mux.Handle("POST /orders", idempotent(createOrder))

Situation	Response
No key	Passes through, or 400 with `RequireKey`
First request with a key	The handler runs; its response is stored
Repeat of a completed request	Stored response, `Idempotent-Replayed: true`
Repeat while the first is running	409 Conflict
Same key, different method, path, query or body	422 Unprocessable Content
Malformed key, or a body that cannot be read	400
Store unreachable	503, or pass through with `FailOpen`

Responses are stored unless their status asks the client to retry (408, 425, 429 and every 5xx). Errors are reported as RFC 9457 problem details. The middleware acts on POST and PATCH by default.

Scope your keys. Clients choose their keys, so two clients can choose the same one. Without Scope, the second client receives the first client's stored response. Bind keys to the authenticated caller and install the middleware after authentication.

gRPC

server := grpc.NewServer(grpc.ChainUnaryInterceptor(
    idemgrpc.UnaryServerInterceptor(engine, idemgrpc.Scope(tenantFromContext)),
))

The key travels in the idempotency-key metadata entry. A repeat while the first call is running fails with Aborted, a key reused for a different request with InvalidArgument, and an unreachable store with Unavailable. Only successful responses are stored.

Stores

// In memory: tests, examples, single instances.
engine := idem.New(memstore.New())

// Redis, with go-redis. Any client that can run EVAL works.
store := redisstore.New(redisstore.EvalFunc(
    func(ctx context.Context, script string, keys []string, args ...any) (any, error) {
        return rdb.Eval(ctx, script, keys, args...).Result()
    }))

// PostgreSQL, with whichever database/sql driver you use.
store := pgstore.New(db)
err := store.CreateTable(ctx) // or put store.Schema() in a migration

Committing the record with your data

With Redis, or with Do on PostgreSQL, the work and the record are two separate writes. A crash between them leaves the work done and the record missing, and the retry repeats the work. DoTx removes that window:

order, replayed, err := pgstore.DoTx(ctx, engine, key,
    func(ctx context.Context, tx *sql.Tx) (Order, error) {
        o := Order{ID: newID(), Total: 4200}
        _, err := tx.ExecContext(ctx,
            `INSERT INTO orders (id, total) VALUES ($1, $2)`, o.ID, o.Total)
        return o, err
    })

The insert and the idempotency record commit together or not at all. If the caller lost its lease in the meantime, the transaction rolls back and the caller that took the key over does the work instead. DoTx runs the transaction at READ COMMITTED and needs a pool of at least two connections, one for the transaction and one for the heartbeat.

Options

Option	Default	Meaning
`WithLease(d)`	30s	How long a key stays claimed without a heartbeat
`WithHeartbeat(d)`	lease / 3	How often the lease is renewed; 0 turns it off
`WithTTL(d)`	24h	How long a completed result is replayed
`WithWait(d)`	0	Wait this long for a running duplicate instead of failing
`WithFingerprint(fp)`	none	Reject a key reused for a different request
`WithNamespace(ns)`	none	Key prefix, so engines can share a store
`WithCodec(c)`	JSON	How `Do` encodes results

What is and is not guaranteed

While a lease is held, at most one caller runs the operation for a key. Once it succeeds, every caller with that key gets the stored result until the record expires.

A lease is not a transaction. A holder that cannot renew its lease, because its process is frozen or because it cannot reach the store, loses the key when the lease runs out, and another caller may then start the same work. The first holder's context is cancelled as soon as it can tell, and the store refuses its result once another caller holds the key, but whatever it already did outside the store has happened. pgstore.DoTx closes that gap for writes to the same database. For side effects in other systems, pass the key on to them so that they deduplicate too.

idemhttp reads the request body and buffers the response in memory, 1 MiB each by default. It is meant for API calls, not for uploads or streams.

The HTTP middleware, the gRPC interceptor and Do each store their own kind of record. If they share a store, give each its own engine with its own WithNamespace.

The module is at v0: the API may still change before v1.0.0.

Writing a store

Implement the four methods of idem.Store and run the conformance suite against them:

func TestStore(t *testing.T) {
    storetest.Run(t, func(t *testing.T) idem.Store { return mystore.New(connect(t)) })
}

Development

make test          # unit tests with the race detector
make run           # start the example service on localhost:8080
make services      # start Redis and PostgreSQL with Docker Compose
make integration   # run every store against those servers, plus the gRPC tests
make tidy          # go mod tidy in all three modules

The repository holds three modules. The root module has no dependencies. idemgrpc depends on gRPC. integration tests the stores with go-redis, pgx and lib/pq against real servers and is never imported.

Skills demonstrated

Distributed systems: leases, heartbeats, ownership tokens, at-most-once execution, crash recovery, and an explicit statement of what cannot be guaranteed.
Go API design: generics (Do[T]), functional options, small interfaces, context propagation and cancellation causes, a dependency-free core.
PostgreSQL: atomic claim with INSERT ... ON CONFLICT, row-level locking under READ COMMITTED, transactional completion, server-side clocks.
Redis: atomic Lua scripts, key expiry as lease enforcement, a client-agnostic adapter.
HTTP and gRPC: middleware and interceptors, response buffering and replay, RFC 9457 problem details, the IETF Idempotency-Key draft.
Testing: a reusable conformance suite checked against deliberately broken stores, race-detector runs, integration tests on real Redis and PostgreSQL, runnable documentation examples.

Links

License

MIT

Documentation ¶

Overview ¶

Package idem makes operations safe to retry: it runs an operation at most once per idempotency key and replays the stored result to every later caller.

A client that times out cannot tell whether its request was lost or whether only the reply was. Retrying is the only way forward, and without protection a retry repeats the side effect: a second charge, a second order, a second email. With an idempotency key the client names the operation, and the server uses the name to recognise a retry.

Model ¶

Every key moves through three states:

absent ──Acquire──▶ in progress ──Complete──▶ completed ──ttl──▶ absent
                        │
                        └──Release, or the lease runs out──▶ absent

The caller that acquires a key holds a lease on it. While the operation runs, a heartbeat keeps extending the lease. If the process dies the lease runs out and the key can be acquired again, so a crash blocks retries for one lease, 30 seconds by default, rather than for the lifetime of a fixed lock.

Every acquisition carries a random token, and a store only accepts Complete, Release and Extend from the token that currently holds the key. A worker that stalled past its lease therefore cannot overwrite the record of the worker that took over; it gets ErrLeaseLost instead.

Using it ¶

Create one Engine per store and share it:

engine := idem.New(memstore.New())

Do protects any function, whatever delivered the work: an HTTP handler, a queue consumer, a scheduled job.

receipt, replayed, err := idem.Do(ctx, engine, msg.ID,
	func(ctx context.Context) (Receipt, error) {
		return charge(ctx, msg)
	})

For request handlers, package idemhttp wraps an net/http.Handler and module idemgrpc provides a unary server interceptor. Both are built on Engine.Begin, which is exported for flows that Do does not fit.

Stores ¶

A Store keeps the records. Three ship with this module, and none of them adds a dependency to your build:

memstore keeps records in process memory, for tests and single instances.
redisstore runs on Redis through whichever client you already use.
pgstore runs on PostgreSQL through database/sql, and can write the record in the same transaction as your own changes.

Package storetest checks any Store against the contract.

What is and is not guaranteed ¶

While the lease is held, at most one caller runs the operation for a key. After it succeeds, every caller with that key receives the stored result until the record expires.

A lease is not a transaction. A holder that cannot renew its lease, because its process is frozen or because it cannot reach the store, loses the key when the lease runs out, and a second caller may then start the same work. The first holder's context is cancelled as soon as it can tell, and the store refuses its result once another caller holds the key, but whatever it already did outside the store has happened. Only a store that commits the record together with the side effects closes that gap; see pgstore.DoTx. For side effects in other systems, pass the key on to them so that they deduplicate too.

Do keeps a result only when the function returns a nil error. A function that fails releases its key, and a retry runs it again. The transport packages document which responses they keep.

Index ¶

Constants
Variables
func Do[T any](ctx context.Context, e *Engine, key string, ...) (v T, replayed bool, err error)
func Fingerprint(parts ...[]byte) string
func Provisional(ctx context.Context) context.Context
type Attempt
type Claim
type Codec
type Engine
- func New(store Store, opts ...Option) *Engine
type Option
type State
- func (s State) String() string
type Store

Constants ¶

View Source

const (
	// DefaultLease is how long a key stays claimed without a heartbeat.
	DefaultLease = 30 * time.Second
	// DefaultTTL is how long a completed result is kept and replayed.
	DefaultTTL = 24 * time.Hour
)

Defaults used by New unless an Option overrides them.

Variables ¶

View Source

var (
	// ErrInProgress means another caller holds the key and has not finished.
	ErrInProgress = errors.New("idem: an operation with this key is already in progress")

	// ErrFingerprintMismatch means the key was first used with a different
	// request. Reusing a key for different work is a client bug.
	ErrFingerprintMismatch = errors.New("idem: key was already used with a different request")

	// ErrLeaseLost means the caller no longer holds the key: its lease ran
	// out, or another caller has acquired the key since.
	ErrLeaseLost = errors.New("idem: lease lost")

	// ErrInvalidKey means the key was empty.
	ErrInvalidKey = errors.New("idem: invalid key")

	// ErrNotRecorded means the operation finished but its result could not be
	// stored, so a retry with the same key may run the operation again. The
	// value returned alongside it is the real result of the operation.
	ErrNotRecorded = errors.New("idem: operation finished but its result was not recorded")
)

Errors returned by this package. Test for them with errors.Is.

Functions ¶

func Do ¶

func Do[T any](ctx context.Context, e *Engine, key string, fn func(context.Context) (T, error), opts ...Option) (v T, replayed bool, err error)

Do runs fn at most once for key and returns its result. If the operation already completed, Do does not call fn: it returns the stored result and reports replayed as true.

fn receives a context that is cancelled if the lease on the key is lost; see Attempt.Context.

If fn returns an error or panics, nothing is stored and the key is released, so the next call with the same key runs fn again. Only successful results are replayed.

The result is encoded with the engine's Codec, JSON by default, so T must survive a round trip through it. If fn succeeds but the result cannot be encoded or stored, Do returns the result together with an error that wraps ErrNotRecorded.

Do returns ErrInProgress, ErrFingerprintMismatch and ErrInvalidKey under the conditions described at Engine.Begin.

Example ¶

A client retries a payment three times because its connection keeps dropping. The card is charged once.

package main

import (
	"context"
	"fmt"

	"github.com/udaykishore-resu/idem"
	"github.com/udaykishore-resu/idem/memstore"
)

type Receipt struct {
	ID     string `json:"id"`
	Amount int    `json:"amount"`
}

func main() {
	ctx := context.Background()
	engine := idem.New(memstore.New())

	charges := 0
	charge := func(ctx context.Context) (Receipt, error) {
		charges++
		return Receipt{ID: fmt.Sprintf("rcpt_%d", charges), Amount: 500}, nil
	}

	for try := 1; try <= 3; try++ {
		receipt, replayed, err := idem.Do(ctx, engine, "payment-42", charge)
		if err != nil {
			fmt.Println("error:", err)
			return
		}
		fmt.Printf("try %d: %s replayed=%t\n", try, receipt.ID, replayed)
	}
	fmt.Println("charges made:", charges)

}

Output:
try 1: rcpt_1 replayed=false
try 2: rcpt_1 replayed=true
try 3: rcpt_1 replayed=true
charges made: 1

Example (Consumer) ¶

A queue that delivers at least once will sooner or later deliver a message twice. Using the message ID as the key turns the redelivery into a no-op.

package main

import (
	"context"
	"fmt"

	"github.com/udaykishore-resu/idem"
	"github.com/udaykishore-resu/idem/memstore"
)

func main() {
	ctx := context.Background()
	engine := idem.New(memstore.New(), idem.WithNamespace("welcome-email"))

	type message struct{ ID, Email string }
	sent := 0
	handle := func(m message) error {
		_, _, err := idem.Do(ctx, engine, m.ID, func(ctx context.Context) (struct{}, error) {
			sent++ // send the email
			return struct{}{}, nil
		})
		return err
	}

	deliveries := []message{
		{ID: "msg-1", Email: "ada@example.com"},
		{ID: "msg-2", Email: "alan@example.com"},
		{ID: "msg-1", Email: "ada@example.com"}, // redelivered
	}
	for _, m := range deliveries {
		if err := handle(m); err != nil {
			fmt.Println("error:", err)
		}
	}
	fmt.Println("emails sent:", sent)

}

Output:
emails sent: 2

Example (Failure) ¶

A failed operation is not remembered: the key is released and the retry runs it again.

package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/udaykishore-resu/idem"
	"github.com/udaykishore-resu/idem/memstore"
)

func main() {
	ctx := context.Background()
	engine := idem.New(memstore.New())

	tries := 0
	reserve := func(ctx context.Context) (string, error) {
		tries++
		if tries == 1 {
			return "", errors.New("inventory service timed out")
		}
		return "reservation-7", nil
	}

	for range 3 {
		v, replayed, err := idem.Do(ctx, engine, "cart-9", reserve)
		fmt.Printf("%q replayed=%t err=%v\n", v, replayed, err)
	}

}

Output:
"" replayed=false err=inventory service timed out
"reservation-7" replayed=false err=<nil>
"reservation-7" replayed=true err=<nil>

func Fingerprint ¶

func Fingerprint(parts ...[]byte) string

Fingerprint returns a digest of parts for use with WithFingerprint. Feed it everything that makes two requests different, such as the operation name and the request body. Each part is length-prefixed, so ("ab", "c") and ("a", "bc") produce different fingerprints.

func Provisional ¶

func Provisional(ctx context.Context) context.Context

Provisional returns a copy of ctx that marks a call to Attempt.Complete as not final yet: the store will only make the record permanent later, or not at all, as when the write is part of a database transaction that has still to commit. Stores that can do this wrap the context themselves; see pgstore.WithTx.

After a provisional Complete the attempt stays open. A later Attempt.Release still reaches the store, which frees the key if the record did not become permanent and does nothing if it did.

Types ¶

type Attempt ¶

type Attempt struct {
	// contains filtered or unexported fields
}

Attempt is one caller's hold on a key. The holder runs the operation and then calls Attempt.Complete or Attempt.Release; until it does, a background goroutine keeps the lease alive, so an Attempt must always be ended. The simplest way is to defer Release, which does nothing once Complete has succeeded:

attempt, stored, err := engine.Begin(ctx, key)
...
defer attempt.Release(ctx)
...
return attempt.Complete(ctx, result)

func (*Attempt) Complete ¶

func (a *Attempt) Complete(ctx context.Context, result []byte) error

Complete stores result as the outcome of the operation and ends the attempt. Later calls to Engine.Begin with the same key return result.

Complete returns ErrLeaseLost if the attempt no longer holds the key; the result is then not stored. If Complete fails, the attempt has not ended and a deferred Attempt.Release still gives the key up. Complete stops the heartbeat in every case.

The values of ctx are passed to the store but its cancellation is not: a client that has gone away must not prevent the outcome from being recorded. The call is bounded by the lease instead, and by five seconds at most.

func (*Attempt) Context ¶

func (a *Attempt) Context() context.Context

Context returns a context derived from the one passed to Engine.Begin. It is cancelled, with ErrLeaseLost as its cause, as soon as the lease is known to be gone: the store reports that the key is no longer held, or no extension has succeeded for a whole lease. Run the operation with this context so that it stops when it has lost the right to finish. The context is also cancelled when the attempt ends.

func (*Attempt) Key ¶

func (a *Attempt) Key() string

Key returns the key as stored, including the engine's namespace.

func (*Attempt) Release ¶

func (a *Attempt) Release(ctx context.Context) error

Release gives the key up without storing a result and ends the attempt, so that a retry can run the operation again straight away. Call it when the operation failed. Like Attempt.Complete, it ignores the cancellation of ctx.

Release does nothing and returns nil if the attempt has already ended, so it is safe to defer. Releasing a key that is no longer held returns ErrLeaseLost, which callers can usually ignore.

func (*Attempt) Token ¶

func (a *Attempt) Token() string

Token returns the token that identifies this attempt to the store.

type Claim ¶

type Claim struct {
	// State says whether the caller acquired the key and, if it did not,
	// what holds it.
	State State

	// Fingerprint is the fingerprint stored with the existing record. It is
	// set when State is StateInProgress or StateCompleted.
	Fingerprint string

	// Result is the stored result. It is set when State is StateCompleted.
	Result []byte
}

Claim is the outcome of Store.Acquire.

type Codec ¶

type Codec interface {
	Marshal(v any) ([]byte, error)
	Unmarshal(data []byte, v any) error
}

Codec turns the result of an operation into bytes for the store and back.

type Engine ¶

type Engine struct {
	// contains filtered or unexported fields
}

Engine runs operations at most once per key against a Store. An Engine is safe for concurrent use; create one per store and share it.

func New ¶

func New(store Store, opts ...Option) *Engine

New returns an Engine that keeps its records in store. It panics if store is nil.

func (*Engine) Begin ¶

func (e *Engine) Begin(ctx context.Context, key string, opts ...Option) (*Attempt, []byte, error)

Begin claims key. It is the building block under Do and the transport packages; use it directly when the result is not a single Go value.

Exactly one of three things happens:

The key was free. Begin returns an Attempt; the caller runs the operation and then calls Attempt.Complete or Attempt.Release.
The operation already completed. Begin returns a nil Attempt and the stored result.
Begin returns an error: ErrInProgress if another caller holds the key, ErrFingerprintMismatch if the key belongs to a different request, ErrInvalidKey if key is empty, or the store's error.

Example ¶

Begin is for results that are not a single Go value. The caller holds an Attempt while it works and ends it with Complete or Release.

package main

import (
	"context"
	"fmt"

	"github.com/udaykishore-resu/idem"
	"github.com/udaykishore-resu/idem/memstore"
)

func main() {
	ctx := context.Background()
	engine := idem.New(memstore.New())

	export := func() {
		attempt, stored, err := engine.Begin(ctx, "export-7")
		switch {
		case err != nil:
			fmt.Println("error:", err)
		case attempt == nil:
			fmt.Printf("already done: %s\n", stored)
		default:
			// Release frees the key if anything below fails. Once Complete
			// has succeeded it does nothing.
			defer attempt.Release(ctx)

			// Work with attempt.Context(), then record what happened.
			if err := attempt.Complete(ctx, []byte("s3://exports/7.csv")); err != nil {
				fmt.Println("error:", err)
				return
			}
			fmt.Println("exported")
		}
	}
	export()
	export()

}

Output:
exported
already done: s3://exports/7.csv

func (*Engine) Codec ¶

func (e *Engine) Codec() Codec

Codec returns the codec Do uses to encode results.

func (*Engine) Store ¶

func (e *Engine) Store() Store

Store returns the store the engine was created with.

type Option ¶

type Option func(*config)

Option configures an Engine. Options passed to New set the engine's defaults; unless noted otherwise, the same options can be passed to Engine.Begin and Do to override a default for a single call.

func WithCodec ¶

func WithCodec(codec Codec) Option

WithCodec sets how Do encodes results for storage. The default is JSON. It only has an effect when passed to New.

func WithFingerprint ¶

func WithFingerprint(fp string) Option

WithFingerprint attaches a digest of the request to the key. A later call that presents the same key with a different fingerprint fails with ErrFingerprintMismatch instead of receiving the result of other work. An empty fingerprint is never compared. See Fingerprint.

Example ¶

A key identifies one request. Presenting it with a different request is rejected instead of answered with the wrong result.

package main

import (
	"context"
	"fmt"

	"github.com/udaykishore-resu/idem"
	"github.com/udaykishore-resu/idem/memstore"
)

func main() {
	ctx := context.Background()
	engine := idem.New(memstore.New())
	transfer := func(ctx context.Context) (string, error) { return "ok", nil }

	original := idem.Fingerprint([]byte("transfer"), []byte(`{"to":"alice","amount":100}`))
	_, _, err := idem.Do(ctx, engine, "key-1", transfer, idem.WithFingerprint(original))
	fmt.Println("original:", err)

	changed := idem.Fingerprint([]byte("transfer"), []byte(`{"to":"mallory","amount":100}`))
	_, _, err = idem.Do(ctx, engine, "key-1", transfer, idem.WithFingerprint(changed))
	fmt.Println("changed: ", err)

}

Output:
original: <nil>
changed:  idem: key was already used with a different request

func WithHeartbeat ¶

func WithHeartbeat(d time.Duration) Option

WithHeartbeat sets how often the lease is extended while an operation runs. The default is a third of the lease. A value of zero or less turns the heartbeat off; the operation must then finish within one lease. An interval that is not shorter than the lease could never keep it alive and is replaced by the default.

func WithLease ¶

func WithLease(d time.Duration) Option

WithLease sets how long a key stays claimed when its holder stops sending heartbeats. It bounds how long a crashed process can block retries. The default is DefaultLease. WithLease panics if d is not positive.

func WithNamespace ¶

func WithNamespace(ns string) Option

WithNamespace prefixes every key with ns and a colon, so that several engines can share one store. It only has an effect when passed to New.

func WithTTL ¶

func WithTTL(d time.Duration) Option

WithTTL sets how long a completed result is kept and replayed. After it expires the key can be used for new work. The default is DefaultTTL. WithTTL panics if d is not positive.

func WithWait ¶

func WithWait(d time.Duration) Option

WithWait makes a call wait up to d for an operation that is in progress under the same key, instead of failing with ErrInProgress at once. If the other operation completes in time, its result is replayed; if it fails and releases the key, this call runs the operation itself. The default is not to wait.

Example ¶

By default a caller that finds the key held gets ErrInProgress at once. With WithWait it waits for the holder and takes over its result.

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/udaykishore-resu/idem"
	"github.com/udaykishore-resu/idem/memstore"
)

func main() {
	ctx := context.Background()
	engine := idem.New(memstore.New())

	started := make(chan struct{})
	go func() {
		_, _, _ = idem.Do(ctx, engine, "report-2024", func(ctx context.Context) (string, error) {
			close(started)
			time.Sleep(50 * time.Millisecond)
			return "report built by the first caller", nil
		})
	}()
	<-started

	build := func(ctx context.Context) (string, error) { return "report built by the second caller", nil }

	_, _, err := idem.Do(ctx, engine, "report-2024", build)
	fmt.Println("without waiting:", err)

	v, replayed, err := idem.Do(ctx, engine, "report-2024", build, idem.WithWait(5*time.Second))
	fmt.Println("with waiting:", v, replayed, err)

}

Output:
without waiting: idem: an operation with this key is already in progress
with waiting: report built by the first caller true <nil>

type State ¶

type State int

State is what Store.Acquire found under a key.

const (
	// StateAcquired means the key was free and the caller now holds it.
	StateAcquired State = iota + 1
	// StateInProgress means another caller holds an unexpired lease.
	StateInProgress
	// StateCompleted means the operation already finished and its result is
	// stored.
	StateCompleted
)

func (State) String ¶

func (s State) String() string

String returns the state's name.

type Store ¶

type Store interface {
	// Acquire claims key if it has no live record: it stores an in-progress
	// record holding token and fingerprint that expires after lease, and
	// reports StateAcquired.
	//
	// If key has a live record, Acquire leaves it unchanged and reports
	// StateInProgress or StateCompleted together with the stored fingerprint
	// and, for a completed record, the stored result.
	//
	// When several callers race for a free key, exactly one acquires it.
	Acquire(ctx context.Context, key, token, fingerprint string, lease time.Duration) (Claim, error)

	// Complete turns the live in-progress record held by token into a
	// completed record that stores result and expires after ttl.
	//
	// Calling Complete again with the same token on the completed record is a
	// no-op that returns nil, so the call can be retried. In every other case
	// Complete changes nothing and returns ErrLeaseLost.
	Complete(ctx context.Context, key, token string, result []byte, ttl time.Duration) error

	// Release removes the live in-progress record held by token, so that the
	// key can be acquired again. In every other case it changes nothing and
	// returns ErrLeaseLost.
	Release(ctx context.Context, key, token string) error

	// Extend moves the expiry of the live in-progress record held by token to
	// lease from now. In every other case it changes nothing and returns
	// ErrLeaseLost.
	Extend(ctx context.Context, key, token string, lease time.Duration) error
}

Store keeps one record per key. Implementations must be safe for concurrent use, and every method must be atomic with respect to the others, including across processes that share the store.

A record is live until its expiry time. An expired record must behave exactly as if it were absent.

Package storetest verifies an implementation against this contract.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
examples
orders command Command orders is a small HTTP service that shows idem at work.	Command orders is a small HTTP service that shows idem at work.
idemhttp Package idemhttp makes HTTP handlers idempotent with the Idempotency-Key request header.	Package idemhttp makes HTTP handlers idempotent with the Idempotency-Key request header.
memstore Package memstore provides an in-memory store for package idem.	Package memstore provides an in-memory store for package idem.
pgstore Package pgstore provides a PostgreSQL store for package idem, with transactional completion.	Package pgstore provides a PostgreSQL store for package idem, with transactional completion.
redisstore Package redisstore provides a Redis store for package idem.	Package redisstore provides a Redis store for package idem.
storetest Package storetest is a conformance suite for implementations of idem.Store.	Package storetest is a conformance suite for implementations of idem.Store.

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL