idempotency

README

GoPotency

A flexible, framework-agnostic Go package for handling idempotency in HTTP APIs.

🎯 Features

• ✅ Framework Agnostic: Works with Gin, standard net/http, Echo, and more.
• ✅ Multiple Storage Backends: In-memory, Redis, SQL, and GORM support.
• ✅ Database Agnostic: Use any DB with GORM (PostgreSQL, MySQL, SQL Server, SQLite).
• ✅ Distributed Locking: Built-in support for multiple instances.
• ✅ Production Ready: Comprehensive testing, benchmarks, and CI/CD.

📦 Installation

go get github.com/fco-gt/gopotency

🚀 Quick Start

With Gin

package main

import (
    "github.com/fco-gt/gopotency"
    ginmw "github.com/fco-gt/gopotency/middleware/gin"
    "github.com/fco-gt/gopotency/storage/memory"
    "github.com/gin-gonic/gin"
    "time"
)

func main() {
    store := memory.NewMemoryStorage()
    manager, _ := idempotency.NewManager(idempotency.Config{
        Storage: store,
        TTL:     24 * time.Hour,
    })

    r := gin.Default()
    r.Use(ginmw.Idempotency(manager))

    r.POST("/orders", func(c *gin.Context) {
        c.JSON(201, gin.H{"order_id": "ORD-123", "status": "created"})
    })

    r.Run(":8080")
}

With Standard HTTP

package main

import (
    "github.com/fco-gt/gopotency"
    httpmw "github.com/fco-gt/gopotency/middleware/http"
    "github.com/fco-gt/gopotency/storage/memory"
    "net/http"
    "time"
)

func main() {
    store := memory.NewMemoryStorage()
    manager, _ := idempotency.NewManager(idempotency.Config{
        Storage: store,
        TTL:     24 * time.Hour,
    })

    mux := http.NewServeMux()
    handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte(`{"status": "processed"}`))
    })

    mux.Handle("/process", httpmw.Idempotency(manager)(handler))
    http.ListenAndServe(":8080", mux)
}

📖 Documentation

Configuration Options

type Config struct {
    Storage        Storage       // Required: Memory, Redis, SQL, or GORM
    TTL            time.Duration // Default: 24h
    LockTimeout    time.Duration // Default: 5m
    KeyStrategy    KeyStrategy   // Default: HeaderBased("Idempotency-Key")
    AllowedMethods []string      // Default: ["POST", "PUT", "PATCH", "DELETE"]
    RequireKey     bool          // If true, returns 400 if key is missing (Default: false)
    ErrorHandler   func(error) (int, any)
}

Route-Specific Middleware

GoPotency allows you to be granular. If you provide an Idempotency-Key in the request, the middleware will process it regardless of the method.

For critical routes, you can enable RequireKey: true to ensure no one accidentally skips idempotency.

Storage Backends

In-Memory (Dev/Single Instance)

import "github.com/fco-gt/gopotency/storage/memory"
store := memory.NewMemoryStorage()

Redis (Distributed)

import "github.com/fco-gt/gopotency/storage/redis"
store, err := redis.NewRedisStorage(ctx, "localhost:6379", "password")

GORM (Database Agnostic)

import (
    idempotencyGorm "github.com/fco-gt/gopotency/storage/gorm"
    "gorm.io/gorm"
)
store := idempotencyGorm.NewGormStorage(db)

SQL (Postgres/SQLite)

import idempotencySQL "github.com/fco-gt/gopotency/storage/sql"
store := idempotencySQL.NewSQLStorage(db, "idempotency_records")

�️ Development

We use a Makefile to streamline development:

make test    # Run all tests
make bench   # Run performance benchmarks
make build   # Build all examples

📊 Benchmarks

GoPotency is optimized for high-performance APIs.

Operation	Time
Idempotency Check	~520 ns/op
Full Flow (Lock/Store)	~1500 ns/op

🤝 Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📄 License

Distributed under the MIT License. See LICENSE for more information.

📚 Examples

• Gin Basic
• HTTP Basic
• Redis Basic
• SQL Basic
• GORM Basic

Documentation

Overview

Package idempotency provides a flexible, framework-agnostic solution for handling idempotency in HTTP APIs.

Idempotency ensures that performing the same request multiple times produces the same result as performing it once, without additional side effects. This is crucial for payment processing, resource creation, and other critical operations.

Quick Start

Create a manager with in-memory storage:

import (
	"github.com/fco-gt/gopotency"
	"github.com/fco-gt/gopotency/storage/memory"
	"time"
)

store := memory.NewMemoryStorage()
manager, err := idempotency.NewManager(idempotency.Config{
	Storage: store,
	TTL:     24 * time.Hour,
})

Use with Gin:

import ginmw "github.com/fco-gt/gopotency/middleware/gin"

router := gin.Default()
router.Use(ginmw.Idempotency(manager))

Use with standard library:

import httpmw "github.com/fco-gt/gopotency/middleware/http"

mux := http.NewServeMux()
handler := httpmw.Idempotency(manager)(yourHandler)

Key Strategies

The package supports multiple key generation strategies:

• HeaderBased: Extracts key from request header (default: "Idempotency-Key")
• BodyHash: Generates key from request content hash
• Composite: Tries header first, falls back to body hash

Storage Backends

• memory: In-memory storage (development/testing)
• redis: Redis-backed storage (coming soon)

Configuration

Customize behavior with Config options:

config := idempotency.Config{
	Storage:        store,
	TTL:            24 * time.Hour,
	LockTimeout:    5 * time.Minute,
	KeyStrategy:    key.HeaderBased("Idempotency-Key"),
	AllowedMethods: []string{"POST", "PUT", "PATCH", "DELETE"},
}

Index

• Constants
• Variables
• func NewStorageError(operation string, err error) error
• type CachedResponse
• type Config
• type KeyStrategy
• type Manager
  • func NewManager(config Config) (*Manager, error)
  • func (m *Manager) Check(ctx context.Context, req *Request) (*CachedResponse, error)
  • func (m *Manager) Close() error
  • func (m *Manager) Config() Config
  • func (m *Manager) IsMethodAllowed(method string) bool
  • func (m *Manager) Lock(ctx context.Context, req *Request) error
  • func (m *Manager) Store(ctx context.Context, key string, resp *Response) error
  • func (m *Manager) Unlock(ctx context.Context, key string) error
• type Record
• type RecordStatus
• type Request
• type RequestHasher
• type Response
  • func (r *Response) ToCachedResponse() *CachedResponse
• type Storage
• type StorageError
  • func (e *StorageError) Error() string
  • func (e *StorageError) Unwrap() error

Constants

const (
	// Version is the package version
	Version = "1.0.0"

	// DefaultHeaderName is the default header name for idempotency keys
	DefaultHeaderName = "Idempotency-Key"
)

Variables

var (
	// ErrStorageNotConfigured is returned when no storage backend is configured
	ErrStorageNotConfigured = errors.New("idempotency: storage backend not configured")

	// ErrKeyGenerationFailed is returned when the key generator fails to generate a key
	ErrKeyGenerationFailed = errors.New("idempotency: failed to generate idempotency key")

	// ErrRequestInProgress is returned when a request with the same idempotency key is already being processed
	ErrRequestInProgress = errors.New("idempotency: request with this idempotency key is already in progress")

	// ErrInvalidConfiguration is returned when the configuration is invalid
	ErrInvalidConfiguration = errors.New("idempotency: invalid configuration")

	// ErrRequestMismatch is returned when a duplicate request has different content
	ErrRequestMismatch = errors.New("idempotency: request with same key has different content")

	// ErrStorageOperation is returned when a storage operation fails
	ErrStorageOperation = errors.New("idempotency: storage operation failed")

	// ErrLockTimeout is returned when unable to acquire a lock within the timeout period
	ErrLockTimeout = errors.New("idempotency: lock acquisition timeout")

	// ErrNoIdempotencyKey is returned when no idempotency key could be extracted or generated
	ErrNoIdempotencyKey = errors.New("idempotency: no idempotency key found or generated")
)

Common errors returned by the idempotency package

Functions

func NewStorageError

func NewStorageError(operation string, err error) error

NewStorageError creates a new storage error

Types

type CachedResponse

type CachedResponse struct {
	// StatusCode is the HTTP status code
	StatusCode int

	// Headers are the response headers
	Headers map[string][]string

	// Body is the response body
	Body []byte

	// ContentType is the content type of the response
	ContentType string
}

CachedResponse represents a cached HTTP response

type Config

type Config struct {
	// Storage is the backend storage implementation (required)
	Storage Storage

	// TTL is the time-to-live for idempotency records
	// Default: 24 hours
	TTL time.Duration

	// LockTimeout is the maximum time a lock can be held
	// This prevents deadlocks if a server crashes while processing
	// Default: 5 minutes
	LockTimeout time.Duration

	// KeyStrategy is the strategy for generating idempotency keys
	// Default: HeaderBased("Idempotency-Key")
	KeyStrategy KeyStrategy

	// RequestHasher computes a hash of the request for validation
	// Default: BodyHasher
	RequestHasher RequestHasher

	// AllowedMethods specifies which HTTP methods should have idempotency applied
	// Default: ["POST", "PUT", "PATCH", "DELETE"]
	AllowedMethods []string

	// ErrorHandler is called when an error occurs, allowing custom error responses
	// Default: returns standard error responses
	ErrorHandler func(error) (statusCode int, body any)

	// OnCacheHit is called when a cached response is returned (optional)
	OnCacheHit func(key string)

	// OnCacheMiss is called when no cached response is found (optional)
	OnCacheMiss func(key string)

	// OnLockConflict is called when a request is already in progress (optional)
	OnLockConflict func(key string)

	// RequireKey if true, the middleware will return an error if the idempotency key is missing
	// for an allowed method/route.
	// Default: false
	RequireKey bool
}

Config holds the configuration for the idempotency manager

type KeyStrategy

type KeyStrategy interface {
	// Generate generates an idempotency key from the request
	// Returns empty string if no key can be generated
	Generate(req *Request) (string, error)
}

KeyStrategy is the interface for generating idempotency keys

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager handles idempotency checks and response caching

func NewManager

func NewManager(config Config) (*Manager, error)

NewManager creates a new idempotency manager with the given configuration

func (*Manager) Check

func (m *Manager) Check(ctx context.Context, req *Request) (*CachedResponse, error)

Check verifies if a request should be processed or if a cached response exists Returns: - *CachedResponse: if the request was already processed successfully - error: ErrRequestInProgress if currently being processed, or other errors

func (*Manager) Close

func (m *Manager) Close() error

Close closes the manager and underlying storage

func (*Manager) Config

func (m *Manager) Config() Config

Config returns the manager's configuration (read-only)

func (*Manager) IsMethodAllowed

func (m *Manager) IsMethodAllowed(method string) bool

IsMethodAllowed checks if idempotency should be applied to the given HTTP method

func (*Manager) Lock

func (m *Manager) Lock(ctx context.Context, req *Request) error

Lock attempts to acquire a lock for processing the request

func (*Manager) Store

func (m *Manager) Store(ctx context.Context, key string, resp *Response) error

Store saves the response for a successfully processed request

func (*Manager) Unlock

func (m *Manager) Unlock(ctx context.Context, key string) error

Unlock releases the lock for a request (typically called on error)

type Record

type Record struct {
	// Key is the idempotency key
	Key string

	// RequestHash is a hash of the request content for validation
	RequestHash string

	// Status is the current status of the request
	Status RecordStatus

	// Response contains the cached response if status is completed
	Response *CachedResponse

	// CreatedAt is when the record was created
	CreatedAt time.Time

	// ExpiresAt is when the record should expire
	ExpiresAt time.Time
}

Record represents a stored idempotency record

type RecordStatus

type RecordStatus string

RecordStatus represents the status of an idempotency record

const (
	// StatusPending indicates a request is currently being processed
	StatusPending RecordStatus = "pending"

	// StatusCompleted indicates a request has been successfully processed
	StatusCompleted RecordStatus = "completed"

	// StatusFailed indicates a request processing failed
	StatusFailed RecordStatus = "failed"
)

type Request

type Request struct {
	// Method is the HTTP method (GET, POST, etc.)
	Method string

	// Path is the request path
	Path string

	// Headers are the request headers
	Headers map[string][]string

	// Body is the request body
	Body []byte

	// IdempotencyKey is the extracted or generated idempotency key
	IdempotencyKey string
}

Request represents an incoming HTTP request for idempotency checking

type RequestHasher

type RequestHasher interface {
	// Hash computes a hash of the request for validation
	Hash(req *Request) (string, error)
}

RequestHasher is the interface for hashing requests

type Response

type Response struct {
	// StatusCode is the HTTP status code
	StatusCode int

	// Headers are the response headers
	Headers map[string][]string

	// Body is the response body
	Body []byte

	// ContentType is the content type of the response
	ContentType string
}

Response represents an HTTP response to be cached

func (*Response) ToCachedResponse

func (r *Response) ToCachedResponse() *CachedResponse

ToCachedResponse converts a Response to a CachedResponse

type Storage

type Storage interface {
	// Get retrieves an idempotency record by key
	Get(ctx context.Context, key string) (*Record, error)

	// Set stores an idempotency record
	Set(ctx context.Context, record *Record, ttl time.Duration) error

	// Delete removes an idempotency record
	Delete(ctx context.Context, key string) error

	// Exists checks if a record exists
	Exists(ctx context.Context, key string) (bool, error)

	// TryLock attempts to acquire a lock for the given key
	// Returns true if lock was acquired, false if already locked
	TryLock(ctx context.Context, key string, ttl time.Duration) (bool, error)

	// Unlock releases a lock for the given key
	Unlock(ctx context.Context, key string) error

	// Close closes the storage connection
	Close() error
}

Storage is the interface for storing and retrieving idempotency records

type StorageError

type StorageError struct {
	Operation string
	Err       error
}

StorageError wraps errors from storage operations

func (*StorageError) Error

func (e *StorageError) Error() string

func (*StorageError) Unwrap

func (e *StorageError) Unwrap() error