disruptor

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 15, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

README

lib-disruptor

Go Reference Release CI codecov Apache-2.0 license

A generic, dependency-free Go implementation of the core LMAX Disruptor protocol for bounded, ordered event pipelines.

lib-disruptor uses a reusable, preallocated ring buffer to coordinate producers and ordered batch consumers. It gives applications explicit backpressure and topology control without cgo, unsafe, Java bindings, or a channel-backed substitute.

[!IMPORTANT] lib-disruptor is a pre-v1 module. Minor releases may include breaking changes; review the changelog before upgrading.

Why use it?

  • Reuse factory-allocated events instead of allocating on every handoff.
  • Choose single-producer or CAS-based multi-producer publication.
  • Build broadcast and staged consumer pipelines with explicit gating.
  • Select consumer and producer waiting policies for your latency and CPU budget.

This library is for in-process, performance-sensitive event processing. It is not a distributed queue, persistence layer, topology DSL, or general replacement for Go channels.

Install

Requires Go 1.25 or newer.

go get github.com/ayeshLK/lib-disruptor@v0.1.0

Import the root package as disruptor:

import disruptor "github.com/ayeshLK/lib-disruptor"

Quick start

The basic lifecycle is: create a ring, attach and gate a processor, publish events, then drain and close the ring.

ring, err := disruptor.New(
    1024,
    disruptor.SingleProducer,
    func() *OrderEvent { return new(OrderEvent) },
    disruptor.BlockingWait(),
)
if err != nil {
    return err
}

processor, err := disruptor.NewBatchProcessor(
    ring,
    ring.NewBarrier(),
    func(event *OrderEvent, sequence int64, endOfBatch bool) error {
        log.Printf("sequence=%d order=%d", sequence, event.OrderID)
        return nil
    },
)
if err != nil {
    return err
}

ring.AddGatingSequences(processor.Sequence())
go processor.Run(context.Background())

return ring.Publish(context.Background(), func(event *OrderEvent, _ int64) error {
    event.OrderID = 42
    return nil
})

See examples/basic for a complete, runnable example with orderly shutdown and processor-result handling.

Resolve raw claims

The Next and TryNext APIs transfer ownership of a claimed event to the caller. Resolve every successful claim with PublishSequence/PublishRange or DiscardSequence/DiscardRange, including failure and panic paths. Discarded sequences advance barriers and gating without invoking consumer handlers; use IsDiscarded when reading a raw barrier range. See the usage guide for a safe deferred-resolution pattern.

Choose your setup

Producer mode
Mode Use when Notes
SingleProducer One goroutine will claim and publish for the ring's lifetime Lowest claim overhead; do not call publishing APIs from another goroutine.
MultiProducer Publisher calls may overlap Uses CAS claims and preserves ordered consumer visibility across publication gaps.
Consumer wait strategy
Strategy Best fit
BlockingWait() Shared hosts and low CPU use
SleepingWait() A balance of CPU use and latency
YieldingWait() Busy systems with spare cores
BusySpinWait() Dedicated cores and lowest jitter

The consumer wait strategy is passed to New. Producer capacity waiting is configured separately with WithProducerWait; see the usage guide.

Model your consumer graph

Topology Barrier and gate arrangement
Broadcast producer -> A and producer -> B; gate both A and B.
Pipeline producer -> A -> B; create B's barrier from A.Sequence() and gate only terminal consumer B.

Gating sequences prevent a producer from overwriting an event before the terminal consumers have acknowledged it. Assemble a topology before publishing; a gate added later starts at the current claim cursor and does not replay old events.

Next steps

  • Read the usage guide for batch claims, producer capacity waits, graph setup, shutdown, error handling, and event-ownership rules.
  • Browse the API reference for the complete public contract.
  • See PERFORMANCE.md for the measurement model and BENCHMARKS.md for dated, machine-specific results.

Contributing and security

Contributions are welcome. Read CONTRIBUTING.md for the development and validation workflow, SECURITY.md for private vulnerability reporting, and CODE_OF_CONDUCT.md for community expectations.

License

Copyright 2026 Ayesh Almeida. Licensed under the Apache License, Version 2.0. See LICENSE.

Documentation

Overview

Package disruptor provides an in-process, bounded event exchange based on a preallocated ring buffer and monotonically increasing sequences.

Producers claim a sequence, mutate its preallocated event, and publish the sequence. Consumers wait through a SequenceBarrier, process available events in batches, and advance their own Sequence. The slowest registered gating sequence prevents producers from overwriting unread entries.

Index

Constants

View Source
const InitialSequence int64 = -1

InitialSequence is the logical position before the first ring entry.

Variables

View Source
var (
	// ErrInvalidBufferSize indicates that a ring size is not a positive power of two.
	ErrInvalidBufferSize = errors.New("disruptor: buffer size must be a positive power of two")
	// ErrInvalidProducerType indicates that the selected producer mode is unknown.
	ErrInvalidProducerType = errors.New("disruptor: invalid producer type")
	// ErrInvalidProducerWaitMode indicates that the selected producer wait mode is unknown.
	ErrInvalidProducerWaitMode = errors.New("disruptor: invalid producer wait mode")
	// ErrInvalidClaimSize indicates that a batch claim is outside the ring bounds.
	ErrInvalidClaimSize = errors.New("disruptor: claim size must be between one and the buffer size")
	// ErrInsufficientCapacity indicates that a non-blocking claim would overtake a gate.
	ErrInsufficientCapacity = errors.New("disruptor: insufficient capacity")
	// ErrNilFactory indicates that New received no event factory.
	ErrNilFactory = errors.New("disruptor: event factory must not be nil")
	// ErrNilTranslator indicates that Publish or TryPublish received no translator.
	ErrNilTranslator = errors.New("disruptor: event translator must not be nil")
	// ErrNilHandler indicates that NewBatchProcessor received no event handler.
	ErrNilHandler = errors.New("disruptor: event handler must not be nil")
	// ErrInvalidBatchSize indicates that a processor batch limit is not positive.
	ErrInvalidBatchSize = errors.New("disruptor: batch size must be positive")
	// ErrAlerted indicates that a barrier alert interrupted a wait.
	ErrAlerted = errors.New("disruptor: barrier alerted")
	// ErrClosed indicates that the ring was closed before an operation completed.
	ErrClosed = errors.New("disruptor: ring buffer closed")
	// ErrAlreadyRunning indicates that Run was called on an active processor.
	ErrAlreadyRunning = errors.New("disruptor: processor already running")
)

Functions

This section is empty.

Types

type BatchProcessor

type BatchProcessor[T any] struct {
	// contains filtered or unexported fields
}

BatchProcessor waits on a barrier, handles all currently available published events in order, skips discarded claims, then advances its consumer sequence once per batch.

func NewBatchProcessor

func NewBatchProcessor[T any](ring *RingBuffer[T], barrier *SequenceBarrier, handler EventHandler[T], options ...ProcessorOption) (*BatchProcessor[T], error)

NewBatchProcessor creates an ordered consumer for ring using barrier.

func (*BatchProcessor[T]) Halt

func (p *BatchProcessor[T]) Halt()

Halt alerts the barrier and causes the active Run call to return after its current handler invocation and selected batch complete. Halt is a no-op when the processor is idle.

func (*BatchProcessor[T]) Run

func (p *BatchProcessor[T]) Run(ctx context.Context) error

Run processes events until Halt is called, the context is cancelled, the ring is closed, or a handler returns an error. Ring closure returns ErrClosed. On handler error or panic the current batch is not acknowledged, so restarting the processor replays that batch. Handler failures are returned as *HandlerError and recovered handler panics as *HandlerPanicError.

func (*BatchProcessor[T]) Running

func (p *BatchProcessor[T]) Running() bool

Running reports whether Run is ready to process events or is finishing after Halt. It remains true after Halt until that Run call returns.

func (*BatchProcessor[T]) Sequence

func (p *BatchProcessor[T]) Sequence() *Sequence

Sequence returns the processor's last fully acknowledged batch position.

type BlockingWaitStrategy

type BlockingWaitStrategy struct {
	// contains filtered or unexported fields
}

BlockingWaitStrategy sleeps consumers until a producer publishes. After the producer cursor passes the desired sequence, dependency waiting yields.

type BusySpinWaitStrategy

type BusySpinWaitStrategy struct{}

BusySpinWaitStrategy continuously polls a dependency and consumes a CPU core.

type EventFactory

type EventFactory[T any] func() T

EventFactory creates one reusable event for each physical ring slot.

type EventHandler

type EventHandler[T any] func(event T, sequence int64, endOfBatch bool) error

EventHandler processes one published event in sequence order.

type EventTranslator

type EventTranslator[T any] func(T, int64) error

EventTranslator initializes a claimed event before its sequence is published.

type HandlerError added in v0.2.0

type HandlerError struct {
	Sequence int64
	Err      error
}

HandlerError reports an error returned by an event handler.

func (*HandlerError) Error added in v0.2.0

func (e *HandlerError) Error() string

func (*HandlerError) Unwrap added in v0.2.0

func (e *HandlerError) Unwrap() error

Unwrap returns the error returned by the handler.

type HandlerPanicError added in v0.2.0

type HandlerPanicError struct {
	Sequence int64
	Value    any
}

HandlerPanicError reports a panic recovered from an event handler.

func (*HandlerPanicError) Error added in v0.2.0

func (e *HandlerPanicError) Error() string

func (*HandlerPanicError) Unwrap added in v0.2.0

func (e *HandlerPanicError) Unwrap() error

Unwrap returns the panic value when that value implements error.

type ProcessorOption

type ProcessorOption func(*processorConfig) error

ProcessorOption configures a BatchProcessor.

func WithMaxBatchSize

func WithMaxBatchSize(size int64) ProcessorOption

WithMaxBatchSize limits the number of events acknowledged as one batch.

type ProducerType

type ProducerType uint8

ProducerType selects the claim and publication protocol.

const (
	// SingleProducer selects the non-atomic claim path for exactly one producer goroutine.
	SingleProducer ProducerType = iota
	// MultiProducer selects the CAS claim path for concurrent producer goroutines.
	MultiProducer
)

type ProducerWaitMode added in v0.2.0

type ProducerWaitMode uint8

ProducerWaitMode controls how a blocking producer claim waits for consumer gating sequences to release capacity.

const (
	// ProducerWaitYielding yields to the Go scheduler while capacity is unavailable.
	// It is the default and preserves the original producer wait behavior.
	ProducerWaitYielding ProducerWaitMode = iota
	// ProducerWaitBlocking sleeps until a gate advances, a gate is removed, the
	// context is cancelled, or the ring closes.
	ProducerWaitBlocking
	// ProducerWaitBusySpin continuously checks capacity and is intended for
	// latency-sensitive workloads with a dedicated CPU core.
	ProducerWaitBusySpin
)

type RingBuffer

type RingBuffer[T any] struct {
	// contains filtered or unexported fields
}

RingBuffer is a preallocated, power-of-two store coordinated by logical sequences. The same physical event is reused on every lap.

A handler must not retain or mutate an event after it advances its consumer sequence. Prefer a pointer type for T when callers need in-place translation.

func New

func New[T any](size int64, producerType ProducerType, factory EventFactory[T], wait WaitStrategy, options ...RingOption) (*RingBuffer[T], error)

New creates a ring buffer with preallocated events and the selected producer mode. Producer claims yield while waiting for capacity unless configured otherwise.

func (*RingBuffer[T]) AddGatingSequences

func (r *RingBuffer[T]) AddGatingSequences(sequences ...*Sequence)

AddGatingSequences prevents producers from wrapping over the supplied consumers.

func (*RingBuffer[T]) BufferSize

func (r *RingBuffer[T]) BufferSize() int64

BufferSize returns the fixed number of physical ring slots.

func (*RingBuffer[T]) Close

func (r *RingBuffer[T]) Close()

Close immediately prevents future claims and unblocks waits for unavailable sequences with ErrClosed. Use Shutdown to drain registered gating sequences before closing consumers.

func (*RingBuffer[T]) Cursor

func (r *RingBuffer[T]) Cursor() int64

Cursor returns the producer cursor.

func (*RingBuffer[T]) DiscardRange added in v0.3.0

func (r *RingBuffer[T]) DiscardRange(low, high int64)

DiscardRange resolves every claimed sequence from low through high without delivering their events to processors. Duplicate resolution calls are ignored; the first terminal resolution wins.

func (*RingBuffer[T]) DiscardSequence added in v0.3.0

func (r *RingBuffer[T]) DiscardSequence(sequence int64)

DiscardSequence resolves a claimed sequence without delivering its event to processors. Duplicate resolution calls are ignored; the first terminal resolution wins.

func (*RingBuffer[T]) Get

func (r *RingBuffer[T]) Get(sequence int64) T

Get returns the reusable event stored at sequence's physical ring slot.

func (*RingBuffer[T]) IsDiscarded added in v0.3.0

func (r *RingBuffer[T]) IsDiscarded(sequence int64) bool

IsDiscarded reports whether sequence was resolved as discarded in its current ring lap. Discarded sequences are visible to barriers but are skipped by BatchProcessor handlers.

func (*RingBuffer[T]) NewBarrier

func (r *RingBuffer[T]) NewBarrier(dependencies ...*Sequence) *SequenceBarrier

NewBarrier creates a consumer barrier over the producer cursor or dependencies.

func (*RingBuffer[T]) Next

func (r *RingBuffer[T]) Next(ctx context.Context) (int64, error)

Next blocks until one sequence can be claimed or ctx is cancelled.

func (*RingBuffer[T]) NextN

func (r *RingBuffer[T]) NextN(ctx context.Context, count int64) (int64, error)

NextN blocks until count contiguous sequences can be claimed and returns the highest.

func (*RingBuffer[T]) Publish

func (r *RingBuffer[T]) Publish(ctx context.Context, translator EventTranslator[T]) (err error)

Publish claims one sequence, invokes translator on its preallocated event, and publishes the sequence even when translator returns an error.

func (*RingBuffer[T]) PublishRange

func (r *RingBuffer[T]) PublishRange(low, high int64)

PublishRange makes every claimed sequence from low through high visible. Every successful raw claim must be resolved by either publishing or discarding it.

func (*RingBuffer[T]) PublishSequence

func (r *RingBuffer[T]) PublishSequence(sequence int64)

PublishSequence makes one claimed sequence visible to consumers. Every successful raw claim must be resolved by either publishing or discarding it.

func (*RingBuffer[T]) RemainingCapacity

func (r *RingBuffer[T]) RemainingCapacity() int64

RemainingCapacity returns the number of sequences claimable without overtaking a gate.

func (*RingBuffer[T]) RemoveGatingSequence

func (r *RingBuffer[T]) RemoveGatingSequence(sequence *Sequence) bool

RemoveGatingSequence removes a gate and reports whether it was registered.

func (*RingBuffer[T]) Shutdown added in v0.2.0

func (r *RingBuffer[T]) Shutdown(ctx context.Context) error

Shutdown prevents future claims, waits for the gating sequences present when it begins to acknowledge the final claimed sequence, and then closes consumer waits. Publisher operations must have returned before Shutdown is called. Shutdown returns a context error if draining does not complete; the ring is closed whether draining succeeds or fails.

func (*RingBuffer[T]) TryNext

func (r *RingBuffer[T]) TryNext() (int64, error)

TryNext claims one sequence without blocking.

func (*RingBuffer[T]) TryNextN

func (r *RingBuffer[T]) TryNextN(count int64) (int64, error)

TryNextN claims count contiguous sequences without blocking and returns the highest.

func (*RingBuffer[T]) TryPublish

func (r *RingBuffer[T]) TryPublish(translator EventTranslator[T]) (err error)

TryPublish is the non-blocking form of Publish.

type RingOption added in v0.2.0

type RingOption func(*ringConfig) error

RingOption configures a RingBuffer.

func WithProducerWait added in v0.2.0

func WithProducerWait(mode ProducerWaitMode) RingOption

WithProducerWait selects how blocking producer claims wait for capacity.

type Sequence

type Sequence struct {
	// contains filtered or unexported fields
}

Sequence is a padded, atomically updated logical position. A Sequence must not be copied after first use; pass it by pointer.

func NewSequence

func NewSequence(initial int64) *Sequence

NewSequence creates an atomic sequence at initial.

func (*Sequence) Add

func (s *Sequence) Add(delta int64) int64

Add atomically adds delta and returns the new sequence.

func (*Sequence) CompareAndSwap

func (s *Sequence) CompareAndSwap(old, new int64) bool

CompareAndSwap atomically replaces old with new when the current value equals old.

func (*Sequence) Load

func (s *Sequence) Load() int64

Load atomically returns the current sequence.

func (*Sequence) Store

func (s *Sequence) Store(value int64)

Store atomically replaces the current sequence.

type SequenceBarrier

type SequenceBarrier struct {
	// contains filtered or unexported fields
}

SequenceBarrier coordinates a consumer with the producer cursor and optional upstream consumer sequences.

func (*SequenceBarrier) Alert

func (b *SequenceBarrier) Alert()

Alert unblocks current waiters with ErrAlerted.

func (*SequenceBarrier) ClearAlert

func (b *SequenceBarrier) ClearAlert()

ClearAlert permits the barrier to wait again.

func (*SequenceBarrier) Cursor

func (b *SequenceBarrier) Cursor() int64

Cursor returns the slowest dependency position observed by the barrier.

func (*SequenceBarrier) IsAlerted

func (b *SequenceBarrier) IsAlerted() bool

IsAlerted reports whether the barrier is currently alerted.

func (*SequenceBarrier) WaitFor

func (b *SequenceBarrier) WaitFor(ctx context.Context, desired int64) (int64, error)

WaitFor blocks until desired is available, the context is cancelled, the barrier is alerted, or the ring is closed. Multi-producer barriers never cross a publication gap.

type Sequencer

type Sequencer interface {
	Next(context.Context, int64) (int64, error)
	TryNext(int64) (int64, error)
	Publish(int64, int64)
	IsAvailable(int64) bool
	HighestPublished(int64, int64) int64
	Cursor() *Sequence
	BufferSize() int64
	RemainingCapacity() int64
	AddGatingSequences(...*Sequence)
	RemoveGatingSequence(*Sequence) bool
	NewBarrier(...*Sequence) *SequenceBarrier
	Close()
	Shutdown(context.Context) error
}

Sequencer coordinates producer claims, publication, capacity, and consumer gates.

type SleepingWaitStrategy

type SleepingWaitStrategy struct {
	// SpinTries is the number of polling iterations before yielding.
	SpinTries int
	// YieldTries is the number of scheduler yields before sleeping.
	YieldTries int
	// Sleep is the duration of each sleep after spinning and yielding.
	Sleep time.Duration
}

SleepingWaitStrategy spins, yields, and then sleeps while awaiting publication.

type WaitStrategy

type WaitStrategy interface {
	// contains filtered or unexported methods
}

WaitStrategy controls how consumers wait for a sequence to become visible. Implementations must be safe for concurrent use by multiple barriers.

func BlockingWait

func BlockingWait() WaitStrategy

BlockingWait returns a strategy that sleeps until producers signal publication.

func BusySpinWait

func BusySpinWait() WaitStrategy

BusySpinWait returns a strategy intended for consumers on dedicated cores.

func SleepingWait

func SleepingWait() WaitStrategy

SleepingWait returns a balanced wait strategy with default retry counts.

func YieldingWait

func YieldingWait() WaitStrategy

YieldingWait returns a scheduler-yielding wait strategy.

type YieldingWaitStrategy

type YieldingWaitStrategy struct{}

YieldingWaitStrategy yields to the scheduler between publication checks.

Directories

Path Synopsis
cmd
loadtest command
examples
basic command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL