disruptor

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 11, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

README

lib-disruptor

Go Reference Release CI codecov License

A generic, standard-library-only Go implementation of the core LMAX Disruptor protocol: reusable preallocated events, monotonic sequences, bounded backpressure, consumer barriers, publication-gap detection, and batch handling.

This is an independent Go implementation. It does not bind to the Java library, wrap a channel, use cgo, or use unsafe. Version 0.1.0 requires Go 1.25 or newer and may change before a stable release.

Install

Install the current release explicitly so builds remain reproducible:

go get github.com/ayeshLK/lib-disruptor@v0.1.0

Import the root package as disruptor:

import disruptor "github.com/ayeshLK/lib-disruptor"

Because this is a pre-v1 module, review the changelog before upgrading to a new minor version.

Quick start

ring, err := disruptor.New(
    1024,
    disruptor.SingleProducer,
    func() *OrderEvent { return new(OrderEvent) },
    disruptor.BlockingWait(),
)
if err != nil {
    return err
}

processor, err := disruptor.NewBatchProcessor(
    ring,
    ring.NewBarrier(),
    func(event *OrderEvent, sequence int64, endOfBatch bool) error {
        log.Printf("sequence=%d order=%d", sequence, event.OrderID)
        return nil
    },
)
if err != nil {
    return err
}

ring.AddGatingSequences(processor.Sequence())
go processor.Run(context.Background())

return ring.Publish(context.Background(), func(event *OrderEvent, _ int64) error {
    event.OrderID = 42
    return nil
})

See examples/basic for the complete lifecycle.

Lower-level claims

high, err := ring.NextN(ctx, 4)
if err != nil {
    return err
}
low := high - 3
for sequence := low; sequence <= high; sequence++ {
    ring.Get(sequence).OrderID = nextOrderID()
}
ring.PublishRange(low, high)

TryNext, TryNextN, and TryPublish return immediately with ErrInsufficientCapacity instead of waiting for consumers.

Producer modes

  • SingleProducer avoids atomic claims. Exactly one goroutine may claim or publish for the lifetime of the ring.
  • MultiProducer uses CAS claims and per-slot lap flags. Consumers stop at the first unpublished gap even if later sequences have been published.

Use MultiProducer whenever publisher calls can overlap.

Consumer graphs

Parallel broadcast:

a, _ := disruptor.NewBatchProcessor(ring, ring.NewBarrier(), handlerA)
b, _ := disruptor.NewBatchProcessor(ring, ring.NewBarrier(), handlerB)
ring.AddGatingSequences(a.Sequence(), b.Sequence())

Pipeline A → B:

a, _ := disruptor.NewBatchProcessor(ring, ring.NewBarrier(), handlerA)
b, _ := disruptor.NewBatchProcessor(ring, ring.NewBarrier(a.Sequence()), handlerB)
ring.AddGatingSequences(b.Sequence())

Only terminal consumers should gate reuse. B cannot pass A, so gating on B protects both pipeline stages.

Wait strategies

Strategy Behavior Typical use
BlockingWait() Sleep until publication Default/shared hosts
SleepingWait() Spin, yield, then sleep Balanced CPU and latency
YieldingWait() Yield while waiting Busy systems with spare cores
BusySpinWait() Continuously check Dedicated cores, lowest jitter

Wait strategies govern consumers. Producers waiting for capacity yield to the Go scheduler.

Ownership and safety

  • Events are created once by EventFactory and reused.
  • A producer owns a claimed event until publication.
  • Consumers may access an event only while handling its sequence.
  • Never retain an event after the consumer sequence advances.
  • Downstream handlers may observe mutations made by upstream handlers.
  • Handler errors leave the current batch unacknowledged. Restarting that processor replays the batch, so restartable handlers should be idempotent.
  • Build and test applications with the race detector.

Go atomic publication and observation establish the visibility order. Go atomics are sequentially consistent.

Validation

go test ./...
go test -race ./...
go vet ./...
go run ./examples/basic

Microbenchmarks include an equivalent buffered-channel SPSC baseline:

go test -run='^$' -bench=. -benchmem ./...

Run concurrent load with sampled latency:

go run ./cmd/loadtest \
  -events=1000000 \
  -producers=1 \
  -consumers=1 \
  -ring-size=65536 \
  -batch-size=256

The load command prints JSON with publication and delivery throughput plus sampled p50/p95/p99 end-to-end latency.

Design notes

  • Sequences start at -1.
  • Physical index is uint64(sequence) & uint64(bufferSize-1).
  • A multi-producer availability flag is the sequence's lap number.
  • Gating lists use copy-on-write atomic snapshots.
  • Sequence pads atomic.Int64 without architecture-specific unsafe logic.
  • Assemble topology before publishing. A newly added runtime gate starts at the current claim cursor and does not replay older events.

Deferred from v0.1: fluent topology DSL, worker pools, replaying dynamic graph changes, CPU affinity, unsafe padding, persistence, and cross-process delivery.

Project policies

See CONTRIBUTING.md for validation and release conventions, SECURITY.md for private vulnerability reporting, and BENCHMARKS.md for dated local performance measurements.

License

Copyright 2026 Ayesh Almeida. Licensed under the Apache License, Version 2.0. See LICENSE.

Documentation

Overview

Package disruptor provides an in-process, bounded event exchange based on a preallocated ring buffer and monotonically increasing sequences.

Producers claim a sequence, mutate its preallocated event, and publish the sequence. Consumers wait through a SequenceBarrier, process available events in batches, and advance their own Sequence. The slowest registered gating sequence prevents producers from overwriting unread entries.

Index

Constants

View Source
const InitialSequence int64 = -1

InitialSequence is the logical position before the first ring entry.

Variables

View Source
var (
	// ErrInvalidBufferSize indicates that a ring size is not a positive power of two.
	ErrInvalidBufferSize = errors.New("disruptor: buffer size must be a positive power of two")
	// ErrInvalidProducerType indicates that the selected producer mode is unknown.
	ErrInvalidProducerType = errors.New("disruptor: invalid producer type")
	// ErrInvalidClaimSize indicates that a batch claim is outside the ring bounds.
	ErrInvalidClaimSize = errors.New("disruptor: claim size must be between one and the buffer size")
	// ErrInsufficientCapacity indicates that a non-blocking claim would overtake a gate.
	ErrInsufficientCapacity = errors.New("disruptor: insufficient capacity")
	// ErrNilFactory indicates that New received no event factory.
	ErrNilFactory = errors.New("disruptor: event factory must not be nil")
	// ErrNilTranslator indicates that Publish or TryPublish received no translator.
	ErrNilTranslator = errors.New("disruptor: event translator must not be nil")
	// ErrNilHandler indicates that NewBatchProcessor received no event handler.
	ErrNilHandler = errors.New("disruptor: event handler must not be nil")
	// ErrInvalidBatchSize indicates that a processor batch limit is not positive.
	ErrInvalidBatchSize = errors.New("disruptor: batch size must be positive")
	// ErrAlerted indicates that a barrier alert interrupted a wait.
	ErrAlerted = errors.New("disruptor: barrier alerted")
	// ErrClosed indicates that the ring was closed before an operation completed.
	ErrClosed = errors.New("disruptor: ring buffer closed")
	// ErrAlreadyRunning indicates that Run was called on an active processor.
	ErrAlreadyRunning = errors.New("disruptor: processor already running")
)

Functions

This section is empty.

Types

type BatchProcessor

type BatchProcessor[T any] struct {
	// contains filtered or unexported fields
}

BatchProcessor waits on a barrier, handles all currently available events in order, then advances its consumer sequence once per batch.

func NewBatchProcessor

func NewBatchProcessor[T any](ring *RingBuffer[T], barrier *SequenceBarrier, handler EventHandler[T], options ...ProcessorOption) (*BatchProcessor[T], error)

NewBatchProcessor creates an ordered consumer for ring using barrier.

func (*BatchProcessor[T]) Halt

func (p *BatchProcessor[T]) Halt()

Halt alerts the barrier and causes the active Run call to return.

func (*BatchProcessor[T]) Run

func (p *BatchProcessor[T]) Run(ctx context.Context) error

Run processes events until Halt is called, the context is cancelled, or a handler returns an error. On handler error the current batch is not acknowledged, so restarting the processor replays that batch.

func (*BatchProcessor[T]) Running

func (p *BatchProcessor[T]) Running() bool

Running reports whether Run currently owns the processor lifecycle.

func (*BatchProcessor[T]) Sequence

func (p *BatchProcessor[T]) Sequence() *Sequence

Sequence returns the processor's last fully acknowledged batch position.

type BlockingWaitStrategy

type BlockingWaitStrategy struct {
	// contains filtered or unexported fields
}

BlockingWaitStrategy sleeps consumers until a producer publishes. After the producer cursor passes the desired sequence, dependency waiting yields.

type BusySpinWaitStrategy

type BusySpinWaitStrategy struct{}

BusySpinWaitStrategy continuously polls a dependency and consumes a CPU core.

type EventFactory

type EventFactory[T any] func() T

EventFactory creates one reusable event for each physical ring slot.

type EventHandler

type EventHandler[T any] func(event T, sequence int64, endOfBatch bool) error

EventHandler processes one published event in sequence order.

type EventTranslator

type EventTranslator[T any] func(T, int64) error

EventTranslator initializes a claimed event before its sequence is published.

type ProcessorOption

type ProcessorOption func(*processorConfig) error

ProcessorOption configures a BatchProcessor.

func WithMaxBatchSize

func WithMaxBatchSize(size int64) ProcessorOption

WithMaxBatchSize limits the number of events acknowledged as one batch.

type ProducerType

type ProducerType uint8

ProducerType selects the claim and publication protocol.

const (
	// SingleProducer selects the non-atomic claim path for exactly one producer goroutine.
	SingleProducer ProducerType = iota
	// MultiProducer selects the CAS claim path for concurrent producer goroutines.
	MultiProducer
)

type RingBuffer

type RingBuffer[T any] struct {
	// contains filtered or unexported fields
}

RingBuffer is a preallocated, power-of-two store coordinated by logical sequences. The same physical event is reused on every lap.

A handler must not retain or mutate an event after it advances its consumer sequence. Prefer a pointer type for T when callers need in-place translation.

func New

func New[T any](size int64, producerType ProducerType, factory EventFactory[T], wait WaitStrategy) (*RingBuffer[T], error)

New creates a ring buffer with preallocated events and the selected producer mode.

func (*RingBuffer[T]) AddGatingSequences

func (r *RingBuffer[T]) AddGatingSequences(sequences ...*Sequence)

AddGatingSequences prevents producers from wrapping over the supplied consumers.

func (*RingBuffer[T]) BufferSize

func (r *RingBuffer[T]) BufferSize() int64

BufferSize returns the fixed number of physical ring slots.

func (*RingBuffer[T]) Close

func (r *RingBuffer[T]) Close()

Close unblocks waiters and prevents further claims.

func (*RingBuffer[T]) Cursor

func (r *RingBuffer[T]) Cursor() int64

Cursor returns the producer cursor.

func (*RingBuffer[T]) Get

func (r *RingBuffer[T]) Get(sequence int64) T

Get returns the reusable event stored at sequence's physical ring slot.

func (*RingBuffer[T]) NewBarrier

func (r *RingBuffer[T]) NewBarrier(dependencies ...*Sequence) *SequenceBarrier

NewBarrier creates a consumer barrier over the producer cursor or dependencies.

func (*RingBuffer[T]) Next

func (r *RingBuffer[T]) Next(ctx context.Context) (int64, error)

Next blocks until one sequence can be claimed or ctx is cancelled.

func (*RingBuffer[T]) NextN

func (r *RingBuffer[T]) NextN(ctx context.Context, count int64) (int64, error)

NextN blocks until count contiguous sequences can be claimed and returns the highest.

func (*RingBuffer[T]) Publish

func (r *RingBuffer[T]) Publish(ctx context.Context, translator EventTranslator[T]) (err error)

Publish claims one sequence, invokes translator on its preallocated event, and publishes the sequence even when translator returns an error.

func (*RingBuffer[T]) PublishRange

func (r *RingBuffer[T]) PublishRange(low, high int64)

PublishRange makes every claimed sequence from low through high visible.

func (*RingBuffer[T]) PublishSequence

func (r *RingBuffer[T]) PublishSequence(sequence int64)

PublishSequence makes one claimed sequence visible to consumers.

func (*RingBuffer[T]) RemainingCapacity

func (r *RingBuffer[T]) RemainingCapacity() int64

RemainingCapacity returns the number of sequences claimable without overtaking a gate.

func (*RingBuffer[T]) RemoveGatingSequence

func (r *RingBuffer[T]) RemoveGatingSequence(sequence *Sequence) bool

RemoveGatingSequence removes a gate and reports whether it was registered.

func (*RingBuffer[T]) TryNext

func (r *RingBuffer[T]) TryNext() (int64, error)

TryNext claims one sequence without blocking.

func (*RingBuffer[T]) TryNextN

func (r *RingBuffer[T]) TryNextN(count int64) (int64, error)

TryNextN claims count contiguous sequences without blocking and returns the highest.

func (*RingBuffer[T]) TryPublish

func (r *RingBuffer[T]) TryPublish(translator EventTranslator[T]) (err error)

TryPublish is the non-blocking form of Publish.

type Sequence

type Sequence struct {
	// contains filtered or unexported fields
}

Sequence is a padded, atomically updated logical position. A Sequence must not be copied after first use; pass it by pointer.

func NewSequence

func NewSequence(initial int64) *Sequence

NewSequence creates an atomic sequence at initial.

func (*Sequence) Add

func (s *Sequence) Add(delta int64) int64

Add atomically adds delta and returns the new sequence.

func (*Sequence) CompareAndSwap

func (s *Sequence) CompareAndSwap(old, new int64) bool

CompareAndSwap atomically replaces old with new when the current value equals old.

func (*Sequence) Load

func (s *Sequence) Load() int64

Load atomically returns the current sequence.

func (*Sequence) Store

func (s *Sequence) Store(value int64)

Store atomically replaces the current sequence.

type SequenceBarrier

type SequenceBarrier struct {
	// contains filtered or unexported fields
}

SequenceBarrier coordinates a consumer with the producer cursor and optional upstream consumer sequences.

func (*SequenceBarrier) Alert

func (b *SequenceBarrier) Alert()

Alert unblocks current waiters with ErrAlerted.

func (*SequenceBarrier) ClearAlert

func (b *SequenceBarrier) ClearAlert()

ClearAlert permits the barrier to wait again.

func (*SequenceBarrier) Cursor

func (b *SequenceBarrier) Cursor() int64

Cursor returns the slowest dependency position observed by the barrier.

func (*SequenceBarrier) IsAlerted

func (b *SequenceBarrier) IsAlerted() bool

IsAlerted reports whether the barrier is currently alerted.

func (*SequenceBarrier) WaitFor

func (b *SequenceBarrier) WaitFor(ctx context.Context, desired int64) (int64, error)

WaitFor blocks until desired is available, the context is cancelled, or the barrier is alerted. Multi-producer barriers never cross a publication gap.

type Sequencer

type Sequencer interface {
	Next(context.Context, int64) (int64, error)
	TryNext(int64) (int64, error)
	Publish(int64, int64)
	IsAvailable(int64) bool
	HighestPublished(int64, int64) int64
	Cursor() *Sequence
	BufferSize() int64
	RemainingCapacity() int64
	AddGatingSequences(...*Sequence)
	RemoveGatingSequence(*Sequence) bool
	NewBarrier(...*Sequence) *SequenceBarrier
	Close()
}

Sequencer coordinates producer claims, publication, capacity, and consumer gates.

type SleepingWaitStrategy

type SleepingWaitStrategy struct {
	// SpinTries is the number of polling iterations before yielding.
	SpinTries int
	// YieldTries is the number of scheduler yields before sleeping.
	YieldTries int
	// Sleep is the duration of each sleep after spinning and yielding.
	Sleep time.Duration
}

SleepingWaitStrategy spins, yields, and then sleeps while awaiting publication.

type WaitStrategy

type WaitStrategy interface {
	// contains filtered or unexported methods
}

WaitStrategy controls how consumers wait for a sequence to become visible. Implementations must be safe for concurrent use by multiple barriers.

func BlockingWait

func BlockingWait() WaitStrategy

BlockingWait returns a strategy that sleeps until producers signal publication.

func BusySpinWait

func BusySpinWait() WaitStrategy

BusySpinWait returns a strategy intended for consumers on dedicated cores.

func SleepingWait

func SleepingWait() WaitStrategy

SleepingWait returns a balanced wait strategy with default retry counts.

func YieldingWait

func YieldingWait() WaitStrategy

YieldingWait returns a scheduler-yielding wait strategy.

type YieldingWaitStrategy

type YieldingWaitStrategy struct{}

YieldingWaitStrategy yields to the scheduler between publication checks.

Directories

Path Synopsis
cmd
loadtest command
examples
basic command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL