evt

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 16, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

README

evt

Event sourcing for Go, without the framework tax.

Documentation · Concepts · Getting Started · Invariants

evt gives you the parts of event sourcing that are genuinely hard to get right — append-only logs, optimistic concurrency, snapshots, replayable read models, and pluggable persistence (DynamoDB or PostgreSQL) — as small, composable Go packages. There is no runtime to adopt, no base application to inherit from, and no magic. You wire the pieces you need and ignore the rest.

It grew out of patterns we run in production.

Why evt

  • Events are the source of truth. Domain events are immutable facts. Every view, projection, and read model is derived state you can delete and rebuild by replaying the log.
  • The same aggregate runs everywhere. Write your command and event logic once. Test it against an in-memory store in microseconds, then point it at DynamoDB or PostgreSQL in production without touching the aggregate. A backend-neutral conformance suite holds every backend to the same storage contract.
  • Concurrency and ordering are handled. Conditional writes protect per-entity sequence ordering. Stable command IDs make retries safe instead of duplicating facts.
  • Rebuilds stay cheap as logs grow. Inline snapshots, versioned upcasters, and snapshot-verified compaction keep replay quick without rewriting history. An optional entity-heads table tracks each entity's latest sequence so you can rebuild only the entities that changed instead of reprocessing the whole log.
  • Built for AWS-native systems. First-class DynamoDB Streams projectors and SNS publishers, with idempotency and retry classification, not as an afterthought.

Install

go get github.com/photon-grove/evt

Quick Start

An aggregate handles commands, emits events, and folds those events into state. Here is the shape of one — see examples/banking for the full runnable version:

func (a *Account) Handle(_ context.Context, cmd evt.Command) (evt.CommandResult, error) {
    switch c := cmd.(type) {
    case *OpenAccount:
        if a.Opened {
            return evt.CommandResult{}, evt.NewConflictError("account already opened")
        }
        return evt.CommandResult{Events: []evt.Event{
            AccountOpened{AccountID: c.AccountID, Balance: c.InitialBalance},
        }}, nil
    // ...
    }
}

func (a *Account) Apply(event evt.Event) error {
    switch e := event.(type) {
    case AccountOpened:
        a.ID, a.Balance, a.Opened = e.AccountID, e.Balance, true
    // ...
    }
    return nil
}

Drive it with an in-memory store in tests:

store := mem.NewStore()

// Execute replays history onto the instance you pass, so hand it a fresh one per command.
acct := NewAccount("acct-1")
store.Execute(ctx, acct, "acct-1", &OpenAccount{AccountID: "acct-1", InitialBalance: 100}, evt.Metadata{})

acct = NewAccount("acct-1")
store.Execute(ctx, acct, "acct-1", &Deposit{AccountID: "acct-1", Amount: 25}, evt.Metadata{})

fmt.Println(acct.Balance) // 125

Move the same aggregate to DynamoDB for production — the contract does not change:

repo := dynamo.NewRepository(dynamoClient, "event-log")
store := snapshots.NewStore(repo, 25) // snapshot every 25 events

When aggregates carry injected dependencies, construct them with a factory:

account, err := evt.ExecuteWithFactory(ctx, store, func() evt.Entity {
    return NewAccount("acct-1", projectors...)
}, "acct-1", command, metadata)

Packages

Package Purpose
evt Core aggregate, command, event, metadata, transaction, repository, and rebuild contracts
evt/mem In-memory repository and store for unit tests
evt/dynamo DynamoDB event log, snapshots, views, stream helpers, and transaction groups
evt/postgres PostgreSQL event log and snapshots — the same evt.Repository contract on a relational store
evt/snapshots Store with snapshot loading and write thresholds
evt/stream Stream handler and publisher helpers for event fanout
evt/projectors DynamoDB Streams Lambda projector runtime with idempotency and retry classification
evt/publishers DynamoDB Streams publisher handler with ingress and retry budgets
evt/policy Backend-neutral retry helpers for commit paths
evt/policy/dynamodb DynamoDB-specific retry classification
evt/viewstore Typed JSON helper around evt.ViewRepository
evt/test Test aggregate, commands, events, and helpers for adopters and framework tests

All packages live under github.com/photon-grove/evt.

The Core Discipline

One rule holds the design together: event rows are the source of truth; view rows are derived state. Any projection or view table must be safe to wipe and rebuild from the immutable log. Human decisions, external signals, publish flags, and review verdicts have to become events before they show up in a view — never written straight to a view table with no backing fact.

Event logs are append-only and retained in full by default. Two opt-in mechanisms trim them safely:

  • Compaction (evt.Compactor.CompactBelow) deletes only events a durable snapshot already covers, then rebuilds seed from that snapshot rather than replaying from event 1. See ADR 0001.
  • Per-type retention (dynamo.Repository.WithRetention) stamps a DynamoDB ttl on terminal, short-lived, fully transient streams so the table expires them automatically. Because TTL expires rows individually — it cannot atomically drop a whole stream — each event expires at committedAt + duration, so this is safe only when a stream's entire lifetime is much shorter than the duration and it is never appended to after going terminal; otherwise an older prefix can expire while newer events survive and a load replays a partial suffix. For streams that accumulate, keep a snapshot and compact instead.

The raw dynamo.Delete is snapshot-unsafe, for local fixtures only, and excluded from production builds (-tags prod). See BEHAVIORAL_INVARIANTS.md for the exact serialization and DynamoDB schema guarantees.

Local Development

pnpm install
go test ./...
moon run evt:test

DynamoDB integration tests run against a local AWS emulator (Moto), not real AWS:

docker run -d --name evt-moto -p 4566:5000 motoserver/moto:5.1.22
terraform -chdir=infra/local init
terraform -chdir=infra/local apply -auto-approve
AWS_ENDPOINT_URL=http://localhost:4566 moon run evt:integration

PostgreSQL integration tests run against a local PostgreSQL server. Terraform provisions the database; postgres.Repository.EnsureSchema owns the tables:

docker run -d --name evt-pg -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:17
terraform -chdir=infra/local-postgres init
terraform -chdir=infra/local-postgres apply -auto-approve
moon run evt:integration-postgres

Documentation

The docs site covers architecture, the integration cookbook, and runnable examples. Static docs live in docs/; the interactive site lives in website/.

moon run website:dev
moon run website:build

License

Apache-2.0. See LICENSE.


Built with care by Photon Grove, a Colorado software studio.

Documentation

Overview

Package evt provides an immutable event-sourcing framework with pluggable backends.

Index

Constants

View Source
const DefaultViewSK = "VIEW"

DefaultViewSK is the default sort key for single-item views.

Variables

View Source
var ErrCompactionUncovered = errors.New("compaction refused: no durable snapshot covers the requested range")

ErrCompactionUncovered is returned by a Compactor when a compaction request cannot be proven safe: either the stream has no durable snapshot, or its latest snapshot does not cover the requested sequence. Compaction must never delete an event that is not already captured by a durable snapshot, so the operation is refused rather than risking data loss.

Functions

func CalculateAdditionalEvents

func CalculateAdditionalEvents(currentSequence EventSequence, numEvents int, maxSize int) int

CalculateAdditionalEvents reports how many events from a commit batch should be folded into a snapshot, given the entity's current sequence, the number of events in the batch, and the snapshot interval (maxSize).

It returns 0 when the batch does not reach the next snapshot boundary, meaning no snapshot is taken for this commit. Otherwise it returns numEvents: a snapshot is written capturing the entity state through the entire batch.

The full batch is captured (rather than only the events up to the boundary) because both the in-memory and DynamoDB repositories record the snapshot's EventSequence as the last event in the batch. Capturing a partial batch would leave the snapshot payload inconsistent with that recorded sequence, so a reload would restore stale state. A maxSize of 0 or less disables snapshots.

func DecodeJSONView

func DecodeJSONView[T any](view *SerializedView, target T) error

DecodeJSONView decodes a SerializedView payload into target.

target must be a non-nil pointer. Passing a struct value, a nil pointer, or a nil slice/map returns an error rather than silently no-op'ing into a discarded local copy. Use a pointer to preserve constructor-injected fields on domain entities and to receive the decoded JSON payload.

func GetJSONView

func GetJSONView[T any](
	ctx context.Context,
	repo ViewRepository,
	pk string,
	newValue NewJSONViewValueFunc[T],
) (T, bool, error)

GetJSONView loads a serialized view by partition key and decodes its JSON payload into T.

The boolean return value is false when no view exists for pk. Repository and JSON decode failures are returned as errors.

func IsConflictErr

func IsConflictErr(err error) bool

IsConflictErr returns true when err is a ConflictError.

func IsDuplicateCommandErr

func IsDuplicateCommandErr(err error) bool

IsDuplicateCommandErr returns true when err is a DuplicateCommandError.

func IsNoOpErr

func IsNoOpErr(err error) bool

IsNoOpErr returns true when err is a NoOpError.

func IsNotFoundErr

func IsNotFoundErr(err error) bool

IsNotFoundErr returns true when err is a NotFoundError.

func IsReplayStrictnessErr

func IsReplayStrictnessErr(err error) bool

IsReplayStrictnessErr returns true when err is a ReplayStrictnessError.

func ListJSONViewsByEntityType

func ListJSONViewsByEntityType[T any](
	ctx context.Context,
	repo ViewRepository,
	entityType EntityType,
	newValue NewJSONViewValueFunc[T],
) ([]T, error)

ListJSONViewsByEntityType lists views by entity type and strictly decodes every JSON payload into T.

Use ListValidJSONViewsByEntityType when invalid payloads should be skipped instead of failing the whole list.

func ListJSONViewsByPK

func ListJSONViewsByPK[T any](
	ctx context.Context,
	repo ViewRepository,
	pk string,
	newValue NewJSONViewValueFunc[T],
) ([]T, error)

ListJSONViewsByPK lists views by partition key and strictly decodes every JSON payload into T.

Use ListValidJSONViewsByPK when invalid payloads should be skipped instead of failing the whole list.

func ListValidJSONViewsByEntityType

func ListValidJSONViewsByEntityType[T any](
	ctx context.Context,
	repo ViewRepository,
	entityType EntityType,
	newValue NewJSONViewValueFunc[T],
	onDecodeError func(*SerializedView, error),
) ([]T, error)

ListValidJSONViewsByEntityType lists views by entity type and decodes valid JSON payloads into T, skipping invalid payloads.

When onDecodeError is non-nil, it is called for each skipped payload.

func ListValidJSONViewsByPK

func ListValidJSONViewsByPK[T any](
	ctx context.Context,
	repo ViewRepository,
	pk string,
	newValue NewJSONViewValueFunc[T],
	onDecodeError func(*SerializedView, error),
) ([]T, error)

ListValidJSONViewsByPK lists views by partition key and decodes valid JSON payloads into T, skipping invalid payloads.

When onDecodeError is non-nil, it is called for each skipped payload.

func LoadEntityWithFactory

func LoadEntityWithFactory(
	ctx context.Context,
	store Store,
	factory EntityFactory,
	entityID EntityID,
) (Entity, Context, error)

LoadEntityWithFactory creates a fresh entity instance and loads it from the store.

func PutJSONView

func PutJSONView(ctx context.Context, repo ViewRepository, view *SerializedView, value any) error

PutJSONView marshals value as JSON and writes it as the Payload of view.

The provided SerializedView is copied before mutation so callers can safely reuse metadata templates across writes.

Types

type BadCommandError

type BadCommandError struct {
	Command Command
}

BadCommandError means an invalid Command was sent to an Entity

func NewBadCommandError

func NewBadCommandError(command Command) *BadCommandError

NewBadCommandError creates a new instance of a BadCommandError

func (*BadCommandError) Error

func (e *BadCommandError) Error() string

Error provides the error message

type BadEventError

type BadEventError struct {
	Event Event
}

BadEventError is returned when an invalid Event is sent to an

func NewBadEventError

func NewBadEventError(event Event) *BadEventError

NewBadEventError creates a new instance of a BadEventError

func (*BadEventError) Error

func (e *BadEventError) Error() string

Error returns the error message string

type BaseEntity

type BaseEntity struct {
	ID        EntityID  `json:"id"`        // A unique identifier
	IsActive  bool      `json:"isActive"`  // Whether this record has been deleted
	CreatedAt time.Time `json:"createdAt"` // The date this record was originally created
	UpdatedAt time.Time `json:"updatedAt"` // The date this record was last updated
}

BaseEntity provides common fields for all entities.

func NewEntity

func NewEntity(id EntityID) BaseEntity

NewEntity creates a new BaseEntity with the given ID, setting IsActive to true and timestamps to now.

type ByCommandID

type ByCommandID []SerializedEvent

ByCommandID allows you to sort SerializedEvents by the CommandID in the Metadata, falling back to sorting by event ID if the CommandID is empty.

func (ByCommandID) Len

func (events ByCommandID) Len() int

Len returns the length of the slice.

func (ByCommandID) Less

func (events ByCommandID) Less(i, j int) bool

Less reports whether the element with index i should sort before the element with index j.

func (ByCommandID) Swap

func (events ByCommandID) Swap(i, j int)

Swap swaps the elements with indexes i and j.

type Command

type Command interface {
	Type() CommandType
	EntityType() EntityType
}

Command is implemented by all command types.

type CommandID

type CommandID string

CommandID is a unique identifier for a Command instance.

type CommandResult

type CommandResult struct {
	Events      []Event     `json:"events"`
	Transaction Transaction `json:"transaction"`
}

CommandResult is the outcome of executing a Command.

type CommandType

type CommandType string

CommandType is a unique identifier for the type of a Command.

type Compactor added in v0.2.0

type Compactor interface {
	// CompactBelow deletes events for the given entity whose sequence is in the range
	// [1, throughSequence], but only after verifying that a durable snapshot exists whose
	// recorded EventSequence is >= throughSequence (i.e. the deleted range is fully captured
	// by the snapshot). The sk=0 snapshot row is never deleted.
	//
	// It returns the number of events deleted. If no covering snapshot exists, it deletes
	// nothing and returns ErrCompactionUncovered. A throughSequence < 1 is a no-op.
	CompactBelow(ctx context.Context, entityID EntityID, throughSequence EventSequence) (int, error)
}

Compactor is an optional capability implemented by Repositories that can truncate an entity's event log below a durable snapshot. It is intentionally NOT part of the core Repository interface so that adding it remains backward compatible for the shared module: callers detect support with a type assertion (repo.(Compactor)).

Compaction forfeits the "replay from sequence 1" property for the affected stream. After a successful CompactBelow, the stream's authoritative starting point is its latest durable snapshot, not its first event. Rebuilds must therefore use a SnapshotStreamer (see RebuildConfig.SeedEntity) rather than assuming events 1..N are present.

type ConflictError

type ConflictError struct {
	// contains filtered or unexported fields
}

ConflictError indicates a domain-level conflict (e.g., duplicate resource)

func NewConflictError

func NewConflictError(msg string) *ConflictError

NewConflictError creates a new ConflictError with the given message.

func (*ConflictError) Error

func (e *ConflictError) Error() string

Error returns the error message string

type Context

type Context struct {
	Entity          Entity
	EntityID        EntityID
	CurrentSequence *EventSequence
	CurrentSnapshot *EventSequence
	SeenCommandIDs  map[CommandID]struct{} // CommandIDs observed during event replay (dedupe guard)
}

Context carries entity context for the EventStore.

func (Context) HasCommandID

func (c Context) HasCommandID(id CommandID) bool

HasCommandID reports whether the given CommandID was already seen during event replay.

func (*Context) RecordCommandID

func (c *Context) RecordCommandID(id CommandID)

RecordCommandID adds a CommandID to the seen set (initializing the map if needed).

type DuplicateCommandError

type DuplicateCommandError struct {
	CommandID CommandID
}

DuplicateCommandError indicates a command with the same CommandID has already been processed. Callers should treat this as an idempotent success rather than a failure.

func NewDuplicateCommandError

func NewDuplicateCommandError(id CommandID) *DuplicateCommandError

NewDuplicateCommandError creates a new DuplicateCommandError.

func (*DuplicateCommandError) Error

func (e *DuplicateCommandError) Error() string

Error returns the error message string.

type Endpoint

type Endpoint string

An Endpoint is the entrypoint the request came in through

type Entity

type Entity interface {
	Type() EntityType
	GetID() EntityID
	Base() BaseEntity
	Handle(context.Context, Command) (CommandResult, error)
	Apply(Event) error
	DeserializeEvent(serializedEvent SerializedEvent) (Event, error)
	EventUpcasters() []EventUpcaster
	Projectors() []EventProjector
}

Entity is an object whose state is made up of aggregated immutable change Events.

func ExecuteWithFactory

func ExecuteWithFactory(
	ctx context.Context,
	store Store,
	factory EntityFactory,
	entityID EntityID,
	command Command,
	metadata Metadata,
) (Entity, error)

ExecuteWithFactory creates a fresh entity instance, executes the command, and returns it.

type EntityFactory

type EntityFactory func() Entity

EntityFactory constructs a fresh zero-state aggregate instance for load/execute operations.

type EntityHeadStreamer added in v0.4.0

type EntityHeadStreamer interface {
	// StreamEntityHeads enumerates the distinct entities in the event log and returns each
	// entity's current head sequence. The head is the larger of (a) the highest event sort key in
	// the partition and (b) the EventSequence recorded by the entity's durable snapshot (the sk=0
	// row), so the result stays correct for streams whose early events have been compacted away
	// (where the snapshot, not event 1, is the authoritative floor). entityType, when non-empty,
	// restricts enumeration to entities of that type.
	//
	// Cost note: for scan-backed stores this is a single key-only Scan. DynamoDB charges read
	// capacity by item size, not by projected attributes, so it reads capacity comparable to a
	// full scan; the win is one pass yielding one bounded entry per entity (the changed set), not
	// lower read cost. It replaces re-reading and re-projecting every stream on every rebuild.
	StreamEntityHeads(ctx context.Context, entityType EntityType) (map[EntityID]EventSequence, error)
}

EntityHeadStreamer is an optional capability implemented by Repositories that can report every entity's current head sequence in a single enumeration pass. It is the change-detection primitive for incremental projection rebuilds: a caller compares each head against a per-entity projection checkpoint (the sequence a view was last built from) to find the entities whose projections are stale, and reprojects only those.

It is designed to preserve the event log's write scaling: detection reads from the existing partition key and sort key, so it needs no secondary index and no global sequence counter — the two structures that would reintroduce a hot write-path partition. Each append still touches only its own entity partition.

Like Compactor and SnapshotStreamer, it is kept off the core Repository interface for backward compatibility and detected via a type assertion (repo.(EntityHeadStreamer)).

type EntityHeadVisitor added in v0.5.0

type EntityHeadVisitor interface {
	// StreamEntityHeadsFunc enumerates entity heads and invokes visit once per entity, in whatever
	// order the store pages them, without holding the full set in memory. The head sequence has the
	// same meaning as StreamEntityHeads. entityType, when non-empty, restricts enumeration to that
	// type. If visit returns an error, enumeration stops and that error is returned; a paging error
	// is returned as-is. A nil error means every entity was visited exactly once.
	StreamEntityHeadsFunc(
		ctx context.Context,
		entityType EntityType,
		visit func(EntityID, EventSequence) error,
	) error
}

EntityHeadVisitor is an optional streaming variant of EntityHeadStreamer. Where StreamEntityHeads materializes every head into a map (O(entities) memory), StreamEntityHeadsFunc delivers heads one at a time to a visitor callback and never accumulates them — so a rebuild's enumeration memory ceiling stays constant no matter how many entities exist.

This is only constant-memory when the underlying store holds one row per entity (a registry such as a heads table): such a source is naturally unique, needs no dedup set, and is resumable from the last key it paged. It is deliberately backend-neutral — no DynamoDB types appear here — so a future SQL backend can satisfy it with a streamed cursor over SELECT entity_id, MAX(sequence) …. Like EntityHeadStreamer, it is kept off the core Repository interface and detected via a type assertion (source.(EntityHeadVisitor)).

type EntityID

type EntityID string

EntityID is a unique identifier for an aggregated Entity instance.

type EntityType

type EntityType string

EntityType is the type of an aggregated Entity.

type Event

type Event interface {
	Type() EventType
	Version() EventVersion
	EntityType() EntityType
	EntityID() EntityID
}

Event is implemented by all event types.

func DeserializeEvent

func DeserializeEvent(serializedEvent SerializedEvent, entity Entity) (Event, error)

DeserializeEvent decodes Events with the Entity within the given Context, applying upcasters.

type EventID

type EventID string

EventID combines the entity ID and event sequence.

func GetEventID

func GetEventID(entityID EntityID, sequence EventSequence) EventID

GetEventID builds a stable event ID from the entity ID and sequence.

type EventProjector

type EventProjector interface {
	// Project produces a transaction group for the given entity state. Returning nil
	// indicates no operations are required.
	Project(context.Context, Entity, []Event) (TransactionGroup, error)
}

EventProjector generates transactional view operations that should be committed alongside events.

type EventSequence

type EventSequence int

EventSequence is a unique sequence number within an entity instance.

type EventType

type EventType string

EventType identifies an event kind.

type EventUpcaster

type EventUpcaster interface {
	CanUpcast(EventType, EventVersion) bool
	Upcast(SerializedEvent) (SerializedEvent, error)
}

EventUpcaster migrates a serialized event from an older version to a newer version before aggregate replay.

type EventVersion

type EventVersion int

EventVersion is the schema version of an event.

type JSONViewDecodeError

type JSONViewDecodeError struct {
	PK  string
	SK  string
	Err error
}

JSONViewDecodeError describes a payload decode failure for a serialized view.

func (*JSONViewDecodeError) Error

func (e *JSONViewDecodeError) Error() string

Error returns a human-readable decode error message.

func (*JSONViewDecodeError) Unwrap

func (e *JSONViewDecodeError) Unwrap() error

Unwrap returns the underlying JSON decode error.

type Metadata

type Metadata struct {
	CommandID *CommandID              `json:"commandId"`
	Trace     *propagation.MapCarrier `json:"trace"`
	Origin    *Origin                 `json:"origin"`
	Address   *string                 `json:"address"`
	Region    string                  `json:"region"`
	Timestamp string                  `json:"timestamp"`
}

Metadata is a set of additional data carried along with an Event. Any changes should be backwards compatible, and bad data should always be tested.

func NewMetadata

func NewMetadata(_ context.Context, region *string, opts ...MetadataOption) Metadata

NewMetadata initializes a Metadata k-v store

type MetadataOption

type MetadataOption func(Metadata) Metadata

MetadataOption is used to add properties to metadata

func WithAddress

func WithAddress(address string) MetadataOption

WithAddress adds the caller-provided address to metadata.

func WithCommandID

func WithCommandID(cmdid CommandID) MetadataOption

WithCommandID adds the provided commandId to metadata

func WithOrigin

func WithOrigin(origin Origin) MetadataOption

WithOrigin adds the Origin to the request Metadata

func WithTrace

func WithTrace(ctx context.Context) MetadataOption

WithTrace adds trace context to metadata

type NewJSONViewValueFunc

type NewJSONViewValueFunc[T any] func() T

NewJSONViewValueFunc creates a destination value for JSON view decoding.

For pointer-backed domain entities that need constructor-injected dependencies, pass the app-specific constructor:

entity, found, err := evt.GetJSONView(ctx, repo, pk, func() *Entity {
	return NewEntity(projectors...)
})

For plain value types, callers may pass nil and the zero value of T will be used as the decode destination.

type NoOpError

type NoOpError struct {
	// contains filtered or unexported fields
}

NoOpError indicates a command would produce no changes.

func NewNoOpError

func NewNoOpError(msg string) *NoOpError

NewNoOpError creates a new NoOpError with the given message.

func (*NoOpError) Error

func (e *NoOpError) Error() string

Error returns the error message string.

type NotFoundError

type NotFoundError struct {
	// contains filtered or unexported fields
}

A NotFoundError indicates a required record was not found

func NewNotFoundError

func NewNotFoundError(msg string) *NotFoundError

NewNotFoundError creates a new instance of a NotFoundError from a message

func (*NotFoundError) Error

func (e *NotFoundError) Error() string

Error returns the error message string

type Origin

type Origin struct {
	Source   Source   `json:"source"`
	Endpoint Endpoint `json:"endpoint"`
}

An Origin is the combination of the source and the endpoint

type PagedResult

type PagedResult struct {
	Views      []*SerializedView
	NextCursor string
}

PagedResult wraps a page of views with an optional cursor for the next page.

type RebuildConfig

type RebuildConfig struct {
	// EntityType filters entities to rebuild. Only entities matching this type are processed.
	EntityType EntityType

	// Projectors are the projection functions to run against each entity.
	Projectors []EventProjector

	// CommitGroup writes a TransactionGroup produced by a projector. Required when DryRun is false.
	// For DynamoDB-backed views this typically executes the ViewPutGroup's write items via
	// BatchWriteItem or TransactWriteItems.
	CommitGroup func(ctx context.Context, group TransactionGroup) error

	// DryRun when true skips writing views and only reports what would change.
	DryRun bool

	// MaxErrors caps the number of per-entity errors collected before the rebuild stops early.
	// Zero means no limit.
	MaxErrors int

	// SeedEntity, when non-nil, switches the rebuild onto the snapshot-aware replay path: each
	// entity is reconstructed from its latest durable snapshot before its post-snapshot events
	// are applied. This is REQUIRED to rebuild correctly after CompactBelow has truncated events
	// below a snapshot, because events 1..N are no longer guaranteed to exist. When set, the
	// repository must implement SnapshotStreamer or RebuildProjections returns an error. When nil,
	// the legacy full-replay path (StreamEntities, sequence 1..N) is used.
	SeedEntity SnapshotSeeder

	// OnProgress, if non-nil, is called after each entity is successfully processed
	// by the projectors or when an error is encountered (stream, projector, or commit).
	// Skipped entities (nil or wrong type) do not trigger this callback.
	// The processed argument is the count of successfully processed entities so far,
	// and errors is the cumulative number of errors encountered.
	OnProgress func(processed, errors int)

	// Logger receives rebuild telemetry. If nil, slog.Default() is used.
	Logger *slog.Logger
}

RebuildConfig configures a projection rebuild run.

type RebuildResult

type RebuildResult struct {
	// Processed is the number of entities successfully replayed through all projectors.
	Processed int

	// Skipped is the number of entities skipped (wrong type or nil).
	Skipped int

	// Errors collects per-entity errors that did not halt the run.
	// Capped by MaxErrors if configured.
	Errors []error
}

RebuildResult summarizes a projection rebuild run.

Note: Processed counts only entities where all projectors succeeded. Entities that produced errors are counted in Errors, not Processed or Skipped.

func RebuildProjections

func RebuildProjections(
	ctx context.Context,
	repo Repository,
	applyEvent func(context.Context, SerializedEvent, Entity) (Entity, error),
	cfg RebuildConfig,
) (*RebuildResult, error)

RebuildProjections replays all entities from the repository through the configured projectors, regenerating view data. Entities are streamed via StreamEntities, so this works with any Repository implementation.

The applyEvent callback is the same function used by StreamEntities to reconstitute entities from serialized events. Callers typically pass the same function they use for normal event replay (deserialize + apply).

StreamEntities buffers the matched events in memory for the duration of a table scan. For large event logs, build the entity stream with a bounded-memory source (e.g. a DynamoDB repository's StreamEntitiesByQuery) and pass it to RebuildProjectionsFromStream instead.

func RebuildProjectionsFromStream added in v0.2.0

func RebuildProjectionsFromStream(
	ctx context.Context,
	stream <-chan result.Result[Entity],
	cfg RebuildConfig,
) (*RebuildResult, error)

RebuildProjectionsFromStream runs the configured projectors against a caller-supplied stream of reconstituted entities. RebuildProjections is the common case (it builds the stream from a Repository); use this directly when you need a different streaming strategy, such as a bounded-memory, parallel enumerate-then-query source for very large tables.

The caller owns the stream's reconstitution semantics (entity-type filtering, ordering); the per-entity type check below is a defensive backstop, not the primary filter.

type ReplayStrictnessError

type ReplayStrictnessError struct {
	EventType EventType
	Version   EventVersion
	Phase     string
	Err       error
}

ReplayStrictnessError indicates replay failed closed while upcasting or deserializing an event.

func (*ReplayStrictnessError) Error

func (e *ReplayStrictnessError) Error() string

Error returns the underlying error message.

func (*ReplayStrictnessError) Unwrap

func (e *ReplayStrictnessError) Unwrap() error

Unwrap returns the underlying error.

type Repository

type Repository interface {
	// Commit new Events for an Entity instance.
	Commit(ctx context.Context, result SerializedResult) error

	// Commit a stream of Events using whatever batch size works best for the Repository.
	CommitStream(ctx context.Context, channel <-chan result.Result[SerializedResult]) []error

	// Commit new Events for an Entity instance while also saving the given Snapshot.
	CommitWithSnapshot(
		ctx context.Context,
		result SerializedResult,
		entityType EntityType,
		entityID EntityID,
		payload []byte,
		currentSnapshot EventSequence,
	) error

	// Get all Events for an Entity instance.
	GetEvents(
		ctx context.Context,
		entityID EntityID,
	) ([]SerializedEvent, error)

	// Get the latest Events beyond the given sequence number for an Entity, used for
	// finding Events after a particular Snapshot.
	GetLatestEvents(
		ctx context.Context,
		entityID EntityID,
		lastSequence EventSequence,
	) ([]SerializedEvent, error)

	// Get the latest Snapshot for an Entity instance.
	GetSnapshot(
		ctx context.Context,
		entityID EntityID,
	) (*SerializedSnapshot, error)

	// StreamAllEvents returns a channel of all Events in the Repository, optionally narrowed by the
	// backend-neutral StreamFilter. A zero filter streams every event.
	StreamAllEvents(
		ctx context.Context,
		filter StreamFilter,
	) <-chan result.Result[[]SerializedEvent]

	// StreamEntities returns a channel of all Entities in the Repository, optionally narrowed by the
	// backend-neutral StreamFilter, with the given function applied to each Entity to fold the
	// application-specific Events into each instance. A zero filter streams every entity.
	StreamEntities(
		ctx context.Context,
		filter StreamFilter,
		applyEvent func(context.Context, SerializedEvent, Entity) (Entity, error),
	) <-chan result.Result[Entity]
}

Repository manages events for entities consistently and persists snapshots to improve load time.

type SerializedEvent

type SerializedEvent struct {
	ID         EventID       `json:"id"` // Composed of the EntityID and EventSequence
	Sequence   EventSequence `json:"sequence"`
	Type       EventType     `json:"type"`
	Version    EventVersion  `json:"version"`
	EntityID   EntityID      `json:"entityId"`
	EntityType EntityType    `json:"entityType"`
	Payload    []byte        `json:"payload"`
	Metadata   Metadata      `json:"metadata"`
}

SerializedEvent is a common event format that is ready to be committed to an Event Store and streamed to downstream event listeners.

func SerializeEvents

func SerializeEvents(
	events []Event,
	sequence EventSequence,
	entityID EntityID,
	metadata Metadata,
) ([]SerializedEvent, error)

SerializeEvents prepares an Event for serialization and storage.

func SerializeEventsWithContext

func SerializeEventsWithContext(events []Event, eventContext *Context, metadata Metadata) ([]SerializedEvent, error)

SerializeEventsWithContext serializes Events with the Entity within the given Context.

type SerializedResult

type SerializedResult struct {
	Events      []SerializedEvent `json:"events"`
	Transaction Transaction       `json:"transactions"`
}

SerializedResult is similar to CommandResult but uses SerializedEvents instead.

func SerializeResult

func SerializeResult(
	eventContext Context,
	entity Entity,
	events []Event,
	transaction Transaction,
) (SerializedResult, error)

SerializeResult creates a SerializedResult from events using the given context and entity. This is a convenience function for testing and integration scenarios.

type SerializedSnapshot

type SerializedSnapshot struct {
	EntityType    EntityType    `json:"entityType"`
	EntityID      EntityID      `json:"entityID"`
	Sequence      EventSequence `json:"sequence"`
	EventSequence EventSequence `json:"eventSequence"`
	Payload       []byte        `json:"payload"`
}

SerializedSnapshot represents a snapshot of current Entity state that is ready to be committed to a Repository.

type SerializedView

type SerializedView struct {
	PK         string     `json:"pk"`
	SK         string     `json:"sk"`
	EntityID   EntityID   `json:"entityID"`
	EntityType EntityType `json:"entityType"`
	Payload    []byte     `json:"payload"`
	TTL        int64      `json:"ttl,omitempty"`
}

SerializedView is a simple representation of a materialized entity view stored in the backing store.

type SnapshotSeeder added in v0.2.0

type SnapshotSeeder func(ctx context.Context, snapshot SerializedSnapshot) (Entity, error)

SnapshotSeeder reconstructs a fresh entity instance from a durable snapshot payload. It is the snapshot-aware counterpart to the applyEvent callback used during replay: applyEvent builds an entity from its events, while a SnapshotSeeder builds one from the JSON state a snapshot captured. Callers typically look up the concrete type by snapshot.EntityType and json.Unmarshal(snapshot.Payload, entity).

type SnapshotStreamer added in v0.2.0

type SnapshotStreamer interface {
	// StreamEntitiesFromSnapshots streams completed entities, seeding each from its latest durable
	// snapshot (the sk=0 row) via seedEntity before applying only the events recorded after that
	// snapshot. Events at or below the snapshot's EventSequence are skipped because the snapshot
	// already captures them. Streams with no snapshot fall back to full replay from sequence 1 via
	// applyEvent. entityType, when non-empty, restricts the rebuild to entities of that type.
	//
	// Implementations should reconstruct each entity from its own partition (a per-entity query),
	// not from scan order, so the result is correct regardless of how the underlying store returns
	// rows.
	StreamEntitiesFromSnapshots(
		ctx context.Context,
		entityType EntityType,
		seedEntity SnapshotSeeder,
		applyEvent func(context.Context, SerializedEvent, Entity) (Entity, error),
	) <-chan result.Result[Entity]
}

SnapshotStreamer is an optional capability implemented by Repositories that can rebuild entities starting from each stream's durable snapshot instead of from sequence 1. It is the replay path that stays correct after CompactBelow has removed events below a snapshot.

Like Compactor, it is kept off the core Repository interface for backward compatibility and detected via a type assertion (repo.(SnapshotStreamer)).

type Source

type Source string

A Source is the system that initiated the request

type StorageType

type StorageType string

StorageType identifies the backend storage system a Repository uses.

type Store

type Store interface {
	// LoadEntity loads the given Entity instance with the given id, retrieving Snapshots
	// and Events from the Repository.
	LoadEntity(
		ctx context.Context,
		entity Entity,
		entityID EntityID,
	) (Context, error)

	// Commit new Events to the Entity within the given context, with optional Metadata.
	Commit(
		ctx context.Context,
		result CommandResult,
		context Context,
		metadata Metadata,
	) ([]SerializedEvent, error)

	// Execute takes an empty Entity instance, loads it from the Repository, handles the given
	// Command, commits the resulting Events using the given Metadata, and applies those Events to the
	// current Entity instance.
	Execute(
		ctx context.Context,
		entity Entity,
		entityID EntityID,
		command Command,
		metadata Metadata,
	) error
}

Store handles executing Commands, committing Events yielded by those Commands, and loading Entity instances from a Repository.

type StreamFilter added in v0.6.0

type StreamFilter struct {
	// EntityType, when non-empty, restricts the stream to entities of that type.
	EntityType EntityType
}

StreamFilter narrows which entities a table-wide stream or projection rebuild visits. It is a backend-neutral description of the filter: each Repository translates it into its own query mechanism (the DynamoDB backend, for example, compiles it into a Scan FilterExpression; a future SQL backend would translate it into a WHERE clause). The zero value imposes no constraint and matches every entity.

It deliberately exposes only entity-type filtering — the single predicate the framework's rebuild paths require — so the core Repository contract carries no backend-specific query types. Backends that support richer server-side filtering may offer it through their own extension interface (for example the dynamo package's ExpressionStreamer) without widening this type.

func (StreamFilter) Matches added in v0.6.0

func (f StreamFilter) Matches(entityType EntityType) bool

Matches reports whether an entity of the given type passes the filter. A zero filter (empty EntityType) matches every entity. Backends without server-side filtering can use this to filter client-side and still honor the contract.

type Transaction

type Transaction = []TransactionGroup

A Transaction is a collection of grouped operations that should be rolled back together in the event of failure.

func MergeTransactions

func MergeTransactions(transaction, with Transaction) Transaction

MergeTransactions takes two Transactions to merge together, and returns a new Transaction that contains the groups from both Transactions (or either, if one is nil)

type TransactionGroup

type TransactionGroup interface {
	Len() int
	Merge(with TransactionGroup) (TransactionGroup, error)
	StorageType() StorageType
	TransactionType() TransactionType
	HandleError(error, int) error
}

A TransactionGroup is a group of operations included in a Transaction

func Merge

func Merge(group, with TransactionGroup) (TransactionGroup, error)

Merge combines two TransactionGroups into one, gracefully handling nil values.

func MergeTransactionGroups

func MergeTransactionGroups(group, with TransactionGroup) (TransactionGroup, error)

MergeTransactionGroups takes two TransactionGroups to merge together, and returns a new TransactionGroup that contains the operations from both TransactionGroups (or either, if one is nil)

type TransactionType

type TransactionType string

A TransactionType is a type of operation included in the Transaction.

type UpcastError

type UpcastError struct {
	Err             error           `json:"err"`
	SerializedEvent SerializedEvent `json:"serializedEvent"`
}

UpcastError wraps a failure that occurred while upcasting a serialized event.

func NewUpcastError

func NewUpcastError(err error, serializedEvent SerializedEvent) *UpcastError

NewUpcastError creates a new UpcastError instance.

func (*UpcastError) Error

func (e *UpcastError) Error() string

Error returns the wrapped error message.

type ViewRepository

type ViewRepository interface {
	// GetView retrieves a view by partition key (pk). Returns nil if not found.
	GetView(ctx context.Context, pk string) (*SerializedView, error)

	// PutView writes or replaces the given view.
	PutView(ctx context.Context, view *SerializedView) error

	// ListViewsByEntityType returns serialized views for the provided entity type.
	//
	// This buffers the entire result set in memory. For large entity types prefer
	// ListViewsByEntityTypePaged (caller-driven paging), or the bounded-memory streaming iterators
	// on the optional ViewStreamer interface.
	ListViewsByEntityType(ctx context.Context, entityType EntityType) ([]*SerializedView, error)

	// ListViewsByEntityTypePaged returns a page of views for the provided entity type.
	// limit controls page size; cursor is an opaque pagination token (empty for the first page).
	ListViewsByEntityTypePaged(ctx context.Context, entityType EntityType, limit int, cursor string) (*PagedResult, error)

	// ListViewsByPK returns all serialized views for the given partition key.
	// Used for composite keys where pk identifies a collection (e.g., USER#<id>#teams).
	//
	// This buffers the entire result set in memory. For partition keys that can hold many rows
	// prefer ListViewsByPKPaged (caller-driven paging), or the bounded-memory streaming iterators
	// on the optional ViewStreamer interface.
	ListViewsByPK(ctx context.Context, pk string) ([]*SerializedView, error)

	// ListViewsByPKPaged returns a page of views for the given partition key.
	// limit controls page size; cursor is an opaque pagination token (empty for the first page).
	ListViewsByPKPaged(ctx context.Context, pk string, limit int, cursor string) (*PagedResult, error)
}

ViewRepository is a lightweight repository interface for reading/writing entity views.

type ViewStreamer added in v0.2.0

type ViewStreamer interface {
	// ListViewsByEntityTypeEach streams views for the provided entity type, invoking fn for each
	// view without buffering the whole result set. Iteration stops and the error is returned if fn
	// returns an error or the context is cancelled.
	ListViewsByEntityTypeEach(ctx context.Context, entityType EntityType, fn func(*SerializedView) error) error

	// ListViewsByPKEach streams views for the given partition key, invoking fn for each view
	// without buffering the whole result set. Iteration stops and the error is returned if fn
	// returns an error or the context is cancelled.
	ListViewsByPKEach(ctx context.Context, pk string, fn func(*SerializedView) error) error
}

ViewStreamer is an optional interface a ViewRepository may also implement to read views with bounded memory. It is kept separate from ViewRepository so that adding streaming iterators does not break existing ViewRepository implementations. Type-assert a ViewRepository to ViewStreamer when you need it (the DynamoDB-backed repository satisfies it).

Directories

Path Synopsis
cmd
evt-doctor command
Command evt-doctor is a CLI for inspecting and diagnosing an evt DynamoDB event log.
Command evt-doctor is a CLI for inspecting and diagnosing an evt DynamoDB event log.
Package conformance provides a backend-neutral contract test suite for evt.Repository implementations.
Package conformance provides a backend-neutral contract test suite for evt.Repository implementations.
Package dynamo implements an EventStore backed by DynamoDB and related utilities.
Package dynamo implements an EventStore backed by DynamoDB and related utilities.
mock
Package mock provides DynamoDB client mocks nolint:errcheck
Package mock provides DynamoDB client mocks nolint:errcheck
examples
banking command
Command banking is a runnable example of an evt-based event-sourced aggregate.
Command banking is a runnable example of an evt-based event-sourced aggregate.
go module
Package mem provides an in-memory EventStore implementation for tests and local use.
Package mem provides an in-memory EventStore implementation for tests and local use.
Package policy provides shared retry and failure-classification primitives for evt-backed handlers.
Package policy provides shared retry and failure-classification primitives for evt-backed handlers.
dynamodb
Package dynamodb provides DynamoDB-specific retry classification for evt policy.
Package dynamodb provides DynamoDB-specific retry classification for evt policy.
Package postgres implements the evt.Repository and evt.Store contracts on top of PostgreSQL.
Package postgres implements the evt.Repository and evt.Store contracts on top of PostgreSQL.
Package projectors provides an async projector runtime for processing DynamoDB Streams events.
Package projectors provides an async projector runtime for processing DynamoDB Streams events.
Package publishers provides reusable Lambda helpers for publishing event-log records from DynamoDB Streams to downstream event transports.
Package publishers provides reusable Lambda helpers for publishing event-log records from DynamoDB Streams to downstream event transports.
Package result provides a Rust-style Result type for propagating success or error values.
Package result provides a Rust-style Result type for propagating success or error values.
Package snapshots defines a snapshot event store
Package snapshots defines a snapshot event store
Package stream provides interfaces and implementations for publishing events to streaming services.
Package stream provides interfaces and implementations for publishing events to streaming services.
Package test contains fixtures and helpers for testing event-sourced entities.
Package test contains fixtures and helpers for testing event-sourced entities.
Package viewstore provides a typed wrapper around evt.ViewRepository that binds a JSON codec, an evt.EntityType, and an optional value factory so individual stores only need to express their domain key conventions (pk/sk builders) rather than re-implementing JSON encode/decode plumbing.
Package viewstore provides a typed wrapper around evt.ViewRepository that binds a JSON codec, an evt.EntityType, and an optional value factory so individual stores only need to express their domain key conventions (pk/sk builders) rather than re-implementing JSON encode/decode plumbing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL