examples/

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/getpup/pupsourcing

Links

Open Source Insights

README ¶

pupsourcing Examples

Comprehensive, runnable examples demonstrating pupsourcing patterns and use cases.

Overview

Each example is self-contained and demonstrates specific patterns. Start with database-specific examples for fundamentals, then explore projection and scaling patterns.

Database Examples

Basic

Database: PostgreSQL
Difficulty: Beginner
Best for: Understanding the basics with PostgreSQL

Demonstrates basic event appending and reading with PostgreSQL.

Prerequisites:

docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=pupsourcing_example postgres:16

Run it:

cd basic
go run main.go

SQLite Basic

Database: SQLite
Difficulty: Beginner
Best for: Testing, embedded applications, no server required

Demonstrates event sourcing with SQLite (embedded database).

What you'll learn:

Using SQLite adapter (no server needed)
Perfect for testing and CI/CD
Local-first applications
Migration generation for SQLite

Run it:

cd sqlite-basic
go run main.go

MySQL Basic

Database: MySQL/MariaDB
Difficulty: Beginner
Best for: MySQL infrastructure

Demonstrates event sourcing with MySQL/MariaDB.

Prerequisites:

docker run -d -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=pupsourcing_example mysql:8

Run it:

cd mysql-basic
go run main.go

Projection Examples

1. Single Worker

Difficulty: Beginner
Best for: Getting started, development, low-volume production

The simplest projection pattern - one projection, one worker, no partitioning.

What you'll learn:

Basic projection setup
Checkpoint tracking
Graceful shutdown
Event processing

Run it:

cd single-worker
go run main.go

2. Partitioned

Difficulty: Intermediate
Best for: High-volume production, horizontal scaling

Run the same projection across multiple processes with partitioning.

What you'll learn:

Horizontal scaling across processes
Hash-based partitioning
Independent worker operation
CLI-friendly configuration

Run it:

# Terminal 1
cd partitioned
PARTITION_KEY=0 go run main.go

# Terminal 2
PARTITION_KEY=1 go run main.go

# Terminal 3-4: same with PARTITION_KEY=2 and 3

3. Worker Pool

Difficulty: Intermediate
Best for: Medium-scale production, single machine

Run multiple partitions of a projection in the same process using goroutines.

What you'll learn:

In-process parallelism
Thread-safe projections
Runner package usage
Resource sharing

Run it:

cd worker-pool
go run main.go --workers=4

4. Multiple Projections

Difficulty: Intermediate
Best for: CQRS applications, multiple read models

Run different projections concurrently in the same process.

What you'll learn:

Running multiple projections
Independent checkpoints
Different batch sizes per projection
Runner configuration

Run it:

cd multiple-projections
go run main.go

5. Scaling

Difficulty: Advanced
Best for: Understanding scaling mechanics

Demonstrates how to safely scale from 1 worker to N workers dynamically.

What you'll learn:

Adding workers incrementally
Independent catch-up
Load distribution
Production scaling patterns

Run it:

cd scaling

# Append events once
go run main.go --worker-id=0 --append

# Start workers incrementally
WORKER_ID=0 go run main.go  # Terminal 1
WORKER_ID=1 go run main.go  # Terminal 2
WORKER_ID=2 go run main.go  # Terminal 3
WORKER_ID=3 go run main.go  # Terminal 4

6. Stop and Resume

Difficulty: Beginner
Best for: Understanding checkpoint reliability

Shows that projections can be stopped and resumed without data loss.

What you'll learn:

Checkpoint persistence
Graceful shutdown
Resumption behavior
Status checking

Run it:

cd stop-resume

# Append events
go run main.go --mode=append --events=20

# Process some events, then Ctrl+C
go run main.go --mode=process

# Check status
go run main.go --mode=status

# Resume processing
go run main.go --mode=process

7. Scoped Projections

Difficulty: Intermediate
Best for: Understanding scoped vs global projections

Demonstrates the difference between scoped projections (read models filtering by aggregate type) and global projections (integration/outbox publishers receiving all events).

What you'll learn:

Scoped projections that filter by aggregate type
Global projections that receive all events
Use cases for each pattern
Running both types concurrently

Run it:

cd scoped-projections
go run main.go

8. Custom Logging

Difficulty: Beginner
Best for: Production observability integration

Shows how to integrate custom logging with pupsourcing.

What you'll learn:

Implementing the es.Logger interface
Integrating with structured logging libraries
Observability hooks for debugging
Production logging patterns

Run it:

cd with-logging
go run main.go

Advanced Examples

Event Mapping Code Generation

Difficulty: Advanced
Best for: Clean architecture, domain-driven design

Demonstrates type-safe mapping between domain events and event sourcing types using code generation.

What you'll learn:

Pure domain events (no infrastructure dependencies)
Versioned events (v1, v2, etc.)
Code generation for type-safe mappings
Schema evolution patterns

Run it:

cd eventmap-codegen
go generate
go run main.go

Quick Start

Prerequisites

Most examples require:

Go 1.23 or later
PostgreSQL 12+ running locally (except SQLite examples which need no database server)

Start PostgreSQL:

docker run -d -p 5432:5432 \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_DB=pupsourcing_example \
  postgres:16

Running Examples

Navigate to an example directory

Generate and apply migrations (first time only):

cd basic
go generate
psql -h localhost -U postgres -d pupsourcing_example -f ../../migrations/init.sql

Run the example:
```
go run main.go
```

Learning Path

New to event sourcing?

Need to scale projections?

Review Worker Pool for single-machine scaling
Study Partitioned for multi-machine scaling
Understand Scaling for dynamic scaling

Building a CQRS application?

Explore Multiple Projections
Read Scaling Guide

Key Concepts

Events are Immutable

Events are value objects that become immutable once persisted. They don't have identity until the store assigns a global_position.

Transaction Control

You control transaction boundaries. This allows you to combine event appending with other database operations atomically.

Optimistic Concurrency

Version conflicts are detected automatically via database constraints. If concurrent modifications occur, one will fail with ErrOptimisticConcurrency.

Projections

Projections read events sequentially and maintain progress via checkpoints. They can be stopped and resumed without losing position.

Horizontal Scaling

Multiple projection processors can run in parallel using hash-based partitioning. Events for the same aggregate always go to the same partition, maintaining ordering.

Checkpoints

Each projection maintains its own checkpoint in the database. Checkpoints are updated atomically with event processing, ensuring exactly-once semantics.

Common Patterns

Pattern: Graceful Shutdown

All examples demonstrate graceful shutdown:

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)

go func() {
    <-sigChan
    cancel()
}()

err := processor.Run(ctx, projection)

Pattern: CLI Configuration

Examples show CLI-friendly configuration:

partitionKey := flag.Int("partition-key", -1, "Partition key")
totalPartitions := flag.Int("total-partitions", 4, "Total partitions")
flag.Parse()

// Also support env vars
if *partitionKey == -1 {
    if envKey := os.Getenv("PARTITION_KEY"); envKey != "" {
        *partitionKey, _ = strconv.Atoi(envKey)
    }
}

Pattern: Idempotent Projections

Make projections safe for reprocessing:

func (p *Projection) Handle(ctx context.Context, event es.PersistedEvent) error {
    // Get database connection from your application context or use a field
    // Most projections will update a separate read model database
    _, err := db.ExecContext(ctx,
        "INSERT INTO read_model (id, data) VALUES ($1, $2)"+
        "ON CONFLICT (id) DO UPDATE SET data = EXCLUDED.data",
        id, data)
    return err
}

Troubleshooting

PostgreSQL Connection Errors

Ensure PostgreSQL is running:

docker ps  # Check if container is running
psql -h localhost -U postgres -d pupsourcing_example -c "SELECT 1"

Migrations Not Applied

Generate and apply migrations:

cd basic && go generate
psql -h localhost -U postgres -d pupsourcing_example -f ../../migrations/init.sql

Projection Not Processing

Check events exist:

SELECT COUNT(*) FROM events;

Check projection checkpoint:

SELECT * FROM projection_checkpoints;

Next Steps

Contributing

Found a bug or have an example idea? Please open an issue or submit a PR!

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
basic Package main demonstrates basic usage of the pupsourcing library.	Package main demonstrates basic usage of the pupsourcing library.
eventmap-codegen
domain/user/events/v1
domain/user/events/v2
infrastructure/persistence
multiple-projections Package main demonstrates running multiple projections with the runner package.	Package main demonstrates running multiple projections with the runner package.
mysql-basic Package main demonstrates basic event sourcing with MySQL/MariaDB.	Package main demonstrates basic event sourcing with MySQL/MariaDB.
partitioned Package main demonstrates horizontal scaling with partitioning across multiple processes.	Package main demonstrates horizontal scaling with partitioning across multiple processes.
scaling Package main demonstrates how to safely scale projections from 1 → N workers.	Package main demonstrates how to safely scale projections from 1 → N workers.
scoped-projections Package main demonstrates the use of scoped projections vs global projections.	Package main demonstrates the use of scoped projections vs global projections.
single-worker Package main demonstrates a single projection running on a single worker.	Package main demonstrates a single projection running on a single worker.
sqlite-basic Package main demonstrates basic event sourcing with SQLite.	Package main demonstrates basic event sourcing with SQLite.
stop-resume Package main demonstrates that projections can be stopped and resumed without data loss.	Package main demonstrates that projections can be stopped and resumed without data loss.
with-logging Package main demonstrates using a custom logger with pupsourcing.	Package main demonstrates using a custom logger with pupsourcing.
worker-pool Package main demonstrates running a single projection with N partitions in the same process.	Package main demonstrates running a single projection with N partitions in the same process.