README
¶
journal
A thin Go library for context-based event journaling.
Install
go get github.com/go-devkit/journal
Purpose
This is not a big library. The actual logic is minimal: attach a persistence strategy to a context, and chain events through it by threading a lastID from one entry to the next. Any project could implement this in ~40 lines.
The value is in the convention, not the code. If you have multiple services that all need ordered, append-only event trails, extracting the pattern into a shared library avoids subtle divergence in how chaining works. It defines a contract — the Handler interface — and provides consistent plumbing so every service journals events the same way.
Persistence and retrieval are entirely the consumer's responsibility. This library only orchestrates the write path.
Error handling
Journaling is an optional aspect of the application flow, never a mandatory step — it should not be able to break your business logic or affect its performance. For this reason, Add is intentionally silent and returns the context unchanged when:
Setupwas never called (the context is missing a handler),- the context is already cancelled (
ctx.Err() != nil), or - the handler reports failure by returning an empty ID — the existing chain is preserved rather than broken by an empty link.
In the same spirit, WithLastID ignores an empty ID and returns the context unchanged — seeding one would masquerade as a root event and silently break the chain.
There is nothing to check, nothing to handle. The handler owns its own error handling: it must never surface a persistence failure to the caller.
Setup panics if a nil handler is passed. This is a programming error, not a runtime condition — similar to how the standard library panics on misuse of sync.Mutex. Validate your handler (e.g. connection checks) before passing it to Setup.
ctx = journal.Setup(ctx, handler)
// Add is always safe to call — silent no-op if Setup was not called.
ctx = journal.Add(ctx, "order.created", nil)
Usage
Set up a handler and start journaling events:
ctx = journal.Setup(ctx, myHandler)
ctx = journal.Add(ctx, "order.created", map[string]any{
"order_id": "ord_abc123",
"total": 99.95,
})
ctx = journal.Add(ctx, "order.payment.charged", map[string]any{
"order_id": "ord_abc123",
"payment_id": "pay_xyz789",
})
Each call to Add returns a new context carrying the last event's ID. The next Add picks it up as lastID, forming a chain:
order.created (last: "")
└─ order.payment.charged (last: "evt_001")
Best Practices
Event keys
Use dot-separated hierarchical keys. This makes it easy to query, filter, and apply policies to entire categories of events at the storage level.
// Top-level category
ctx = journal.Add(ctx, "article.sync", nil)
// Specific entity within that category
ctx = journal.Add(ctx, "article.sync.12345", map[string]any{
"source": "cms",
"status": "updated",
})
At the storage level, this lets you query broadly (WHERE key LIKE 'article.sync%') or target specific entries.
Data retention
The consumer's Handler owns storage, so retention policies live there too. Consider deleting events by key prefix after a certain age. Hierarchical keys make this straightforward:
-- Purge all article sync events older than 30 days
DELETE FROM journal_events
WHERE key LIKE 'article.sync%'
AND timestamp < NOW() - INTERVAL '30 days';
Additional context
When events need extra context — e.g. which user triggered them — add it to the context via a custom key and retrieve it in the Handler. The Handler receives the full context.Context, so it can extract whatever the consumer has attached upstream. This keeps the journal API lean while allowing arbitrary enrichment at the persistence layer.
type ctxKey struct{}
func WithUserID(ctx context.Context, userID string) context.Context {
return context.WithValue(ctx, ctxKey{}, userID)
}
func UserIDFromContext(ctx context.Context) string {
id, _ := ctx.Value(ctxKey{}).(string)
return id
}
Then in your handler:
type MyHandler struct {
db *sql.DB
}
func (h *MyHandler) Add(ctx context.Context, lastID string, key string, ts time.Time, data map[string]any) string {
userID := UserIDFromContext(ctx)
id := generateID()
h.db.ExecContext(ctx,
"INSERT INTO journal_events (id, last_id, user_id, key, timestamp, data) VALUES ($1, $2, $3, $4, $5, $6)",
id, lastID, userID, key, ts, data,
)
return id
}
The caller just attaches the user to the context once, and every journal entry in that request automatically gets attributed:
ctx = WithUserID(ctx, "usr_42")
ctx = journal.Setup(ctx, handler)
ctx = journal.Add(ctx, "order.created", map[string]any{"total": 99.95})
// → stored with user_id = "usr_42"
Moving the event ID across boundaries
Event chains don't have to stop at a single service. When one service hands off work to another — e.g. via NATS, Kafka, or an HTTP call — it can include the current event ID so the receiving service continues the chain.
Extract the last ID with journal.LastID(ctx) and pass it along as metadata:
lastID, _ := journal.LastID(ctx)
msg := nats.NewMsg("orders.process")
msg.Data = payload
msg.Header.Set("X-Journal-Last-ID", lastID)
nc.PublishMsg(msg)
On the receiving side, use journal.WithLastID to seed the context so the chain continues seamlessly:
func handleMsg(msg *nats.Msg) {
ctx := context.Background()
ctx = journal.Setup(ctx, handler)
// An empty header (sender had no chain) is a safe no-op.
ctx = journal.WithLastID(ctx, msg.Header.Get("X-Journal-Last-ID"))
// This event now chains from the sender's last event ID.
ctx = journal.Add(ctx, "order.received", nil)
}
This gives you a single audit trail that spans multiple services, connected by the lastID chain.
Performance
Add invokes the handler synchronously, so the handler's write is on the caller's path. That's deliberate: it keeps the event chain sound (each lastID references an event that is actually persisted) and durable (nothing is buffered in memory to be lost on a crash).
The right place to make this fast is the storage layer, not the library. Pick a fast write path — an append-optimised table, a bulk COPY, a log-structured store, or a purpose-built event store — and let it do what it's good at. In-memory buffering or async workers in front of the handler trade away chain integrity and durability for throughput the storage layer should be providing anyway.
Handler implementation
Keep the Handler focused on persistence. Avoid side effects like sending notifications or triggering workflows — journal entries should be a reliable record, not an event bus.
// Good: the handler persists and returns an ID.
func (h *MyHandler) Add(ctx context.Context, lastID string, key string, ts time.Time, data map[string]any) string {
id := generateID()
h.db.ExecContext(ctx, "INSERT INTO journal_events ...", id, lastID, key, ts, data)
return id
}
// Avoid: mixing persistence with side effects.
func (h *MyHandler) Add(ctx context.Context, lastID string, key string, ts time.Time, data map[string]any) string {
id := generateID()
h.db.ExecContext(ctx, "INSERT INTO journal_events ...", id, lastID, key, ts, data)
h.slack.Send("#alerts", fmt.Sprintf("new event: %s", key)) // don't do this
return id
}
Documentation
¶
Overview ¶
Package journal provides context-based event journaling.
A Handler — the user-provided persistence layer — is attached to a context with Setup. Events are then recorded with Add, which threads a lastID through the context so each event links to the one before it, forming an ordered, append-only chain.
Journaling is an optional aspect of the application flow, never a mandatory step: Add is a silent no-op when Setup was not called, when the context is cancelled, or when the handler reports failure. It never surfaces an error to the caller, so it cannot break business logic. All state lives in the context; the package holds no global state.
Example ¶
package main
import (
"context"
"fmt"
"time"
"github.com/go-devkit/journal"
)
// recorder is a minimal Handler standing in for a real persistence layer. It
// assigns sequential IDs and remembers the lastID it was handed.
type recorder struct {
n int
lastSeen string
}
func (h *recorder) Add(_ context.Context, lastID string, _ string, _ time.Time, _ map[string]any) string {
h.n++
h.lastSeen = lastID
return fmt.Sprintf("evt_%d", h.n)
}
func main() {
ctx := journal.Setup(context.Background(), &recorder{})
ctx = journal.Add(ctx, "order.created", map[string]any{"total": 99.95})
ctx = journal.Add(ctx, "order.payment.charged", nil)
id, _ := journal.LastID(ctx)
fmt.Println(id)
}
Output: evt_2
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Add ¶
Add records an event. Journaling is an optional aspect of the application flow, never a mandatory step: it is a silent no-op if Setup was not called or if the context is already cancelled, and the handler owns its own error handling. This keeps call sites clean and ensures journaling never affects application correctness or performance.
func LastID ¶
LastID returns the most recent event ID from the context. The boolean indicates whether an ID is present.
Example ¶
package main
import (
"context"
"fmt"
"time"
"github.com/go-devkit/journal"
)
// recorder is a minimal Handler standing in for a real persistence layer. It
// assigns sequential IDs and remembers the lastID it was handed.
type recorder struct {
n int
lastSeen string
}
func (h *recorder) Add(_ context.Context, lastID string, _ string, _ time.Time, _ map[string]any) string {
h.n++
h.lastSeen = lastID
return fmt.Sprintf("evt_%d", h.n)
}
func main() {
ctx := journal.Setup(context.Background(), &recorder{})
ctx = journal.Add(ctx, "order.created", nil)
id, ok := journal.LastID(ctx)
fmt.Println(id, ok)
}
Output: evt_1 true
func WithLastID ¶
WithLastID seeds the context with an event ID so that the next call to Add chains from it. The event chain is stateless — it lives entirely in the context, so setting this does not interfere with Setup or any prior state. This is useful when continuing an event chain across system boundaries, e.g. after receiving an event ID from another service via NATS or HTTP.
An empty id is a no-op: seeding one would masquerade as a root event and silently break the chain, so the context is returned unchanged.
Example ¶
package main
import (
"context"
"fmt"
"time"
"github.com/go-devkit/journal"
)
// recorder is a minimal Handler standing in for a real persistence layer. It
// assigns sequential IDs and remembers the lastID it was handed.
type recorder struct {
n int
lastSeen string
}
func (h *recorder) Add(_ context.Context, lastID string, _ string, _ time.Time, _ map[string]any) string {
h.n++
h.lastSeen = lastID
return fmt.Sprintf("evt_%d", h.n)
}
func main() {
h := &recorder{}
ctx := journal.Setup(context.Background(), h)
// Continue a chain started elsewhere, e.g. an ID received from another service.
ctx = journal.WithLastID(ctx, "evt_from_upstream")
journal.Add(ctx, "order.received", nil)
fmt.Println(h.lastSeen)
}
Output: evt_from_upstream
Types ¶
type Handler ¶
type Handler interface {
Add(ctx context.Context, lastID string, key string, timestamp time.Time, data map[string]any) string
}
Handler is the user-provided persistence layer. Implementations own their own error handling: journaling is optional and must never surface a failure to the caller. On success, Add returns the new event's ID, used as lastID for the next event in the chain. On failure, return an empty string — the chain is preserved and the failed event is skipped.
Source Files