Documentation
¶
Overview ¶
Package durable provides embeddable durable task execution for a single OS process. Register typed tasks, run memoised steps, and resume from a local journal after a crash. A step may return ErrStepPending and complete later via CompleteStep.
e, err := durable.NewEngine(ctx, "./data")
if err != nil { ... }
defer e.Close()
_ = durable.RegisterTask(e, "process-order", durable.Func(
func(ctx context.Context, s *durable.StepRunner, in OrderInput) (OrderOutput, error) {
charged, err := durable.RunStep(ctx, s, "charge", func(ctx context.Context) (string, error) {
return chargeCard(in)
}).Get(ctx)
if err != nil {
return OrderOutput{}, err
}
return OrderOutput{Result: charged}, nil
},
))
run := durable.RunTask[OrderInput, OrderOutput](ctx, e, "process-order", "", input)
output, err := run.Get(ctx)
Index ¶
- Variables
- func CompleteStep[O any](ctx context.Context, e *Engine, token string, result O) error
- func RegisterTask[I, O any](e *Engine, taskID string, task Task[I, O], opts ...TaskOption) error
- type Engine
- func (e *Engine) Close() error
- func (e *Engine) DeleteTask(ctx context.Context, taskID string) error
- func (e *Engine) DeleteTaskRun(ctx context.Context, taskID, runID string) error
- func (e *Engine) GetTask(ctx context.Context, taskID, runID string) (TaskInfo, bool, error)
- func (e *Engine) ListTasks(ctx context.Context, statuses ...TaskStatus) ([]TaskInfo, error)
- func (e *Engine) LoadSteps(ctx context.Context, taskID, runID string) ([]StepRecord, error)
- type EngineOption
- type ReadOnlyEngine
- func (r *ReadOnlyEngine) Close() error
- func (r *ReadOnlyEngine) GetTask(ctx context.Context, taskID, runID string) (TaskInfo, bool, error)
- func (r *ReadOnlyEngine) ListTasks(ctx context.Context, statuses ...TaskStatus) ([]TaskInfo, error)
- func (r *ReadOnlyEngine) LoadSteps(ctx context.Context, taskID, runID string) ([]StepRecord, error)
- type ReadOnlyOption
- type RunOption
- type StepOption
- type StepRecord
- type StepRun
- type StepRunner
- type StepStatus
- type Task
- type TaskFunc
- type TaskInfo
- type TaskOption
- type TaskRun
- type TaskStatus
Constants ¶
This section is empty.
Variables ¶
var ( // ErrTaskNotRegistered is returned by RunTask.Get when the taskID has not // been registered on this Engine. The registry is in-memory only and must // be rebuilt after every NewEngine call. ErrTaskNotRegistered = errors.New("durable: task not registered") // ErrTaskAlreadyRegistered is returned by RegisterTask when the same // taskID is registered twice on one Engine. ErrTaskAlreadyRegistered = errors.New("durable: task already registered") // ErrRunAlreadyFinished is returned by CompleteStep when the target step // is already completed or the run is in a terminal state (completed or failed). ErrRunAlreadyFinished = errors.New("durable: run already completed or failed") // ErrRunActive is returned by DeleteTaskRun and DeleteTask when the target run // (or any run under the task) is currently executing, including while // blocked in StatusWaiting. ErrRunActive = errors.New("durable: run is currently executing") // ErrInvalidRunID is returned when a runID contains path separators or // parent-directory references that would escape the data directory. ErrInvalidRunID = errors.New("durable: invalid run ID") // ErrEngineLocked is returned by NewEngine and NewReadOnlyEngine when // dataDir is already held by another engine instance (same process or // another OS process) and the lock cannot be acquired before the timeout. ErrEngineLocked = errors.New("durable: dataDir locked by another engine") // ErrStepPending is returned from a step function to suspend the run // until CompleteStep delivers a result for that step. The engine writes // StepStatusWaiting and blocks the run goroutine. ErrStepPending = errors.New("durable: step pending external completion") // ErrInvalidToken is returned by CompleteStep when the token cannot be // decoded into taskID, runID, and stepID. ErrInvalidToken = errors.New("durable: invalid step token") )
Functions ¶
func CompleteStep ¶ added in v0.1.2
CompleteStep delivers result to a suspended step identified by token. The payload is appended as a SignalEntry (durable) before the in-process waiter is signalled, so a crash after this call still resumes on the next RunTask. A second call with the same token is a no-op once the SignalEntry is on disk. Returns ErrRunAlreadyFinished if the step or run is already terminal, and ErrInvalidToken if the token cannot be decoded.
func RegisterTask ¶ added in v0.1.2
RegisterTask stores taskID → (closure + config) in the in-memory registry. Must be called before RunTask. Re-registering the same taskID returns ErrTaskAlreadyRegistered. The registry is not persisted; call this again after every NewEngine. taskID must not contain path separators or ':'.
Types ¶
type Engine ¶ added in v0.1.2
type Engine struct {
// contains filtered or unexported fields
}
Engine is the process-level entry point for durable task execution. Create one per dataDir via NewEngine. Safe for concurrent use.
Always call Close to cancel in-flight runs, release the exclusive flock, stop the purger, and close journal file handles.
func NewEngine ¶ added in v0.1.2
NewEngine opens or creates dataDir, acquires an exclusive flock on <dataDir>/.lock, and initialises the in-memory task registry. Fails with ErrEngineLocked if another Engine or ReadOnlyEngine holds the directory.
func (*Engine) Close ¶ added in v0.1.2
Close cancels in-flight runs, waits for them and the auto-purger to finish writing, then releases the exclusive flock and journal handles. Safe to call more than once.
func (*Engine) DeleteTask ¶ added in v0.1.2
DeleteTask removes all runs under taskID. Destructive — use for a full wipe only. Returns ErrRunActive if any run under taskID is currently executing (Running or Waiting in-process).
func (*Engine) DeleteTaskRun ¶ added in v0.1.2
DeleteTaskRun removes one run directory and its journal. No-op if not found. Returns ErrRunActive if the run is currently executing, including while blocked in StatusWaiting — runLocks is held for that entire duration.
func (*Engine) GetTask ¶ added in v0.1.2
GetTask returns a single run's metadata. (zero, false, nil) if not found.
type EngineOption ¶ added in v0.1.2
type EngineOption func(*engineConfig)
EngineOption configures NewEngine.
func WithAutoPurge ¶
func WithAutoPurge(age time.Duration, interval ...time.Duration) EngineOption
WithAutoPurge starts a background goroutine that deletes Completed and Failed runs whose UpdatedAt is older than age. The optional interval controls how often the purger runs; it defaults to one hour. Running and Waiting runs are never purged.
func WithLockTimeout ¶ added in v0.1.2
func WithLockTimeout(d time.Duration) EngineOption
WithLockTimeout sets how long NewEngine waits for the exclusive flock. Default is 2 seconds.
func WithLogger ¶
func WithLogger(l *slog.Logger) EngineOption
WithLogger sets the slog.Logger used for task and step lifecycle events. If nil or omitted, a discard logger is used.
func WithMaxRetries ¶
func WithMaxRetries(n int) EngineOption
WithMaxRetries sets the engine-wide default for task-level retries (re-invoking the task closure). Default is 0 — retries are opt-in so non-idempotent work is not silently repeated. Overridden by WithTaskMaxRetries and WithRunMaxRetries.
func WithTimeout ¶
func WithTimeout(d time.Duration) EngineOption
WithTimeout sets the engine-wide default task deadline. Zero (default) means no timeout. Overridden by WithTaskTimeout and WithRunTimeout.
type ReadOnlyEngine ¶ added in v0.1.2
type ReadOnlyEngine struct {
// contains filtered or unexported fields
}
ReadOnlyEngine is a compile-time-restricted view of a dataDir. It acquires a shared flock so multiple readers can coexist. It cannot register or run tasks. Safe to open from a separate CLI process with no task registration.
func NewReadOnlyEngine ¶ added in v0.1.2
func NewReadOnlyEngine(dataDir string, opts ...ReadOnlyOption) (*ReadOnlyEngine, error)
NewReadOnlyEngine acquires a shared flock on <dataDir>/.lock. Multiple readers coexist. Returns ErrEngineLocked after the lock timeout if a writer holds the exclusive lock.
func (*ReadOnlyEngine) Close ¶ added in v0.1.2
func (r *ReadOnlyEngine) Close() error
Close releases the shared flock. Safe to call more than once.
func (*ReadOnlyEngine) GetTask ¶ added in v0.1.2
GetTask returns a single run's metadata. (zero, false, nil) if not found.
func (*ReadOnlyEngine) ListTasks ¶ added in v0.1.2
func (r *ReadOnlyEngine) ListTasks(ctx context.Context, statuses ...TaskStatus) ([]TaskInfo, error)
ListTasks returns TaskInfo records with the same filter semantics as Engine.ListTasks.
func (*ReadOnlyEngine) LoadSteps ¶ added in v0.1.2
func (r *ReadOnlyEngine) LoadSteps(ctx context.Context, taskID, runID string) ([]StepRecord, error)
LoadSteps returns all StepRecords for a run ordered by Seq.
type ReadOnlyOption ¶ added in v0.1.2
type ReadOnlyOption func(*readOnlyConfig)
ReadOnlyOption configures NewReadOnlyEngine.
func WithROLockTimeout ¶ added in v0.1.2
func WithROLockTimeout(d time.Duration) ReadOnlyOption
WithROLockTimeout sets how long NewReadOnlyEngine waits for the shared flock. Default is 2 seconds. Named distinctly from WithLockTimeout because both option types live in the same package.
func WithROLogger ¶ added in v0.1.2
func WithROLogger(l *slog.Logger) ReadOnlyOption
WithROLogger sets the slog.Logger for the read-only engine. Named distinctly from WithLogger because both option types live in the same package.
type RunOption ¶ added in v0.1.2
type RunOption func(*runConfig)
RunOption configures a single RunTask call.
func WithRunMaxRetries ¶ added in v0.1.2
WithRunMaxRetries overrides the task and engine task-level retry count for this run. An explicit 0 disables a non-zero parent default.
func WithRunTimeout ¶ added in v0.1.2
WithRunTimeout overrides the task and engine timeout for this run. An explicit 0 disables a non-zero parent timeout.
type StepOption ¶ added in v0.1.2
type StepOption func(*stepConfig)
StepOption configures a single RunStep call.
func WithStepMaxRetries ¶ added in v0.1.2
func WithStepMaxRetries(n int) StepOption
WithStepMaxRetries sets how many times this step's function is re-invoked on a non-panic, non-ErrStepPending error. Default is 0 (one attempt). Does not inherit from task or engine retry settings.
func WithStepTimeout ¶ added in v0.1.2
func WithStepTimeout(d time.Duration) StepOption
WithStepTimeout is an inner bound: the step deadline is min(task deadline, step timeout). It cannot extend past the task deadline.
type StepRecord ¶
type StepRecord struct {
StepID string
Seq int
InputHash string // unused in v1; RunStep has no input parameter
Result []byte
Error string
PanicTrace string
Status StepStatus
StartedAt time.Time
CompletedAt time.Time
}
StepRecord is the persistent checkpoint for one memoised step.
type StepRun ¶ added in v0.1.2
type StepRun[O any] struct { // contains filtered or unexported fields }
StepRun is the handle returned by RunStep. Get returns immediately — RunStep itself is synchronous (including the wait on ErrStepPending).
func RunStep ¶ added in v0.1.2
func RunStep[O any](ctx context.Context, s *StepRunner, stepID string, fn func(ctx context.Context) (O, error), opts ...StepOption) *StepRun[O]
RunStep executes fn as a memoised checkpoint. On a completed cache hit, fn is not called. On a miss, fn runs synchronously and the result is persisted. stepID must be unique within the run — a duplicate panics. Concurrent calls on the same StepRunner panic.
type StepRunner ¶
type StepRunner struct {
// contains filtered or unexported fields
}
StepRunner is scoped to a single run and provides RunStep, StepToken, and run-context accessors. It is not safe for concurrent use.
func (*StepRunner) Logger ¶ added in v0.1.2
func (s *StepRunner) Logger() *slog.Logger
Logger returns the engine logger pre-scoped with taskID and runID.
func (*StepRunner) RunID ¶ added in v0.1.2
func (s *StepRunner) RunID() string
RunID returns the runID of the current run.
func (*StepRunner) StepSeq ¶ added in v0.1.2
func (s *StepRunner) StepSeq() int
StepSeq returns the number of steps executed so far. Zero before the first RunStep call; incremented after each RunStep returns (including cache hits).
func (*StepRunner) StepToken ¶ added in v0.1.2
func (s *StepRunner) StepToken() string
StepToken returns an opaque token encoding taskID, runID, and the current stepID. Must be called inside a RunStep function — panics if currentStepID is empty. Pass the token to an external caller (DB, email, webhook URL) so they can later call CompleteStep.
func (*StepRunner) TaskID ¶ added in v0.1.2
func (s *StepRunner) TaskID() string
TaskID returns the taskID of the current run.
type StepStatus ¶
type StepStatus string
StepStatus is the lifecycle state of a single step checkpoint.
const ( // StepStatusWaiting means the step is suspended and awaiting CompleteStep. StepStatusWaiting StepStatus = "waiting" // StepStatusCompleted means the step succeeded and its result is cached. StepStatusCompleted StepStatus = "completed" // StepStatusFailed means the step returned an error or panicked. StepStatusFailed StepStatus = "failed" )
type Task ¶
type Task[I, O any] interface { // Exec performs the task logic. ctx is cancelled when the task timeout // elapses or the engine is closed. s is the StepRunner bound to this // run; wrap all memoised work in RunStep calls. Panics are recovered // by the engine, recorded in TaskInfo.PanicTrace, and surfaced to Get. Exec(ctx context.Context, s *StepRunner, input I) (O, error) }
Task is the execution contract for a durable task. Implementations should treat any work with external side effects as a RunStep; non-deterministic logic outside of RunStep calls may not be replayed correctly. I is the input type; O is the output type.
type TaskFunc ¶
type TaskFunc[I, O any] func(ctx context.Context, s *StepRunner, input I) (O, error)
TaskFunc adapts a plain function to Task[I, O].
type TaskInfo ¶
type TaskInfo struct {
TaskID string `json:"task_id"`
RunID string `json:"run_id"`
Name string `json:"name"`
Tags map[string]string `json:"tags"`
Status TaskStatus `json:"status"`
Error string `json:"error"`
PanicTrace string `json:"panic_trace"`
CreatedAt time.Time `json:"created_at"`
StartedAt time.Time `json:"started_at"`
CompletedAt time.Time `json:"completed_at"`
UpdatedAt time.Time `json:"updated_at"`
}
TaskInfo is the persistent metadata for a single run. Input is intentionally omitted — callers own input persistence and reload it on recovery.
type TaskOption ¶
type TaskOption func(*taskConfig)
TaskOption configures RegisterTask.
func WithName ¶
func WithName(name string) TaskOption
WithName sets a human-readable label stored on TaskInfo. It does not affect uniqueness or execution.
func WithTag ¶
func WithTag(k, v string) TaskOption
WithTag attaches an arbitrary key-value annotation to TaskInfo. Call multiple times to set multiple tags.
func WithTaskMaxRetries ¶ added in v0.1.2
func WithTaskMaxRetries(n int) TaskOption
WithTaskMaxRetries overrides the engine default for task-level retries (re-invoking the whole closure). Nil-vs-set is tracked so an explicit 0 disables a non-zero engine default.
func WithTaskTimeout ¶ added in v0.1.2
func WithTaskTimeout(d time.Duration) TaskOption
WithTaskTimeout overrides the engine default task deadline. An explicit 0 disables a non-zero engine default.
type TaskRun ¶ added in v0.1.2
type TaskRun[O any] struct { // contains filtered or unexported fields }
TaskRun is the handle returned by RunTask. RunID is available immediately; Get blocks until the run reaches a terminal state.
func RunTask ¶ added in v0.1.2
func RunTask[I, O any](ctx context.Context, e *Engine, taskID string, runID string, input I, opts ...RunOption) *TaskRun[O]
RunTask starts or resumes a run in a background goroutine and returns immediately. Pass an empty runID to resume the oldest active run for taskID, or to generate a new ULID if none is active. Pass an existing runID to resume; a completed or failed run returns the stored result without spawning a goroutine.
func (*TaskRun[O]) Get ¶ added in v0.1.2
Get blocks until the run completes or ctx is cancelled. The typed result is cached after the first successful wait so later calls do not re-decode.
func (*TaskRun[O]) RunID ¶ added in v0.1.2
RunID returns the resolved runID. Empty if the taskID was not registered or the runID was invalid — check Get for the error.
func (*TaskRun[O]) Status ¶ added in v0.1.2
func (r *TaskRun[O]) Status() TaskStatus
Status returns the current run status without blocking.
type TaskStatus ¶
type TaskStatus string
TaskStatus is the lifecycle state of a run.
const ( // StatusRunning means the run is actively executing. StatusRunning TaskStatus = "running" // StatusWaiting means at least one step is awaiting CompleteStep. StatusWaiting TaskStatus = "waiting" // StatusCompleted means the run finished successfully. StatusCompleted TaskStatus = "completed" // StatusFailed means the run terminated with an error or recovered panic. StatusFailed TaskStatus = "failed" )
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
func-task
command
Package main demonstrates the functional / closure style of durable-go.
|
Package main demonstrates the functional / closure style of durable-go. |
|
resume
command
Package main demonstrates durable-go's core value: crash recovery and step replay.
|
Package main demonstrates durable-go's core value: crash recovery and step replay. |
|
struct-task
command
Package main demonstrates the struct / method-receiver style of durable-go.
|
Package main demonstrates the struct / method-receiver style of durable-go. |
|
Package durablepb contains the protobuf-generated wire types for journal.log entries.
|
Package durablepb contains the protobuf-generated wire types for journal.log entries. |