shift

README

Shift

Shift provides the SQL persistence layer for a simple "finite state machine" domain model. It provides validation, explicit fields and reflex events per state change. It is therefore used to explicitly define the life cycle of the domain model, i.e., the states it can transition through and the data modifications required for each transition.

Overview

A Shift state machine is composed of an initial state followed by multiple subsequent states linked by allowed transitions, i.e., a rooted directed graph.

stateDiagram-v2
    direction LR
    [*] --> Created
    Created --> Pending
    Pending --> Failed
    Pending --> Completed
    Failed --> Pending
    Completed --> [*]

Each state has an associated struct defining the data modified when entering the state.

type create struct {
  UserID string
  Type   int
}

type pending struct {
  ID int64
}

type failed struct {
  ID    int64
  Error string
}

type completed struct {
  ID     int64
  Result string
}

Some properties:

• States are instances of an enum implementing shift.Status interface.
• A state has an allowed set of next states.
• Only one state can be the initial state.
• All subsequent states are reached by explicit transitions from a state.
• Cycles are allowed; transitioning to an upstream state or even to itself.
• It is not allowed to transition to the initial state.
• Entering the initial state always inserts a new row.
• The initial state's struct may therefore not contain an ID field.
• Entering a subsequent states always updates an existing row.
• Subsequent states' structs must therefore contain an ID field.
• int64 and string ID fields are supported.
• Created and updated times are guaranteed to be reliable:
  • By default, time.Now() is used to set the timestamp columns.
  • If specified in the inserter or updater, shift will use the provided time. This can be useful for testing.
  • Shift will error if a zero time is provided (i.e. if time is not set)
  • Columns must be named created_at and updated_at
• All transitions are recorded as reflex events.

Differences of ArcFSM from FSM:

• For improved flexibility, ArcFSM was added without the transition restrictions of FSM.
• It supports arbitrary initial states and arbitrary transitions.

Usage

The above state machine is defined by:

events := rsql.NewEventsTableInt("events")
fsm := shift.NewFSM(events)
  Insert(CREATED, create{}, PENDING).
  Update(PENDING, pending{}, COMPLETED, FAILED).
  Update(FAILED, failed{}, PENDING).
  Update(COMPLETED, completed{}).
  Build()
  
// Note the format: STATE, struct{}, NEXT_STATE_A, NEXT_STATE_B    

Shift requires the state structs to implement Inserter or Updater interfaces which performs the actual SQL queries.

A command shiftgen is provided that generates SQL boilerplate to implement these interfaces.

//go:generate shiftgen -inserter=create -updaters=pending,failed,completed -table=mysql_table_name

The fsm instance is then used by the business logic to drive the state machine.

// Insert a new domain model (in the CREATED) state.
id, err := fsm.Insert(ctx, dbc, create{"user123",TypeDefault})

// Update it from CREATED to PENDING 
err = fsm.Update(ctx, dbc, CREATED, PENDING, pending{id})

// Update it from PENDING to COMPLETED 
err = fsm.Update(ctx, dbc, PENDING, COMPLETED, completed{id, "success!"})

Note that the terms "state" and "status" are effective synonyms in this case. We found "state" to be an overtaxed term, so we use "status" in the code instead.

See GoDoc for details and this example.

Why?

Controlling domain model life cycle with Shift state machines provide the following benefits:

• Improved maintainability since everything is explicit.
• The code acts as documentation for the business logic.
• Decreased chance of inconsistent state.
• State transitions generate events, which other services subscribe to.
• Complex logic is broken down into discrete steps.
• Possible to avoid distributed transactions.

Shift state machines allow for robust fault tolerant systems that are easy to understand and maintain.

Documentation

Overview

Package shift provides the persistence layer for a simple "finite state machine" domain model with validation, explicit fields and reflex events per state change.

shift.NewFSM builds a FSM instance that allows specific mutations of the domain model in the underlying sql table via inserts and updates. All mutations update the status of the model, mutates some fields and inserts a reflex event. Note that FSM is opinionated and has the following restrictions: only a single insert status, no transitions back to insert status, only a single transition per pair of statuses.

shift.NewArcFSM builds a ArcFSM instance which is the same as an FSM but without its restrictions. It supports arbitrary transitions.

Index

• Variables
• func NewArcFSM(events eventInserter[int64], opts ...option) arcbuilder
• func NewFSM(events eventInserter[int64], opts ...option) initer[int64]
• func NewGenFSM[T primary](events eventInserter[T], opts ...option) initer[T]
• func TestFSM(_ testing.TB, dbc *sql.DB, fsm *FSM) error
• func WithMetadata() option
• func WithValidation() option
• type ArcFSM
  • func (fsm *ArcFSM) Insert(ctx context.Context, dbc *sql.DB, st Status, inserter Inserter[int64]) (int64, error)
  • func (fsm *ArcFSM) InsertTx(ctx context.Context, tx *sql.Tx, st Status, inserter Inserter[int64]) (int64, rsql.NotifyFunc, error)
  • func (fsm *ArcFSM) IsValidTransition(from Status, to Status) bool
  • func (fsm *ArcFSM) Update(ctx context.Context, dbc *sql.DB, from, to Status, updater Updater[int64]) error
  • func (fsm *ArcFSM) UpdateTx(ctx context.Context, tx *sql.Tx, from, to Status, updater Updater[int64]) (rsql.NotifyFunc, error)
• type FSM
• type GenFSM
  • func (fsm *GenFSM[T]) Insert(ctx context.Context, dbc *sql.DB, inserter Inserter[T]) (T, error)
  • func (fsm *GenFSM[T]) InsertTx(ctx context.Context, tx *sql.Tx, inserter Inserter[T]) (T, rsql.NotifyFunc, error)
  • func (fsm *GenFSM[T]) IsValidTransition(from Status, to Status) bool
  • func (fsm *GenFSM[T]) Update(ctx context.Context, dbc *sql.DB, from Status, to Status, updater Updater[T]) error
  • func (fsm *GenFSM[T]) UpdateTx(ctx context.Context, tx *sql.Tx, from Status, to Status, updater Updater[T]) (rsql.NotifyFunc, error)
• type Inserter
• type MetadataInserter
• type MetadataUpdater
• type Status
• type Updater
• type ValidatingInserter
• type ValidatingUpdater

Variables

var ErrInvalidStateTransition = errors.New("invalid state transition", j.C("ERR_be8211db784bfb67"))

ErrInvalidStateTransition indicates a state transition that hasn't been registered with the FSM.

var ErrInvalidType = errors.New("invalid type", j.C("ERR_baf1a1f2e99951ec"))

ErrInvalidType indicates that the provided request type isn't valid, and can't be used for the requested transition.

var ErrRowCount = errors.New("unexpected number of rows updated", j.C("ERR_fcb8af57223847b1"))

ErrRowCount is returned by generated shift code when an update failed due unexpected number of rows updated (n != 1). This is usually due to the row not being in the expected from state anymore.

var ErrUnknownStatus = errors.New("unknown status", j.C("ERR_198a4c2d8a654b17"))

ErrUnknownStatus indicates that the status hasn't been registered with the FSM.

Functions

func NewArcFSM

func NewArcFSM(events eventInserter[int64], opts ...option) arcbuilder

NewArcFSM returns a new ArcFSM builder.

func NewFSM

func NewFSM(events eventInserter[int64], opts ...option) initer[int64]

NewFSM returns a new FSM initer that supports a user table with an int64 primary key.

func NewGenFSM

func NewGenFSM[T primary](events eventInserter[T], opts ...option) initer[T]

NewGenFSM returns a new FSM initer. The type T should match the type of the user table's primary key.

func TestFSM

func TestFSM(_ testing.TB, dbc *sql.DB, fsm *FSM) error

TestFSM tests the provided FSM instance by driving it through all possible state transitions using fuzzed data. It ensures all states are reachable and that the sql queries match the schema.

func WithMetadata

func WithMetadata() option

WithMetadata provides an option to enable event metadata with an FSM.

func WithValidation

func WithValidation() option

WithValidation provides an option to enable insert/update validation.

Types

type ArcFSM

type ArcFSM struct {
	// contains filtered or unexported fields
}

ArcFSM is a defined Finite-State-Machine that allows specific mutations of the domain model in the underlying sql table via inserts and updates. All mutations update the status of the model, mutates some fields and inserts a reflex event.

ArcFSM doesn't have the restriction of FSM and can be defined with arbitrary transitions.

func (*ArcFSM) Insert

func (fsm *ArcFSM) Insert(ctx context.Context, dbc *sql.DB, st Status, inserter Inserter[int64]) (int64, error)

func (*ArcFSM) InsertTx

func (fsm *ArcFSM) InsertTx(ctx context.Context, tx *sql.Tx, st Status, inserter Inserter[int64]) (int64, rsql.NotifyFunc, error)

func (*ArcFSM) IsValidTransition

func (fsm *ArcFSM) IsValidTransition(from Status, to Status) bool

IsValidTransition validates status transition without committing the transaction

func (*ArcFSM) Update

func (fsm *ArcFSM) Update(ctx context.Context, dbc *sql.DB, from, to Status, updater Updater[int64]) error

func (*ArcFSM) UpdateTx

func (fsm *ArcFSM) UpdateTx(ctx context.Context, tx *sql.Tx, from, to Status, updater Updater[int64]) (rsql.NotifyFunc, error)

type FSM

type FSM = GenFSM[int64]

type GenFSM

type GenFSM[T primary] struct {
	// contains filtered or unexported fields
}

GenFSM is a defined Finite-State-Machine that allows specific mutations of the domain model in the underlying sql table via inserts and updates. All mutations update the status of the model, mutates some fields and inserts a reflex event.

The type of the GenFSM is the type of the primary key used by the user table.

Note that this FSM is opinionated and has the following restrictions: only a single insert status, no transitions back to insert status, only a single transition per pair of statuses.

func (*GenFSM[T]) Insert

func (fsm *GenFSM[T]) Insert(ctx context.Context, dbc *sql.DB, inserter Inserter[T]) (T, error)

Insert returns the id of the newly inserted domain model.

func (*GenFSM[T]) InsertTx

func (fsm *GenFSM[T]) InsertTx(ctx context.Context, tx *sql.Tx, inserter Inserter[T]) (T, rsql.NotifyFunc, error)

func (*GenFSM[T]) IsValidTransition

func (fsm *GenFSM[T]) IsValidTransition(from Status, to Status) bool

IsValidTransition validates status transition without committing the transaction

func (*GenFSM[T]) Update

func (fsm *GenFSM[T]) Update(ctx context.Context, dbc *sql.DB, from Status, to Status, updater Updater[T]) error

func (*GenFSM[T]) UpdateTx

func (fsm *GenFSM[T]) UpdateTx(ctx context.Context, tx *sql.Tx, from Status, to Status, updater Updater[T]) (rsql.NotifyFunc, error)

type Inserter

type Inserter[T primary] interface {
	// Insert inserts a new row with status and returns an id or an error.
	Insert(ctx context.Context, tx *sql.Tx, status Status) (T, error)
}

Inserter provides an interface for inserting new state machine instance rows.

type MetadataInserter

type MetadataInserter[T primary] interface {
	Inserter[T]

	// GetMetadata returns the metadata to be inserted with the reflex event for the insert.
	GetMetadata(ctx context.Context, tx *sql.Tx, id T, status Status) ([]byte, error)
}

MetadataInserter extends inserter with additional metadata inserted with the reflex event.

type MetadataUpdater

type MetadataUpdater[T primary] interface {
	Updater[T]

	// GetMetadata returns the metadata to be inserted with the reflex event for the update.
	GetMetadata(ctx context.Context, tx *sql.Tx, from Status, to Status) ([]byte, error)
}

MetadataUpdater extends updater with additional metadata inserted with the reflex event.

type Status

type Status interface {
	ShiftStatus() int
	ReflexType() int
}

Status is an individual state in the FSM.

The canonical implementation is:

type MyStatus int
func (s MyStatus) ShiftStatus() int {
	return int(s)
}
func (s MyStatus) ReflexType() int {
	return int(s)
}
const (
	StatusUnknown MyStatus = 0
	StatusInsert  MyStatus = 1
)

type Updater

type Updater[T primary] interface {
	// Update updates the status of an existing row returns an id or an error.
	Update(ctx context.Context, tx *sql.Tx, from Status, to Status) (T, error)
}

Updater provides an interface for updating existing state machine instance rows.

type ValidatingInserter

type ValidatingInserter[T primary] interface {
	Inserter[T]

	// Validate returns an error if the insert is not valid.
	Validate(ctx context.Context, tx *sql.Tx, id T, status Status) error
}

ValidatingInserter extends inserter with validation. Assuming the majority validations will be successful, the validation is done after event insertion to allow maximum flexibility sacrificing invalid path performance.

type ValidatingUpdater

type ValidatingUpdater[T primary] interface {
	Updater[T]

	// Validate returns an error if the update is not valid.
	Validate(ctx context.Context, tx *sql.Tx, from Status, to Status) error
}

ValidatingUpdater extends updater with validation. Assuming the majority validations will be successful, the validation is done after event insertion to allow maximum flexibility sacrificing invalid path performance.