Package stateio implements persistent state mechanism based on log files that interleave indexed state snapshots with state updates. Users maintain state by interaction with Reader and Writer objects. A typical application should reconcile to the current state before writing new log entries. New log entries should be written only after they are known to apply cleanly. (In the following examples, error handling is left as an exercise to the reader):

file, err := os.OpenFile(filename, os.O_RDWR|os.O_CREATE)
state, epoch, updates, err := stateio.RestoreFile(file)
for {
	entry, err := updates.Read()
	if err == io.EOF {

w, err := stateio.NewFileWriter(file)

// Apply new state updates:
var update []byte = ...
err := w.Update(update)

// Produce a new snapshot:
var snap []byte = application.Snapshot()

Data format

State files maintained by package stateio builds on package logio. The log file contains a sequence of epochs, each beginning with a state snapshot (with the exception of the first epoch, which does not need a state snapshot). Each entry is prefixed with the type of the entry as well as the the epoch to which the entry belongs. Snapshot entries are are prefixed with the previous epoch, so that log files can efficiently rewound.

TODO(marius): support log file truncation



This section is empty.


View Source
var ErrCorrupt = errors.New("corrupt state entry")

    ErrCorrupt is returned when a corrupted log is encountered.


    This section is empty.


    type Reader

    type Reader struct {
    	// contains filtered or unexported fields

      Reader reads a single epoch state updates.

      func Restore

      func Restore(r io.ReaderAt, limit int64) (state []byte, epoch uint64, updates *Reader, err error)

        Restore restores the state from the last epoch in the state log read by the provided reader and the given limit. The returned state may be nil if no snapshot was defined for the epoch.

        func RestoreFile

        func RestoreFile(file *os.File) (state []byte, epoch uint64, updates *Reader, err error)

          RestoreFile is a convenience function that restores the file from the provided os file.

          func (*Reader) Read

          func (r *Reader) Read() ([]byte, error)

            Read returns the next state update entry. Read returns ErrCorrupt if a corrupted log entry was encountered, or logio.ErrCorrupt is a corrupt log file was encountered. In the latter case, the user may skip the corrupted entry by issuing another read.

            type Writer

            type Writer struct {
            	// contains filtered or unexported fields

              Writer writes snapshots and update entries to an underlying log stream.

              func NewFileWriter

              func NewFileWriter(file *os.File) (*Writer, error)

                NewFileWriter initializes a state log writer from the provided os file. The file's contents is committed to stable storage after each log write.

                func NewWriter

                func NewWriter(w io.Writer, off int64, epoch uint64) *Writer

                  NewWriter initializes and returns a new state log writer which writes to the stream w, which is positioned at the provided offset. The provided epoch must be the current epoch of the log file. If the provided io.Writer is also a syncer:

                  type Syncer interface {
                  	Sync() error

                  Then Sync() is called (and errors returned) after each log entry has been written.

                  func (*Writer) Snapshot

                  func (w *Writer) Snapshot(snap []byte) error

                    Snapshot writes a new snapshot to the state log. Subsequent updates are based on this snapshot.

                    func (*Writer) Update

                    func (w *Writer) Update(update []byte) error

                      Update writes a new state update to the log. The update refers to the last snapshot written.