wal

Documentation

Overview

Package wal provides an implementation of a write ahead log that is used by etcd.

A WAL is created at a particular directory and is made up of a number of discrete WAL files. Inside of each file the raft state and entries are appended to it with the Save method:

metadata := []byte{}
w, err := wal.Create("/var/lib/etcd", metadata)
...
err := w.Save(s, ents)

When a user has finished using a WAL it must be closed:

w.Close()

WAL files are placed inside of the directory in the following format: $seq-$index.wal

The first WAL file to be created will be 0000000000000000-0000000000000000.wal indicating an initial sequence of 0 and an initial raft index of 0. The first entry written to WAL MUST have raft index 0.

Periodically a user will want to "cut" the WAL and place new entries into a new file. This will increment an internal sequence number and cause a new file to be created. If the last raft index saved was 0x20 and this is the first time Cut has been called on this WAL then the sequence will increment from 0x0 to 0x1. The new file will be: 0000000000000001-0000000000000021.wal. If a second Cut issues 0x10 entries with incremental index later then the file will be called: 0000000000000002-0000000000000031.wal.

At a later time a WAL can be opened at a particular raft index:

w, err := wal.OpenAtIndex("/var/lib/etcd", 0)
...

The raft index must have been written to the WAL. When opening without a snapshot the raft index should always be 0. When opening with a snapshot the raft index should be the index of the last entry covered by the snapshot.

Additional items cannot be Saved to this WAL until all of the items from 0 to the end of the WAL are read first:

id, state, ents, err := w.ReadAll()

This will give you the raft node id, the last raft.State and the slice of raft.Entry items in the log.

Index

• Variables
• func Exist(dirpath string) bool
• func MultiReadCloser(readClosers ...io.ReadCloser) io.ReadCloser
• type WAL
  • func Create(dirpath string, metadata []byte) (*WAL, error)
  • func OpenAtIndex(dirpath string, index uint64) (*WAL, error)
  • func (w *WAL) Close() error
  • func (w *WAL) Cut() error
  • func (w *WAL) ReadAll() (metadata []byte, state raftpb.HardState, ents []raftpb.Entry, err error)
  • func (w *WAL) Save(st raftpb.HardState, ents []raftpb.Entry) error
  • func (w *WAL) SaveEntry(e *raftpb.Entry) error
  • func (w *WAL) SaveState(s *raftpb.HardState) error
• type WalVersion
  • func DetectVersion(dirpath string) (WalVersion, error)

Variables

var (
	ErrMetadataConflict = errors.New("wal: conflicting metadata found")
	ErrFileNotFound     = errors.New("wal: file not found")
	ErrCRCMismatch      = errors.New("wal: crc mismatch")
)

Functions

func Exist

func Exist(dirpath string) bool

func MultiReadCloser

func MultiReadCloser(readClosers ...io.ReadCloser) io.ReadCloser

Types

type WAL

type WAL struct {
	// contains filtered or unexported fields
}

WAL is a logical repersentation of the stable storage. WAL is either in read mode or append mode but not both. A newly created WAL is in append mode, and ready for appending records. A just opened WAL is in read mode, and ready for reading records. The WAL will be ready for appending after reading out all the previous records.

func Create

func Create(dirpath string, metadata []byte) (*WAL, error)

Create creates a WAL ready for appending records. The given metadata is recorded at the head of each WAL file, and can be retrieved with ReadAll.

func OpenAtIndex

func OpenAtIndex(dirpath string, index uint64) (*WAL, error)

OpenAtIndex opens the WAL at the given index. The index SHOULD have been previously committed to the WAL, or the following ReadAll will fail. The returned WAL is ready to read and the first record will be the given index. The WAL cannot be appended to before reading out all of its previous records.

func (*WAL) Close

func (w *WAL) Close() error

func (*WAL) Cut

func (w *WAL) Cut() error

Cut closes current file written and creates a new one ready to append.

func (*WAL) ReadAll

func (w *WAL) ReadAll() (metadata []byte, state raftpb.HardState, ents []raftpb.Entry, err error)

ReadAll reads out all records of the current WAL. If it cannot read out the expected entry, it will return ErrIndexNotFound. After ReadAll, the WAL will be ready for appending new records.

func (*WAL) Save

func (w *WAL) Save(st raftpb.HardState, ents []raftpb.Entry) error

func (*WAL) SaveEntry

func (w *WAL) SaveEntry(e *raftpb.Entry) error

func (*WAL) SaveState

func (w *WAL) SaveState(s *raftpb.HardState) error

type WalVersion

type WalVersion string

WalVersion is an enum for versions of etcd logs.

const (
	WALUnknown  WalVersion = "Unknown WAL"
	WALNotExist WalVersion = "No WAL"
	WALv0_4     WalVersion = "0.4.x"
	WALv0_5     WalVersion = "0.5.x"
)

func DetectVersion

func DetectVersion(dirpath string) (WalVersion, error)