README

wal

Simple, idiomatic write-ahead log.

Usage

Each write-ahead log requires a dedicated directory for writing log files, writing to a directory containing other files is undefined behaviour. Two methods exist to open a new WAL, Open which uses DefaultSize or OpenSize which allows the user to specify a maximum log file size. The log file writer checks the log file size before writing and advances to the next log file if required, but only if the log is not empty, allowing for writes beyond the specified size.

The maximum log file record size is 4294967295 bytes, or 4GB. If larger chunk size is required, the caller has to write the contents in chunks of 4GB and recombine them later.

Documentation

Overview

Package wal implements a Write-Ahead Log.

Writing logs

Each write-ahead log requires a dedicated directory for writing log files, writing to a directory containing other files is undefined behaviour. Two methods exist to open a new log, `Open` which uses `DefaultSize` or `OpenSize` which allows the user to specify a maximum log file size. The log file writer checks the log file size before writing and advances to the next log file if required, but only if the log is not empty, allowing for writes beyond the specified size.

Reading logs

It's the reader's duty to remember where they left off reading. This allows the wal package to return a read only scanner for each Write-Ahead Log.

Record size

The maximum log file record size is 4GiB. If larger chunk size is required, the caller has to write the contents in chunks of 4GiB and recombine them later. Writes that exceed this limit will result in an error.

Index

Examples

Constants

View Source
const (
	// DefaultSize is the default maximum size for a log file.
	DefaultSize = 1 << 24 // 16 MB

	// SyncNever skips file syncing.
	SyncNever time.Duration = -1

	// SyncImmediately syncs after each Write method call.
	SyncImmediately time.Duration = 0
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Log

type Log interface {
	// First entry in the log.
	First() (sequence uint64, exists bool)

	// Last entry in the log.
	Last() (sequence uint64, exists bool)

	// Scanner for the log.
	Scanner(start uint64) Scanner

	// Write new data to the log.
	Write(p []byte) (n int, err error)

	// Close the log.
	Close() error
}

    Log is a Write-Ahead Log implementation. Logs are concurrency safe.

    func Open

    func Open(dir string, syncInterval time.Duration) (Log, error)

      Open a WAL log at the given directory. At most one log may exist in the directory.

      Example
      Output:
      
      

      func OpenSize

      func OpenSize(dir string, syncInterval time.Duration, size int64) (Log, error)

        OpenSize opens a WAL log at the given directory. At most one log may exist in the directory. The supplied size indicates the maximum log file size. If the size <= 0, the DefaultSize shall be used. The sync interval specifies how often the log is synced to disk. If the sync interval == 0, the log is synced immediately after each write. If the sync interval < 0, the log is never synced.

        Example
        Output:
        
        

        type Scanner

        type Scanner interface {
        	// Scan advances the Scanner to the next log record, which will be
        	// available through the Bytes method. After Scan returns false, the
        	// Err method will return any error that occurred during scanning.
        	Scan() bool
        
        	// Bytes returns the most recent token generated by a call to Scan. The
        	// underlying array may point to data that will be overwritten by a
        	// subsequent call to Scan.
        	Bytes() []byte
        
        	// Err returns the last error occurred during scanning. If EOF is
        	// returned, no new log entries are available currently.
        	Err() error
        
        	// Close the scanner and any logs it may be referencing.
        	Close() error
        }

          Scanner can scan over log records. Scanners are not concurrency safe.

          func Read

          func Read(dir string, start uint64) (Scanner, error)

            Read a write-ahead log at the specified directory.

            Example
            Output: