README

Build Status GoDoc Go Report Card

The renameio Go package provides a way to atomically create or replace a file or symbolic link.

Atomicity vs durability

renameio concerns itself only with atomicity, i.e. making sure applications never see unexpected file content (a half-written file, or a 0-byte file).

As a practical example, consider https://manpages.debian.org/: if there is a power outage while the site is updating, we are okay with losing the manpages which were being rendered at the time of the power outage. They will be added in a later run of the software. We are not okay with having a manpage replaced by a 0-byte file under any circumstances, though.

Advantages of this package

There are other packages for atomically replacing files, and sometimes ad-hoc implementations can be found in programs.

A naive approach to the problem is to create a temporary file followed by a call to os.Rename(). However, there are a number of subtleties which make the correct sequence of operations hard to identify:

  • The temporary file should be removed when an error occurs, but a remove must not be attempted if the rename succeeded, as a new file might have been created with the same name. This renders a throwaway defer os.Remove(t.Name()) insufficient; state must be kept.

  • The temporary file must be created on the same file system (same mount point) for the rename to work, but the TMPDIR environment variable should still be respected, e.g. to direct temporary files into a separate directory outside of the webserver’s document root but on the same file system.

  • On POSIX operating systems, the fsync system call must be used to ensure that the os.Rename() call will not result in a 0-length file.

This package attempts to get all of these details right, provides an intuitive, yet flexible API and caters to use-cases where high performance is required.

Disclaimer

This is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google.

This project is not affiliated with the Go project.

Documentation

Overview

Package renameio provides a way to atomically create or replace a file or symbolic link.

Caveat: this package requires the file system rename(2) implementation to be atomic. Notably, this is not the case when using NFS with multiple clients: https://stackoverflow.com/a/41396801

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Symlink(oldname, newname string) error

Symlink wraps os.Symlink, replacing an existing symlink with the same name atomically (os.Symlink fails when newname already exists, at least on Linux).

func TempDir

func TempDir(dest string) string

TempDir checks whether os.TempDir() can be used as a temporary directory for later atomically replacing files within dest. If no (os.TempDir() resides on a different mount point), dest is returned.

Note that the returned value ceases to be valid once either os.TempDir() changes (e.g. on Linux, once the TMPDIR environment variable changes) or the file system is unmounted.

func WriteFile

func WriteFile(filename string, data []byte, perm os.FileMode) error

WriteFile mirrors ioutil.WriteFile, replacing an existing file with the same name atomically.

Types

type PendingFile

type PendingFile struct {
	*os.File
	// contains filtered or unexported fields
}

PendingFile is a pending temporary file, waiting to replace the destination path in a call to CloseAtomicallyReplace.

func TempFile

func TempFile(dir, path string) (*PendingFile, error)

TempFile wraps ioutil.TempFile for the use case of atomically creating or replacing the destination file at path.

If dir is the empty string, TempDir(filepath.Base(path)) is used. If you are going to write a large number of files to the same file system, store the result of TempDir(filepath.Base(path)) and pass it instead of the empty string.

The file's permissions will be 0600 by default. You can change these by explicitly calling Chmod on the returned PendingFile.

Example (Justone)
Output:

Example (Many)
Output:

func (*PendingFile) Cleanup

func (t *PendingFile) Cleanup() error

Cleanup is a no-op if CloseAtomicallyReplace succeeded, and otherwise closes and removes the temporary file.

func (*PendingFile) CloseAtomicallyReplace

func (t *PendingFile) CloseAtomicallyReplace() error

CloseAtomicallyReplace closes the temporary file and atomically replaces the destination file with it, i.e., a concurrent open(2) call will either open the file previously located at the destination path (if any), or the just written file, but the file will always be present.