storagedriver

package
Version: v2.0.0-alpha.1+incompa... Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 28, 2015 License: Apache-2.0 Imports: 7 Imported by: 0

README

Docker-Registry Storage Driver

This document describes the registry storage driver model, implementation, and explains how to contribute new storage drivers.

Provided Drivers

This storage driver package comes bundled with three default drivers.

  1. filesystem: A local storage driver configured to use a directory tree in the local filesystem.
  2. s3: A driver storing objects in an Amazon Simple Storage Solution (S3) bucket.
  3. inmemory: A temporary storage driver using a local inmemory map. This exists solely for reference and testing.

Storage Driver API

The storage driver API is designed to model a filesystem-like key/value storage in a manner abstract enough to support a range of drivers from the local filesystem to Amazon S3 or other distributed object storage systems.

Storage drivers are required to implement the storagedriver.StorageDriver interface provided in storagedriver.go, which includes methods for reading, writing, and deleting content, as well as listing child objects of a specified prefix key.

Storage drivers are intended (but not required) to be written in go, providing compile-time validation of the storagedriver.StorageDriver interface, although an IPC driver wrapper means that it is not required for drivers to be included in the compiled registry. The storagedriver/ipc package provides a client/server protocol for running storage drivers provided in external executables as a managed child server process.

Driver Selection and Configuration

The preferred method of selecting a storage driver is using the StorageDriverFactory interface in the storagedriver/factory package. These factories provide a common interface for constructing storage drivers with a parameters map. The factory model is based off of the Register and Open methods in the builtin database/sql package.

Storage driver factories may be registered by name using the factory.Register method, and then later invoked by calling factory.Create with a driver name and parameters map. If no driver is registered with the given name, this factory will attempt to find an executable storage driver with the executable name "registry-storage-<driver name>" and return an IPC storage driver wrapper managing the driver subprocess. If no such storage driver can be found, factory.Create will return an InvalidStorageDriverError.

Driver Contribution

Writing new storage drivers

To create a valid storage driver, one must implement the storagedriver.StorageDriver interface and make sure to expose this driver via the factory system and as a distributable IPC server executable.

In-process drivers

Storage drivers should call factory.Register with their driver name in an init method, allowing callers of factory.New to construct instances of this driver without requiring modification of imports throughout the codebase.

Out-of-process drivers

As many users will run the registry as a pre-constructed docker container, storage drivers should also be distributable as IPC server executables. Drivers written in go should model the main method provided in storagedriver/filesystem/registry-storage-filesystem/filesystem.go. Parameters to IPC drivers will be provided as a JSON-serialized map in the first argument to the process. These parameters should be validated and then a blocking call to ipc.StorageDriverServer should be made with a new storage driver.

Out-of-process drivers must also implement the ipc.IPCStorageDriver interface, which exposes a Version check for the storage driver. This is used to validate storage driver api compatibility at driver load-time.

Testing

Storage driver test suites are provided in storagedriver/testsuites/testsuites.go and may be used for any storage driver written in go. Two methods are provided for registering test suites, RegisterInProcessSuite and RegisterIPCSuite, which run the same set of tests for the driver imported or managed over IPC respectively.

Drivers written in other languages

Although storage drivers are strongly recommended to be written in go for consistency, compile-time validation, and support, the IPC framework allows for a level of language-agnosticism. Non-go drivers must implement the storage driver protocol by mimicing StorageDriverServer in storagedriver/ipc/server.go. As the IPC framework is a layer on top of docker/libchan, this currently limits language support to Java via ndeloof/chan and Javascript via GraftJS/jschan, although contributions to the libchan project are welcome.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrUnsupportedMethod = errors.New("Unsupported method")

UnsupportedMethodErr may be returned in the case where a StorageDriver implementation does not support an optional method.

View Source
var PathRegexp = regexp.MustCompile(`^(/[a-z0-9]+([._-][a-z0-9]+)*)+$`)

PathRegexp is the regular expression which each file path must match. A file path is absolute, beginning with a slash and containing a positive number of path components separated by slashes, where each component is restricted to lowercase alphanumeric characters, optionally separated by a period, underscore, or hyphen.

Functions

This section is empty.

Types

type FileInfo

type FileInfo interface {
	// Path provides the full path of the target of this file info.
	Path() string

	// Size returns current length in bytes of the file. The return value can
	// be used to write to the end of the file at path. The value is
	// meaningless if IsDir returns true.
	Size() int64

	// ModTime returns the modification time for the file. For backends that
	// don't have a modification time, the creation time should be returned.
	ModTime() time.Time

	// IsDir returns true if the path is a directory.
	IsDir() bool
}

FileInfo returns information about a given path. Inspired by os.FileInfo, it elides the base name method for a full path instead.

type FileInfoFields

type FileInfoFields struct {
	// Path provides the full path of the target of this file info.
	Path string

	// Size is current length in bytes of the file. The value of this field
	// can be used to write to the end of the file at path. The value is
	// meaningless if IsDir is set to true.
	Size int64

	// ModTime returns the modification time for the file. For backends that
	// don't have a modification time, the creation time should be returned.
	ModTime time.Time

	// IsDir returns true if the path is a directory.
	IsDir bool
}

FileInfoFields provides the exported fields for implementing FileInfo interface in storagedriver implementations. It should be used with InternalFileInfo.

type FileInfoInternal

type FileInfoInternal struct {
	FileInfoFields
}

FileInfoInternal implements the FileInfo interface. This should only be used by storagedriver implementations that don't have a specialized FileInfo type.

func (FileInfoInternal) IsDir

func (fi FileInfoInternal) IsDir() bool

IsDir returns true if the path is a directory.

func (FileInfoInternal) ModTime

func (fi FileInfoInternal) ModTime() time.Time

ModTime returns the modification time for the file. For backends that don't have a modification time, the creation time should be returned.

func (FileInfoInternal) Path

func (fi FileInfoInternal) Path() string

Path provides the full path of the target of this file info.

func (FileInfoInternal) Size

func (fi FileInfoInternal) Size() int64

Size returns current length in bytes of the file. The return value can be used to write to the end of the file at path. The value is meaningless if IsDir returns true.

type InvalidOffsetError

type InvalidOffsetError struct {
	Path   string
	Offset int64
}

InvalidOffsetError is returned when attempting to read or write from an invalid offset.

func (InvalidOffsetError) Error

func (err InvalidOffsetError) Error() string

type InvalidPathError

type InvalidPathError struct {
	Path string
}

InvalidPathError is returned when the provided path is malformed.

func (InvalidPathError) Error

func (err InvalidPathError) Error() string

type PathNotFoundError

type PathNotFoundError struct {
	Path string
}

PathNotFoundError is returned when operating on a nonexistent path.

func (PathNotFoundError) Error

func (err PathNotFoundError) Error() string

type StorageDriver

type StorageDriver interface {
	// GetContent retrieves the content stored at "path" as a []byte.
	// This should primarily be used for small objects.
	GetContent(path string) ([]byte, error)

	// PutContent stores the []byte content at a location designated by "path".
	// This should primarily be used for small objects.
	PutContent(path string, content []byte) error

	// ReadStream retrieves an io.ReadCloser for the content stored at "path"
	// with a given byte offset.
	// May be used to resume reading a stream by providing a nonzero offset.
	ReadStream(path string, offset int64) (io.ReadCloser, error)

	// WriteStream stores the contents of the provided io.ReadCloser at a
	// location designated by the given path.
	// May be used to resume writing a stream by providing a nonzero offset.
	// The offset must be no larger than the CurrentSize for this path.
	WriteStream(path string, offset int64, reader io.Reader) (nn int64, err error)

	// Stat retrieves the FileInfo for the given path, including the current
	// size in bytes and the creation time.
	Stat(path string) (FileInfo, error)

	// List returns a list of the objects that are direct descendants of the
	//given path.
	List(path string) ([]string, error)

	// Move moves an object stored at sourcePath to destPath, removing the
	// original object.
	// Note: This may be no more efficient than a copy followed by a delete for
	// many implementations.
	Move(sourcePath string, destPath string) error

	// Delete recursively deletes all objects stored at "path" and its subpaths.
	Delete(path string) error

	// URLFor returns a URL which may be used to retrieve the content stored at
	// the given path, possibly using the given options.
	// May return an UnsupportedMethodErr in certain StorageDriver
	// implementations.
	URLFor(path string, options map[string]interface{}) (string, error)
}

StorageDriver defines methods that a Storage Driver must implement for a filesystem-like key/value object storage.

type Version

type Version string

Version is a string representing the storage driver version, of the form Major.Minor. The registry must accept storage drivers with equal major version and greater minor version, but may not be compatible with older storage driver versions.

const CurrentVersion Version = "0.1"

CurrentVersion is the current storage driver Version.

func (Version) Major

func (version Version) Major() uint

Major returns the major (primary) component of a version.

func (Version) Minor

func (version Version) Minor() uint

Minor returns the minor (secondary) component of a version.

Directories

Path Synopsis
Package s3 provides a storagedriver.StorageDriver implementation to store blobs in Amazon S3 cloud storage.
Package s3 provides a storagedriver.StorageDriver implementation to store blobs in Amazon S3 cloud storage.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL