continuity

README

continuity

A transport-agnostic, filesystem metadata manifest system

This project is a staging area for experiments in providing transport agnostic metadata storage.

Please see https://github.com/opencontainers/specs/issues/11 for more details.

Building Proto Package

If you change the proto file you will need to rebuild the generated Go with go generate.

go generate ./proto

Documentation

Index

• Variables
• func ApplyManifest(ctx Context, manifest *Manifest) error
• func Marshal(m *Manifest) ([]byte, error)
• func MarshalText(w io.Writer, m *Manifest) error
• func VerifyManifest(ctx Context, manifest *Manifest) error
• type ByPath
  • func (bp ByPath) Len() int
  • func (bp ByPath) Less(i, j int) bool
  • func (bp ByPath) Swap(i, j int)
• type ContentProvider
• type Context
  • func NewContext(root string) (Context, error)
  • func NewContextWithOptions(root string, options ContextOptions) (Context, error)
• type ContextOptions
• type Device
• type DeviceInfoDriver
• type Digester
• type Directory
• type Driver
  • func NewSystemDriver() (Driver, error)
• type Hardlinkable
• type LXAttrDriver
• type Manifest
  • func BuildManifest(ctx Context) (*Manifest, error)
  • func Unmarshal(p []byte) (*Manifest, error)
• type NamedPipe
• type RegularFile
• type Resource
  • func Merge(fs ...Resource) (Resource, error)
• type SymLink
• type SymlinkPath
• type XAttrDriver
• type XAttrer

Variables

var (
	ErrNotFound     = fmt.Errorf("not found")
	ErrNotSupported = fmt.Errorf("not supported")
)

Functions

func ApplyManifest

func ApplyManifest(ctx Context, manifest *Manifest) error

ApplyManifest applies on the resources in a manifest to the given context.

func Marshal

func Marshal(m *Manifest) ([]byte, error)

func MarshalText

func MarshalText(w io.Writer, m *Manifest) error

func VerifyManifest

func VerifyManifest(ctx Context, manifest *Manifest) error

VerifyManifest verifies all the resources in a manifest against files from the given context.

Types

type ByPath

type ByPath []Resource

ByPath provides the canonical sort order for a set of resources. Use with sort.Stable for deterministic sorting.

func (ByPath) Len

func (bp ByPath) Len() int

func (ByPath) Less

func (bp ByPath) Less(i, j int) bool

func (ByPath) Swap

func (bp ByPath) Swap(i, j int)

type ContentProvider

type ContentProvider interface {
	Reader(digest.Digest) (io.ReadCloser, error)
}

ContentProvider produces a read stream for a given digest

type Context

type Context interface {
	Apply(Resource) error
	Verify(Resource) error
	Resource(string, os.FileInfo) (Resource, error)
	Walk(filepath.WalkFunc) error
}

Context represents a file system context for accessing resources. The responsibility of the context is to convert system specific resources to generic Resource objects. Most of this is safe path manipulation, as well as extraction of resource details.

func NewContext

func NewContext(root string) (Context, error)

NewContext returns a Context associated with root. The default driver will be used, as returned by NewDriver.

func NewContextWithOptions

func NewContextWithOptions(root string, options ContextOptions) (Context, error)

NewContextWithOptions returns a Context associate with the root.

type ContextOptions

type ContextOptions struct {
	Digester Digester
	Driver   Driver
	Provider ContentProvider
}

type Device

type Device interface {
	Resource
	Hardlinkable
	XAttrer

	Major() uint64
	Minor() uint64
}

type DeviceInfoDriver

type DeviceInfoDriver interface {
	DeviceInfo(fi os.FileInfo) (maj uint64, min uint64, err error)
}

type Digester

type Digester interface {
	Digest(io.Reader) (digest.Digest, error)
}

Digester produces a digest for a given read stream

type Directory

type Directory interface {
	Resource
	XAttrer

	// Directory is a no-op method to identify directory objects by interface.
	Directory()
}

type Driver

type Driver interface {
	Open(path string) (*os.File, error)
	Stat(path string) (os.FileInfo, error)
	Lstat(path string) (os.FileInfo, error)
	Readlink(p string) (string, error)
	Mkdir(path string, mode os.FileMode) error
	Remove(path string) error

	Link(oldname, newname string) error
	Lchmod(path string, mode os.FileMode) error
	Lchown(path, uid, gid string) error
	Symlink(oldname, newname string) error

	// TODO(aaronl): These methods might move outside the main Driver
	// interface in the future as more platforms are added.
	Mknod(path string, mode os.FileMode, major int, minor int) error
	Mkfifo(path string, mode os.FileMode) error
}

Driver provides all of the system-level functions in a common interface. The context should call these with full paths and should never use the `os` package or any other package to access resources on the filesystem. This mechanism let's us carefully control access to the context and maintain path and resource integrity. It also gives us an interface to reason about direct resource access.

Implementations don't need to do much other than meet the interface. For example, it is not required to wrap os.FileInfo to return correct paths for the call to Name().

func NewSystemDriver

func NewSystemDriver() (Driver, error)

type Hardlinkable

type Hardlinkable interface {
	// Paths returns all paths of the resource, including the primary path
	// returned by Resource.Path. If len(Paths()) > 1, the resource is a hard
	// link.
	Paths() []string
}

Hardlinkable is an interface that a resource type satisfies if it can be a hardlink target.

type LXAttrDriver

type LXAttrDriver interface {
	// LGetxattr returns all of the extended attributes for the file at path
	// and does not follow symlinks. Typically, this takes a syscall call to
	// Llistxattr and Lgetxattr.
	LGetxattr(path string) (map[string][]byte, error)

	// LSetxattr sets all of the extended attributes on file at path, without
	// following symbolic links. All attributes on the target are replaced by
	// the values from attr. If the operation fails to set any attribute,
	// those already applied will not be rolled back.
	LSetxattr(path string, attr map[string][]byte) error
}

LXAttrDriver should be implemented by drivers on operating systems and filesystems that support setting and getting extended attributes on symbolic links. If this is not implemented, extended attributes will be ignored on symbolic links.

type Manifest

type Manifest struct {
	// Resources specifies all the resources for a manifest in order by path.
	Resources []Resource
}

Manifest provides the contents of a manifest. Users of this struct should not typically modify any fields directly.

func BuildManifest

func BuildManifest(ctx Context) (*Manifest, error)

BuildManifest creates the manifest for the given context

func Unmarshal

func Unmarshal(p []byte) (*Manifest, error)

type NamedPipe

type NamedPipe interface {
	Resource
	Hardlinkable
	XAttrer

	// Pipe is a no-op method to allow consistent resolution of NamedPipe
	// interface.
	Pipe()
}

type RegularFile

type RegularFile interface {
	Resource
	XAttrer
	Hardlinkable

	Size() int64
	Digests() []digest.Digest
}

type Resource

type Resource interface {
	// Path provides the primary resource path relative to the bundle root. In
	// cases where resources have more than one path, such as with hard links,
	// this will return the primary path, which is often just the first entry.
	Path() string

	// Mode returns the
	Mode() os.FileMode

	UID() string
	GID() string
}

func Merge

func Merge(fs ...Resource) (Resource, error)

Merge two or more Resources into new file. Typically, this should be used to merge regular files as hardlinks. If the files are not identical, other than Paths and Digests, the merge will fail and an error will be returned.

type SymLink

type SymLink interface {
	Resource

	// Target returns the target of the symlink contained in the .
	Target() string
}

type SymlinkPath

type SymlinkPath func(root, linkname, target string) (string, error)

SymlinkPath is intended to give the symlink target value in a root context. Target and linkname are absolute paths not under the given root.

type XAttrDriver

type XAttrDriver interface {
	// Getxattr returns all of the extended attributes for the file at path.
	// Typically, this takes a syscall call to Listxattr and Getxattr.
	Getxattr(path string) (map[string][]byte, error)

	// Setxattr sets all of the extended attributes on file at path, following
	// any symbolic links, if necessary. All attributes on the target are
	// replaced by the values from attr. If the operation fails to set any
	// attribute, those already applied will not be rolled back.
	Setxattr(path string, attr map[string][]byte) error
}

XAttrDriver should be implemented on operation systems and filesystems that have xattr support for regular files and directories.

type XAttrer

type XAttrer interface {
	XAttrs() map[string][]byte
}