baggageclaim

README

baggage claim

a volume manager for garden containers

by atmx

reporting issues and requesting features

please report all issues and feature requests in concourse/concourse

about

baggageclaim allows you to create and layer volumes on a remote server. This is particularly useful when used with bind mounts or the RootFS when using Garden. It allows directories to be made which can be populated before having copy-on-write layers layered on top. e.g. to provide caching.

Documentation

Overview

Package baggageclaim is the interface for communicating with a BaggageClaim volume server.

BaggageClaim is an auxilary service that can be collocated with various container servers (Garden, Docker, etc.) to let them share directories. BaggageClaim provides a number of benefits over regular bind mounts:

By bringing everything into a the same Volume model we can compose different technologies together. For example, a Docker image is a stack of layered volumes which can have a Concourse build cache layered on top of them.

Volumes can be Copy-on-Write (COW) copies of other volumes. This lets us download a Docker image once and then let it be used by untrusted jobs without fear that they'll mutate it in some unexpected way. This same COW strategy can be applied to any volume that BaggageClaim supports.

BaggageClaim volumes go through a three stage lifecycle of being born, existing, and then dying. This state model is required as creating large amounts of data can potentially take a long time to materialize. You are only able to interact with volumes that are in the middle state.

Volumes are given a TTL which means that BaggageClaim can reclaim the disk space used by the volumes over the life of the service. Parent-child relationships between volumes are understood and will prevent a parent being deleted while a child is still active.

The standard way to construct a client is:

import "github.com/concourse/baggageclaim/client"

bcClient := client.New("http://baggageclaim.example.com:7788")
bcClient.CreateVolume(...)

Index

• Constants
• Variables
• func FinalTTL(dur time.Duration) *time.Duration
• type COWStrategy
  • func (strategy COWStrategy) Encode() *json.RawMessage
• type Client
• type EmptyStrategy
  • func (EmptyStrategy) Encode() *json.RawMessage
• type ImportStrategy
  • func (strategy ImportStrategy) Encode() *json.RawMessage
• type PrivilegedRequest
• type PropertyRequest
• type Strategy
• type TTLRequest
• type Volume
• type VolumeProperties
• type VolumeRequest
• type VolumeResponse
• type VolumeSpec
• type VolumeStatsResponse
• type Volumes

Constants

const (
	ListVolumes    = "ListVolumes"
	GetVolume      = "GetVolume"
	GetVolumeStats = "GetVolumeStats"
	CreateVolume   = "CreateVolume"
	DestroyVolume  = "DestroyVolume"

	SetProperty   = "SetProperty"
	SetTTL        = "SetTTL"
	SetPrivileged = "SetPrivileged"
	StreamIn      = "StreamIn"
	StreamOut     = "StreamOut"
)

Variables

var ErrFileNotFound = errors.New("file not found")

var ErrVolumeNotFound = errors.New("volume not found")

var Routes = rata.Routes{
	{Path: "/volumes", Method: "GET", Name: ListVolumes},
	{Path: "/volumes", Method: "POST", Name: CreateVolume},

	{Path: "/volumes/:handle", Method: "GET", Name: GetVolume},
	{Path: "/volumes/:handle/stats", Method: "GET", Name: GetVolumeStats},
	{Path: "/volumes/:handle/properties/:property", Method: "PUT", Name: SetProperty},
	{Path: "/volumes/:handle/ttl", Method: "PUT", Name: SetTTL},
	{Path: "/volumes/:handle/privileged", Method: "PUT", Name: SetPrivileged},
	{Path: "/volumes/:handle/stream-in", Method: "PUT", Name: StreamIn},
	{Path: "/volumes/:handle/stream-out", Method: "PUT", Name: StreamOut},
	{Path: "/volumes/:handle", Method: "DELETE", Name: DestroyVolume},
}

Functions

func FinalTTL

func FinalTTL(dur time.Duration) *time.Duration

Types

type COWStrategy

type COWStrategy struct {
	// The parent volume that we should base the new volume on.
	Parent Volume
}

COWStrategy creates a Copy-On-Write layer of another Volume.

func (COWStrategy) Encode

func (strategy COWStrategy) Encode() *json.RawMessage

type Client

type Client interface {
	// CreateVolume will create a volume on the remote server. By passing in a
	// VolumeSpec with a different strategy you can choose the type of volume
	// that you want to create.
	//
	// You are required to pass in a logger to the call to retain context across
	// the library boundary.
	//
	// CreateVolume returns the volume that was created or an error as to why it
	// could not be created.
	CreateVolume(lager.Logger, string, VolumeSpec) (Volume, error)

	// ListVolumes lists the volumes that are present on the server. A
	// VolumeProperties object can be passed in to filter the volumes that are in
	// the response.
	//
	// You are required to pass in a logger to the call to retain context across
	// the library boundary.
	//
	// ListVolumes returns the volumes that were found or an error as to why they
	// could not be listed.
	ListVolumes(lager.Logger, VolumeProperties) (Volumes, error)

	// LookupVolume finds a volume that is present on the server. It takes a
	// string that corresponds to the Handle of the Volume.
	//
	// You are required to pass in a logger to the call to retain context across
	// the library boundary.
	//
	// LookupVolume returns a bool if the volume is found with the matching volume
	// or an error as to why the volume could not be found.
	LookupVolume(lager.Logger, string) (Volume, bool, error)
}

Client represents a client connection to a BaggageClaim server.

type EmptyStrategy

type EmptyStrategy struct{}

EmptyStrategy created a new empty volume.

func (EmptyStrategy) Encode

func (EmptyStrategy) Encode() *json.RawMessage

type ImportStrategy

type ImportStrategy struct {
	// The location of the directory on the host to import.
	Path string
}

ImportStrategy creates a volume by copying a directory from the host.

func (ImportStrategy) Encode

func (strategy ImportStrategy) Encode() *json.RawMessage

type PrivilegedRequest

type PrivilegedRequest struct {
	Value bool `json:"value"`
}

type PropertyRequest

type PropertyRequest struct {
	Value string `json:"value"`
}

type Strategy

type Strategy interface {
	Encode() *json.RawMessage
}

type TTLRequest

type TTLRequest struct {
	Value uint `json:"value"`
}

type Volume

type Volume interface {
	// Handle returns a per-server unique identifier for the volume. The URL of
	// the server and a handle is enough to universally identify a volume.
	Handle() string

	// Path returns the filesystem path to the volume on the server. This can be
	// supplied to other systems in order to let them use the volume.
	Path() string

	// SetTTL sets the volume's TTL to an absolute value. An error is returned if
	// the TTL could not be set.
	SetTTL(time.Duration) error

	// SetProperty sets a property on the Volume. Properties can be used to
	// filter the results in the ListVolumes call above.
	SetProperty(key string, value string) error

	// SetPrivileged namespaces or un-namespaces the UID/GID ownership of the
	// volume's contents.
	SetPrivileged(bool) error

	// StreamIn calls BaggageClaim API endpoint in order to initialize tarStream
	// to stream the contents of the Reader into this volume at the specified path.
	StreamIn(path string, tarStream io.Reader) error

	StreamOut(path string) (io.ReadCloser, error)

	Expiration() (time.Duration, time.Time, error)

	// Properties returns the currently set properties for a Volume. An error is
	// returned if these could not be retrieved.
	Properties() (VolumeProperties, error)

	// Release stops the Volume being kept alive by the server. A final TTL can
	// be specified.
	Release(*time.Duration)

	// Size returns the exclusive size of the volume on disk in bytes
	SizeInBytes() (int64, error)

	// Destroy removes the volume and its contents. Note that it does not
	// safeguard against child volumes being present. To safely remove a volume
	// that may have children, set a TTL instead.
	Destroy() error
}

Volume represents a volume in the BaggageClaim system.

type VolumeProperties

type VolumeProperties map[string]string

VolumeProperties represents the properties for a particular volume.

type VolumeRequest

type VolumeRequest struct {
	Handle       string           `json:"handle"`
	Strategy     *json.RawMessage `json:"strategy"`
	Properties   VolumeProperties `json:"properties"`
	TTLInSeconds uint             `json:"ttl,omitempty"`
	Privileged   bool             `json:"privileged,omitempty"`
}

type VolumeResponse

type VolumeResponse struct {
	Handle       string           `json:"handle"`
	Path         string           `json:"path"`
	Properties   VolumeProperties `json:"properties"`
	TTLInSeconds uint             `json:"ttl,omitempty"`
	ExpiresAt    time.Time        `json:"expires_at"`
}

type VolumeSpec

type VolumeSpec struct {
	// Strategy is the information that the server requires to materialise the
	// volume. There are examples of these in this package.
	Strategy Strategy

	// Properties is the set of initial properties that the Volume should have.
	Properties VolumeProperties

	// TTL is the initial TTL of the volume.
	TTL time.Duration

	// Privileged is used to determine whether or not we need to perform a UID
	// translation of the files in the volume so that they can be read by a
	// non-privileged user.
	Privileged bool
}

VolumeSpec is a specification representing the kind of volume that you'd like from the server.

type VolumeStatsResponse

type VolumeStatsResponse struct {
	SizeInBytes int64 `json:"size_in_bytes"`
}

type Volumes

type Volumes []Volume

Volumes represents a list of Volume object.