stacker

README

stacker

Stacker is a tool for building OCI images via a declarative yaml format.

Installation

Stacker has various build and runtime dependencies.

Usage

See the tutorial for a short introduction to how to use stacker.

See the stacker.yaml specification for full details on the stacker.yaml specification.

Additionally, there are some tips and tricks for common usage.

TODO / Roadmap

• Upstream something to containers/image that allows for automatic detection of compression
• Design/implement OCIv2 drafts + final spec when it comes out
• Rearrange tests so there's more unprivileged coverage (really should be another row/column in the test matrix like storage type is)

Conference Talks

• An Operator Centric Way to Update Application Containers FOSDEM 2019
  • video
  • slides
• Building OCI Images without Privilege OSS EU 2018
  • slides
• Building OCI Images without Privilege OSS NA 2018
  • slides

(Note that despite the similarity in name of the 2018 talks, the content is mostly disjoint; I need to be more creative with naming.)

License

stacker is released under the Apache License, Version 2.0, and is:

Copyright (C) 2017-2021 Cisco Systems, Inc.

Documentation

Overview

* This file is a little bit strange. The problem is that we want to do * daemonized containers with liblxc, but we can't spawn containers in threaded * environments (i.e. golang), with go-lxc. So instead, we embed some C into * our program that catches execution before golang starts. This way, we can do * a tiny C program to actually spawn the container. * * We do this in the "stacker" package so that if anyone uses the library, the * re-exec will actually work. Of course, this is slightly impolite, but what * can you do.

Index

• Constants
• func CleanImportsDir(c types.StackerConfig, name string, imports types.Imports, cache *BuildCache) error
• func Download(cacheDir string, url string, progress bool, remoteHash string, ...) (string, error)
• func GenerateShellForRunning(rootfs string, cmd []string, outFile string) error
• func GetBase(o BaseLayerOpts) error
• func GitVersion(path string) (string, error)
• func Grab(sc types.StackerConfig, storage types.Storage, name string, source string, ...) error
• func Import(c types.StackerConfig, storage types.Storage, name string, ...) error
• func NewGitLayerTag(path string) (string, error)
• func NewStorage(c types.StackerConfig) (types.Storage, error)
• func SetupRootfs(o BaseLayerOpts) error
• func UnprivSetup(c types.StackerConfig, uid, gid int) error
• type BaseLayerOpts
• type BuildArgs
• type BuildCache
  • func OpenCache(config types.StackerConfig, oci casext.Engine, sfm types.StackerFiles) (*BuildCache, error)
  • func (c *BuildCache) Lookup(name string) (*CacheEntry, bool, error)
  • func (c *BuildCache) Put(name string, manifests map[types.LayerType]ispec.Descriptor) error
• type Builder
  • func NewBuilder(opts *BuildArgs) *Builder
  • func (b *Builder) Build(s types.Storage, file string) error
  • func (b *Builder) BuildMultiple(paths []string) error
• type CacheEntry
• type Container
  • func NewContainer(sc types.StackerConfig, storage types.Storage, name string) (*Container, error)
  • func (c *Container) Close()
  • func (c *Container) Execute(args string, stdin io.Reader) error
  • func (c *Container) SetupLayerConfig(l *types.Layer, name string) error
• type ImportHash
• type ImportType
  • func (it ImportType) IsDir() bool
• type PublishArgs
• type Publisher
  • func NewPublisher(opts *PublishArgs) *Publisher
  • func (p *Publisher) Publish(file string) error
  • func (p *Publisher) PublishMultiple(paths []string) error
• type StackerFilesDAG
  • func NewStackerFilesDAG(sfMap types.StackerFiles) (*StackerFilesDAG, error)
  • func (d *StackerFilesDAG) GetStackerFile(path string) *types.Stackerfile
  • func (d *StackerFilesDAG) Sort() []string

Constants

const (
	GitVersionAnnotation      = "com.cisco.stacker.git_version"
	StackerContentsAnnotation = "com.cisco.stacker.stacker_yaml"
)

const (
	ReasonableDefaultPath = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
)

Functions

func CleanImportsDir

func CleanImportsDir(c types.StackerConfig, name string, imports types.Imports, cache *BuildCache) error

func Download

func Download(cacheDir string, url string, progress bool, remoteHash string, remoteSize string) (string, error)

download with caching support in the specified cache dir.

func GenerateShellForRunning

func GenerateShellForRunning(rootfs string, cmd []string, outFile string) error

GenerateShellForRunning generates a shell script to run inside the container, and writes it to the contianer. It checks that the script already have a shebang? If so, it leaves it as is, otherwise it prepends a shebang.

func GetBase

func GetBase(o BaseLayerOpts) error

GetBase grabs the base layer and puts it in the cache.

func GitVersion

func GitVersion(path string) (string, error)

GitVersion generates a version string similar to what git describe --always does, with -dirty on the end if the git repo had local changes.

func Grab

func Grab(sc types.StackerConfig, storage types.Storage, name string, source string, targetDir string) error

func Import

func Import(c types.StackerConfig, storage types.Storage, name string, imports types.Imports, progress bool) error

func NewGitLayerTag

func NewGitLayerTag(path string) (string, error)

NewGitLayerTag version generates a commit-<id> tag to be used for uploading an image to a docker registry

func NewStorage

func NewStorage(c types.StackerConfig) (types.Storage, error)

func SetupRootfs

func SetupRootfs(o BaseLayerOpts) error

SetupRootfs assumes the base layer is correct in the cache, and sets up the base to the output.

If the layer is a build only layer, this code simply initializes the filesystem in roots to the built tag's filesystem.

func UnprivSetup

func UnprivSetup(c types.StackerConfig, uid, gid int) error

Types

type BaseLayerOpts

type BaseLayerOpts struct {
	Config     types.StackerConfig
	Name       string
	Layer      *types.Layer
	Cache      *BuildCache
	OCI        casext.Engine
	LayerTypes []types.LayerType
	Storage    types.Storage
	Progress   bool
}

type BuildArgs

type BuildArgs struct {
	Config       types.StackerConfig
	LeaveUnladen bool
	NoCache      bool
	Substitute   []string
	OnRunFailure string
	LayerTypes   []types.LayerType
	OrderOnly    bool
	SetupOnly    bool
	Progress     bool
}

type BuildCache

type BuildCache struct {
	Cache   map[string]CacheEntry `json:"cache"`
	Version int                   `json:"version"`
	// contains filtered or unexported fields
}

func OpenCache

func OpenCache(config types.StackerConfig, oci casext.Engine, sfm types.StackerFiles) (*BuildCache, error)

func (*BuildCache) Lookup

func (c *BuildCache) Lookup(name string) (*CacheEntry, bool, error)

func (*BuildCache) Put

func (c *BuildCache) Put(name string, manifests map[types.LayerType]ispec.Descriptor) error

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder is responsible for building the layers based on stackerfiles

func NewBuilder

func NewBuilder(opts *BuildArgs) *Builder

NewBuilder initializes a new Builder struct

func (*Builder) Build

func (b *Builder) Build(s types.Storage, file string) error

Build builds a single stackerfile

func (*Builder) BuildMultiple

func (b *Builder) BuildMultiple(paths []string) error

BuildMultiple builds a list of stackerfiles

type CacheEntry

type CacheEntry struct {
	// A map of LayerType:Manifest this build corresponds to.
	Manifests map[types.LayerType]ispec.Descriptor

	// A map of the import url to the base64 encoded result of mtree walk
	// or sha256 sum of a file, depending on what Type is.
	Imports map[string]ImportHash

	// The name of this layer as it was built. Useful for the BuildOnly
	// case to make sure it still exists, and for printing error messages.
	Name string

	// The layer to cache
	Layer *types.Layer

	// If the layer is of type "built", this is a hash of the base layer's
	// CacheEntry, which contains a hash of its imports. If there is a
	// mismatch with the current base layer's CacheEntry, the layer should
	// be rebuilt.
	Base string
}

type Container

type Container struct {
	// contains filtered or unexported fields
}

our representation of a container

func NewContainer

func NewContainer(sc types.StackerConfig, storage types.Storage, name string) (*Container, error)

func (*Container) Close

func (c *Container) Close()

func (*Container) Execute

func (c *Container) Execute(args string, stdin io.Reader) error

func (*Container) SetupLayerConfig

func (c *Container) SetupLayerConfig(l *types.Layer, name string) error

type ImportHash

type ImportHash struct {
	// Unfortuantely, mtree doesn't work if you just pass it a single file,
	// so we use the sha256sum of the file, or the mtree encoding if it's a
	// directory. This indicates which.
	Type ImportType
	Hash string
}

type ImportType

type ImportType int

const (
	ImportFile ImportType = iota
	ImportDir  ImportType = iota
)

func (ImportType) IsDir

func (it ImportType) IsDir() bool

type PublishArgs

type PublishArgs struct {
	Config     types.StackerConfig
	ShowOnly   bool
	Substitute []string
	Tags       []string
	Url        string
	Username   string
	Password   string
	Force      bool
	Progress   bool
	LayerTypes []types.LayerType
}

type Publisher

type Publisher struct {
	// contains filtered or unexported fields
}

Publisher is responsible for publishing the layers based on stackerfiles

func NewPublisher

func NewPublisher(opts *PublishArgs) *Publisher

NewPublisher initializes a new Publisher struct

func (*Publisher) Publish

func (p *Publisher) Publish(file string) error

Publish layers in a single stackerfile

func (*Publisher) PublishMultiple

func (p *Publisher) PublishMultiple(paths []string) error

PublishMultiple published layers defined in a list of stackerfiles

type StackerFilesDAG

type StackerFilesDAG struct {
	// contains filtered or unexported fields
}

StackerDepsDAG processes the dependencies between different stacker recipes

func NewStackerFilesDAG

func NewStackerFilesDAG(sfMap types.StackerFiles) (*StackerFilesDAG, error)

NewStackerDepsDAG properly initializes a StackerDepsProcessor

func (*StackerFilesDAG) GetStackerFile

func (d *StackerFilesDAG) GetStackerFile(path string) *types.Stackerfile

func (*StackerFilesDAG) Sort

func (d *StackerFilesDAG) Sort() []string

Sort provides a serial build order for the stacker files