lxcri

package module
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 30, 2021 License: Apache-2.0 Imports: 19 Imported by: 0

README

About

lxcri is a wrapper around LXC which can be used as a drop-in container runtime replacement for use by CRI-O.

History

The project started as a fork of lxc/crio-lxc but has undergone several refactorings and yet shares very little code with lxc/crio-lxc and was therefore renamed to lxcri

OCI compliance

With liblxc >= https://github.com/lxc/lxc/commit/b5daeddc5afce1cad4915aef3e71fdfe0f428709 it passes all sonobuoy conformance tests.

Installation

For the installation of the runtime see INSTALL.md
For the installation and initialization of a kubernetes cluster see K8S.md

Glossary

  • runtime the lxcri binary and the command set that implement the OCI runtime spec
  • container process the process that starts and runs the container using liblxc (lxcri-start)
  • container config the LXC config file
  • bundle config the lxcri container state (bundle path, pidfile ....)
  • runtime spec the OCI runtime spec from the bundle

Bugs

Requirements and restrictions

  • Only cgroupv2 (unified cgroup hierarchy) is supported.
  • A recent kernel >= 5.8 is required for full cgroup support.
Unimplemented features

Configuration

The runtime binary implements flags that are required by the OCI runtime spec,
and flags that are runtime specific (timeouts, hooks, logging ...).

Most of the runtime specific flags have corresponding environment variables. See lxcri --help.
The runtime evaluates the flag value in the following order (lower order takes precedence).

  1. cmdline flag from process arguments (overwrites process environment)
  2. process environment variable (overwrites environment file)
  3. environment file (overwrites cmdline flag default)
  4. cmdline flag default
Environment variables

Currently you have to compile to environment file yourself.
To list all available variables:

grep EnvVars cmd/cli.go | grep -o LXCRI_[A-Za-z_]* | xargs -n1 -I'{}' echo "#{}="
Environment file

The default path to the environment file is /etc/defaults/lxcri.
It is loaded on every start of the lxcri binary, so changes take immediate effect.
Empty lines and those commented with a leading # are ignored.

A malformed environment will let the next runtime call fail.
In production it's recommended that you replace the environment file atomically.

E.g the environment file /etc/default/lxcri could look like this:

LXCRI_LOG_LEVEL=debug
LXCRI_CONTAINER_LOG_LEVEL=debug
#LXCRI_LOG_FILE=
#LXCRI_LOG_TIMESTAMP=
#LXCRI_MONITOR_CGROUP=
#LXCRI_LIBEXEC=
#LXCRI_APPARMOR=
#LXCRI_CAPABILITIES=
#LXCRI_CGROUP_DEVICES=
#LXCRI_SECCOMP=
#LXCRI_CREATE_TIMEOUT=
#LXCRI_CREATE_HOOK=/usr/local/bin/lxcri-backup.sh
#LXCRI_CREATE_HOOK_TIMEOUT=
#LXCRI_START_TIMEOUT=
#LXCRI_KILL_TIMEOUT=
#LXCRI_DELETE_TIMEOUT=
Runtime (security) features

All supported runtime security features are enabled by default.
The following runtime (security) features can optionally be disabled.
Details see lxcri --help

  • apparmor
  • capabilities
  • cgroup-devices
  • seccomp
Logging

There is only a single log file for runtime and container process log output.
The log-level for the runtime and the container process can be set independently.

  • containers are ephemeral, but the log file should not be
  • a single logfile is easy to rotate and monitor
  • a single logfile is easy to tail (watch for errors / events ...)
  • robust implementation is easy
Log Filtering

Runtime log lines are written in JSON using zerolog.
The log file can be easily filtered with jq.
For filtering with jq you must strip the container process logs with grep -v '^lxc'

E.g Filter show only errors and warnings for runtime create command:

 grep -v '^lxc ' /var/log/lxcri.log |\
  jq -c 'select(.cmd == "create" and ( .l == "error or .l == "warn")'
Runtime log fields

Fields that are always present:

  • l log level
  • m log message
  • c caller (source file and line number)
  • cid container ID
  • cmd runtime command
  • t timestamp in UTC (format matches container process output)

Log message specific fields:

  • pid a process ID
  • file a path to a file
  • lxc.config the key of a container process config item
  • env the key of an environment variable
Debugging

Apart from the logfile following resources are useful:

  • Systemd journal for cri-o and kubelet services
  • coredumpctl if runtime or container process segfaults.
Create Hook

If a create hook is defined, it is executed before the create command returns.
You can use it to backup the runtime spec and container process config for further analysis.

The create hook executable must

  • not use the standard file descriptors (stdin/stdout/stderr) although they are nulled.
  • not exceed LXCRI_CREATE_HOOK_TIMEOUT or it is killed.
  • not modify/delete any resources created by the runtime or container process

The process environment contains the following variables:

  • CONTAINER_ID the container ID
  • LXC_CONFIG the path to runtime process config
  • RUNTIME_CMD the runtime command which executed the runtime hook
  • RUNTIME_PATH the path to the container runtime directory
  • BUNDLE_PATH the absolute path to the container bundle
  • LOG_FILE the path to the log file
  • RUNTIME_ERROR (optional) the error message if the runtime cmd return with error

Example script lxcri-backup.sh that backs up any container runtime directory:

#!/bin/sh

LOGDIR=$(dirname $LOG_FILE)
OUT=$LOGDIR/$CONTAINER_ID

# backup container runtime directory to log directory
cp -r $RUNTIME_PATH $OUT
# copy OCI runtime spec to container runtime directory

# remove non `grep` friendly runtime files (symlinks, binaries, fifos)
rm $OUT/.lxcri/cwd
rm $OUT/.lxcri/init
rm $OUT/.lxcri/syncfifo

Documentation

Index

Constants

View Source 
const (
	// ExecStart starts the liblxc monitor process, similar to lxc-start
	ExecStart = "lxcri-start"
	// ExecHook is run as liblxc hook and creates additional devices and remounts masked paths.
	ExecHook = "lxcri-hook"
	// ExecInit is the container init process that execs the container process.
	ExecInit = "lxcri-init"
)

Required runtime executables loaded from Runtime.LibexecDir

Variables

View Source 
var (
	CgroupNamespace  = Namespace{"cgroup", unix.CLONE_NEWCGROUP}
	IPCNamespace     = Namespace{"ipc", unix.CLONE_NEWIPC}
	MountNamespace   = Namespace{"mnt", unix.CLONE_NEWNS}
	NetworkNamespace = Namespace{"net", unix.CLONE_NEWNET}
	PIDNamespace     = Namespace{"pid", unix.CLONE_NEWPID}
	TimeNamespace    = Namespace{"time", unix.CLONE_NEWTIME}
	UserNamespace    = Namespace{"user", unix.CLONE_NEWUSER}
	UTSNamespace     = Namespace{"uts", unix.CLONE_NEWUTS}
)
View Source 
var (
	ErrNotExist = fmt.Errorf("container does not exist")
	ErrExist    = fmt.Errorf("container already exists")
)
View Source 
var DefaultRuntime = &Runtime{
	Log:           log.ConsoleLogger(true),
	Root:          "/var/run/lxcri",
	SystemdCgroup: true,
	LibexecDir:    "/usr/libexec/lxcri",

	Features: RuntimeFeatures{
		Seccomp:       true,
		Capabilities:  true,
		Apparmor:      true,
		CgroupDevices: true,
	},
}

Functions

func CheckSystem added in v0.9.8 

func CheckSystem() error

CheckSystem is a wrapper around DefaultRuntime.CheckSystem

func Delete added in v0.9.8 

func Delete(ctx context.Context, c *Container, force bool) error

Delete is a wrapper around DefaultRuntime.Delete

func Kill added in v0.9.8 

func Kill(ctx context.Context, c *Container, signum unix.Signal) error

Kill is a wrapper around DefaultRuntime.Kill

func ReadSpecProcessJSON 

func ReadSpecProcessJSON(src string) (*specs.Process, error)

func Start added in v0.9.8 

func Start(ctx context.Context, c *Container) error

Start is a wrapper around DefaultRuntime.Start

Types

type Container 

type Container struct {
	LinuxContainer *lxc.Container `json:"-"`
	*ContainerConfig

	CreatedAt time.Time
	Pid       int
}

Container is the runtime state of a container instance.

func Create added in v0.9.8 

func Create(ctx context.Context, cfg *ContainerConfig) (*Container, error)

Create is a wrapper around DefaultRuntime.Create

func Load added in v0.9.8 

func Load(cfg *ContainerConfig) (*Container, error)

Load is a wrapper around DefaultRuntime.Load

func (*Container) ContainerState 

func (c *Container) ContainerState() (specs.ContainerState, error)

func (*Container) Exec 

func (c *Container) Exec(args []string, proc *specs.Process) (exitStatus int, err error)

func (*Container) ExecDetached 

func (c *Container) ExecDetached(args []string, proc *specs.Process) (pid int, err error)

func (*Container) GetConfigItem 

func (c *Container) GetConfigItem(key string) string

func (*Container) Release 

func (c *Container) Release() error

func (*Container) SetConfigItem 

func (c *Container) SetConfigItem(key, value string) error

func (*Container) State 

func (c *Container) State() (*specs.State, error)

func (*Container) SupportsConfigItem 

func (c *Container) SupportsConfigItem(keys ...string) bool

type ContainerConfig 

type ContainerConfig struct {
	*specs.Spec

	RuntimeDir string

	ContainerID string

	BundlePath    string
	ConsoleSocket string `json:",omitempty"`

	// PidFile is the absolute PID file path
	// for the container monitor process (ExecStart)
	MonitorCgroupDir string

	CgroupDir string

	// LogFile is the liblxc log file path
	LogFile string
	// LogLevel is the liblxc log level
	LogLevel string

	// Log is the container Logger
	Log zerolog.Logger `json:"-"`

	Hooks `json:"-"`
}

ContainerConfig is the configuration for a single Container instance.

func (ContainerConfig) ConfigFilePath 

func (cfg ContainerConfig) ConfigFilePath() string

func (*ContainerConfig) LoadSpecJson 

func (c *ContainerConfig) LoadSpecJson(p string) error

func (ContainerConfig) RuntimePath 

func (cfg ContainerConfig) RuntimePath(subPath ...string) string

RuntimePath returns the absolute path within the container root.

type Hooks added in v0.9.8 

type Hooks struct {
	// OnCreate is called right after creation of container runtime directory
	// and descriptor, but before the liblxc 'config' file is written.
	// At this point it's possible to add files to the container runtime directory
	// and modify the ContainerConfig.
	OnCreate RuntimeHook
}

RuntimeHooks are callback functions executed within the container lifecycle.

type Namespace 

type Namespace struct {
	Name      string
	CloneFlag int
}

type Runtime 

type Runtime struct {
	// Log is the logger used by the runtime.
	Log zerolog.Logger `json:"-"`
	// Root is the file path to the runtime directory.
	// Directories for containers created by the runtime
	// are created within this directory.
	Root string
	// Use systemd encoded cgroup path (from crio-o/conmon)
	// is true if /etc/crio/crio.conf#cgroup_manager = "systemd"
	SystemdCgroup bool
	// Path for lxc monitor cgroup (lxc specific feature)
	// similar to /etc/crio/crio.conf#conmon_cgroup
	MonitorCgroup string
	// LibexecDir is the the directory that contains the runtime executables.
	LibexecDir string
	//
	Features RuntimeFeatures

	Hooks `json:"-"`
}

func (*Runtime) CheckSystem 

func (rt *Runtime) CheckSystem() error

CheckSystem checks the hosts system configuration. Unsupported runtime features are disabled and a warning message is logged. CheckSystem should be called (once) before using the Runtime.

func (*Runtime) Create 

func (rt *Runtime) Create(ctx context.Context, cfg *ContainerConfig) (*Container, error)

Create creates a single container instance from the given ContainerConfig. Create is the first runtime method to call within the lifecycle of a container. You may have to call Runtime.Delete to cleanup container runtime state, if Create returns with an error.

func (*Runtime) Delete 

func (rt *Runtime) Delete(ctx context.Context, c *Container, force bool) error

func (*Runtime) Kill 

func (rt *Runtime) Kill(ctx context.Context, c *Container, signum unix.Signal) error

func (*Runtime) Load 

func (rt *Runtime) Load(cfg *ContainerConfig) (*Container, error)

func (*Runtime) Start 

func (rt *Runtime) Start(ctx context.Context, c *Container) error

type RuntimeFeatures added in v0.9.8 

type RuntimeFeatures struct {
	Seccomp       bool
	Capabilities  bool
	Apparmor      bool
	CgroupDevices bool
}

RuntimeFeatures are (security) features supported by the Runtime. The supported features are enabled on any Container instance created by Runtime.Create.

type RuntimeHook added in v0.9.8 

type RuntimeHook func(ctx context.Context, c *Container) error

Source Files

View all Source files

Directories

Path Synopsis
cmd
lxcri

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.