fuseoverlayfs

README

fuse-overlayfs snapshotter plugin for containerd

Unlike overlayfs, fuse-overlayfs can be used as a non-root user on almost all recent distros.

You do NOT need this fuse-overlayfs plugin on the following environments, because they support the real overlayfs for non-root users:

• kernel >= 5.11
• Ubuntu kernel, since circa 2015
• Debian 10 kernel
  • Debian 10 needs sudo modprobe overlay permit_mounts_in_userns=1. Future release of Debian with kernel >= 5.11 will not need this modprobe hack.

fuse-overlayfs-snapshotter is a non-core sub-project of containerd.

Requirements

• kernel >= 4.18
• containerd >= 1.4
• fuse-overlayfs >= 0.7.0

Setup

Two installation options are supported:

1. Embed fuse-overlayfs plugin into the containerd binary
2. Execute fuse-overlayfs plugin as a separate binary

Choose 1 if you don't mind recompiling containerd, otherwise choose 2.

Option 1: Embed fuse-overlayfs plugin into the containerd binary

Create builtins_fuseoverlayfs_linux.go under $GOPATH/src/github.com/containerd/containerd/cmd/containerd with the following content, and recompile the containerd binary:

/*
   Copyright The containerd Authors.

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
*/

package main

import _ "github.com/containerd/fuse-overlayfs-snapshotter/plugin"

No extra configuration is needed.

See https://github.com/containerd/containerd/blob/master/docs/rootless.md for how to run containerd as a non-root user.

Option 2: Execute fuse-overlayfs plugin as a separate binary

"Easy way"

The easiest way is to use containerd-rootless-setuptool.sh included in nerdctl.

$ containerd-rootless-setuptool.sh install
$ containerd-rootless-setuptool.sh install-fuse-overlayfs
[INFO] Creating "/home/exampleuser/.config/systemd/user/containerd-fuse-overlayfs.service"
...
[INFO] Installed "containerd-fuse-overlayfs.service" successfully.
[INFO] To control "containerd-fuse-overlayfs.service", run: `systemctl --user (start|stop|restart) containerd-fuse-overlayfs.service`
[INFO] Add the following lines to "/home/exampleuser/.config/containerd/config.toml" manually:
### BEGIN ###
[proxy_plugins]
  [proxy_plugins."fuse-overlayfs"]
    type = "snapshot"
    address = "/run/user/1000/containerd-fuse-overlayfs.sock"
###  END  ###
[INFO] Set `export CONTAINERD_SNAPSHOTTER="fuse-overlayfs"` to use the fuse-overlayfs snapshotter.

Add the [proxy_plugins."fuse-overlayfs"] configuration shown above to ~/.config/containerd/config.toml. "1000" needs to be replaced with your actual UID.

"Hard way"

Click here to show the "hard way"

• Install containerd-fuse-overlayfs-grpc binary. The binary will be installed under $DESTDIR/bin.

$ make && DESTDIR=$HOME make install

• Create the following configuration in ~/.config/containerd/config.toml:

version = 2
# substitute "/home/suda" with your own $HOME
root = "/home/suda/.local/share/containerd"
# substitute "/run/user/1001" with your own $XDG_RUNTIME_DIR
state = "/run/user/1001/containerd"

[grpc]
  address = "/run/user/1001/containerd/containerd.sock"

[proxy_plugins]
  [proxy_plugins."fuse-overlayfs"]
    type = "snapshot"
    address = "/run/user/1001/containerd/fuse-overlayfs.sock"

• Start RootlessKit with sleep infinity (or any kind of "pause" command):

$ rootlesskit \
  --net=slirp4netns --disable-host-loopback \
  --copy-up=/etc --copy-up=/run \
  --state-dir=$XDG_RUNTIME_DIR/rootlesskit-containerd \
  sh -c "rm -rf /run/containerd ; sleep infinity"

(Note: rm -rf /run/containerd is a workaround for containerd/containerd#2767)

• Enter the RootlessKit namespaces and run containerd-fuse-overlayfs-grpc:

$ nsenter -U --preserve-credentials -m -n -t $(cat $XDG_RUNTIME_DIR/rootlesskit-containerd/child_pid) \
  containerd-fuse-overlayfs-grpc $XDG_RUNTIME_DIR/containerd/fuse-overlayfs.sock $HOME/.local/share/containerd-fuse-overlayfs

• Enter the same namespaces and run containerd:

$ nsenter -U --preserve-credentials -m -n -t $(cat $XDG_RUNTIME_DIR/rootlesskit-containerd/child_pid) \
  containerd -c $HOME/.config/containerd/config.toml

Usage

$ export CONTAINERD_SNAPSHOTTER=fuse-overlayfs
$ nerdctl run ...

How to test

To run the test as a non-root user, RootlessKit needs to be installed.

$ go test -exec rootlesskit -test.v -test.root

Project details

fuse-overlayfs-snapshotter is a containerd non-core sub-project, licensed under the Apache 2.0 license. As a containerd non-core sub-project, you will find the:

• Project governance,
• Maintainers,
• and Contributing guidelines

information in our containerd/project repository.

Documentation

Index

• func AsynchronousRemove(config *SnapshotterConfig) error
• func NewSnapshotter(root string, opts ...Opt) (snapshots.Snapshotter, error)
• func Supported(root string) error
• type Opt
• type SnapshotterConfig

Functions

func AsynchronousRemove

func AsynchronousRemove(config *SnapshotterConfig) error

AsynchronousRemove defers removal of filesystem content until the Cleanup method is called. Removals will make the snapshot referred to by the key unavailable and make the key immediately available for re-use.

AsynchronousRemove is untested for fuse-overlayfs

func NewSnapshotter

func NewSnapshotter(root string, opts ...Opt) (snapshots.Snapshotter, error)

NewSnapshotter returns a Snapshotter which uses overlayfs. The overlayfs diffs are stored under the provided root. A metadata file is stored under the root.

func Supported

func Supported(root string) error

Supported returns nil when the overlayfs is functional on the system with the root directory. Supported is not called during plugin initialization, but exposed for downstream projects which uses this snapshotter as a library.

Types

type Opt

type Opt func(config *SnapshotterConfig) error

Opt is an option to configure the overlay snapshotter

type SnapshotterConfig

type SnapshotterConfig struct {
	// contains filtered or unexported fields
}

SnapshotterConfig is used to configure the overlay snapshotter instance