kbuild

README

kernelbuild-buildkit

Build Linux kernels with stock docker build. No local compiler, source checkout, or custom client is required.

kernelbuild-buildkit is a custom BuildKit LLB frontend. It reads a small Kernelfile, resolves the source and toolchain, and runs kbuild over a persistent object tree. An identical build is a full cache hit. A config change re-runs one vertex, where kbuild recompiles only the affected objects.

Status: pre-1.0. The Kernelfile format and Go API may change between minor releases until v1.0.

Requirements

• Docker with BuildKit enabled. The frontend path is tested with Docker 29.7.2 and BuildKit 0.32.2 on native Linux/amd64 and Docker Desktop/arm64.
• Network access to GHCR, kernel.org, and the base image's package mirrors.
• Several gigabytes of free Docker storage. Allow about 3 GB per persisted kernel object tree, plus image layers and exported artifacts.

The default target is x86_64. Building arm64 kernels requires a ready cross-toolchain image; the worker itself may be amd64 or arm64. Go is needed only for the client and development workflows.

Build your first kernel

1. Create a Kernelfile:

   #syntax=ghcr.io/emirb/kernelbuild-buildkit
   KERNEL   6.18.20
   CONFIG   kernel.config
   SHA256   a1415e257075c2fadf070f44bbb029469efbde5b6cf07d1433fe72207acff03c
   EPOCH    1785542400
   TARGETS  vmlinux image config
   

2. Create a file named kernel.config in the same directory as the Kernelfile; the CONFIG line above names it. A fragment is enough because the build runs olddefconfig, so this one line is a valid config:

   CONFIG_OVERLAY_FS=y
   

   The directory now holds exactly two files:

   .
   ├── Kernelfile
   └── kernel.config
   

3. Build from that directory:

   docker build -f Kernelfile --output type=local,dest=out .
   

4. Verify the result:

   file out/vmlinux out/bzImage
   grep '^CONFIG_OVERLAY_FS=y$' out/config
   

out/vmlinux is the uncompressed ELF, out/bzImage is the x86 boot image, and out/config is the resolved config. Repeating the build should report every vertex as cached.

The complete example is in examples/. The bare frontend reference tracks latest; use a release tag or digest in a committed Kernelfile.

Guest kernels

The artifacts boot directly in Firecracker and Cloud Hypervisor. Firecracker normally takes vmlinux; Cloud Hypervisor accepts a PVH-enabled vmlinux or bzImage. Both use the arm64 PE Image for arm64 guests.

Start from a VMM-maintained guest config rather than kbuild defaults:

curl -fsSLo kernel.config \
  https://raw.githubusercontent.com/firecracker-microvm/firecracker/main/resources/guest_configs/microvm-kernel-ci-x86_64-6.1.config
docker build -f Kernelfile --output type=local,dest=out .

Append local overrides below the downloaded config; later lines win through olddefconfig. Cloud Hypervisor needs CONFIG_PVH=y. The minimal tested boot floor is testdata/boot.config.

An empty config builds but does not make a useful microVM guest: initramfs and serial-console support are disabled by default, so it can boot to silence.

How it works

Kernelfile + config + patches
              │
              ▼
      BuildKit LLB frontend
              │
              ▼
  pinned base → kbuild-step → artifacts
                    ↕
           persistent object tree

The frontend resolves a tagged base image to a digest before generating the graph. The compile vertex starts a static kbuild-step binary as plain argv. Fetching, extraction, patching, config validation, seed transfer, and artifact packing are implemented in Go; make is its only child process.

Values from the Kernelfile are validated before they reach the graph. Extraction is confined with os.Root, and the frontend does not interpolate input into shell commands. Kernel Makefiles can still execute shells, so a config or patch must be treated as code. See SECURITY.md for the trust boundary and deployment guidance.

Kernelfile essentials

Lines are KEY VALUE. Whitespace separates fields, # starts a comment, and key order does not matter. Unknown and duplicate keys are errors.

The #syntax= directive is required for docker build. Individual keys are optional because the frontend supplies defaults. The selected config file is still required in the build context; omitting CONFIG selects kernel.config.

Key	Required?	Value / default
KERNEL	No	kernel.org version; 6.18.20
SOURCE_URL	No	derived from KERNEL; supports .tar.gz, .tar.xz, and .tar.zst
SHA256	No	pinned for the default source; cleared when the source changes
EPOCH	No	SOURCE_DATE_EPOCH; 1785542400
CONFIG	No	config filename in the context; kernel.config
EXPECT	No	post-olddefconfig assertions; disabled
BASE_MAKE	No	in-tree config targets applied before CONFIG; none
TARGETS	No	arch default; accepts vmlinux, image, modules, config, kconfigs
ARCH	No	x86_64
CROSS_COMPILE	No	derived from ARCH
TOOLCHAIN	No	apt; use ready for a preinstalled toolchain
PATCHES	No	off
BASE	No	digest-pinned Ubuntu 24.04 image
PROXY_CA	No	CA certificate filename in the context; none

image exports bzImage on x86_64 and Image on arm64. modules exports a stripped modules.tar.zst. config and kconfigs do not compile a kernel.

See Kernelfile reference for target behavior, expectations, base configs, source pinning, Docker flags, and frontend image verification.

Caching

The build has three cache layers:

1. BuildKit vertex cache. An identical build is a full hit. The cache can be exported to a registry or S3-compatible store for fresh workers.
2. Persistent object tree. A config change re-enters the compile vertex but reuses kbuild's dependency state and compiled objects.
3. Remote object-tree seed. BuildKit cache exporters do not include cache mounts. A trusted seeder can publish the object tree to S3-compatible storage so a cold worker can hydrate it before compiling.

The persisted tree is keyed by kernel version, architecture, toolchain image, cross prefix, and patched state. A source or patch-content mismatch discards the tree and rebuilds it. Seed publication is forced without replacing the warm cache mount.

No ccache or sccache is involved. See Operations and design for cache identity, remote seeding, concurrency, garbage collection, proxies, and toolchains.

Measured

Linux 6.18.20, 4 vCPU worker, 26 August 2026:

Scenario	Wall	Objects compiled
Cold build, including vertex-cache export	329s	1740
Identical rebuild	1s	0
One config option changed, warm worker	16s	8
Fresh worker, same config, S3 vertex hit	2s	0
Fresh worker, changed config, 283 MB seed hydrate	49s	0–19
Seed publication	+19s	0

On a 16 vCPU / 32 GB worker, the same suite measured 84s cold, 0.4s for an identical request, and 11s for a one-option change. Client-side graph generation and marshaling measured 22µs per solve on Apple M5.

Every release is gated on the same path a user takes: the exact image about to be published builds a kernel from testdata/boot.config with stock docker build on a cold 4 vCPU runner and boots it in QEMU. For v0.1.0 that was 80s for the full build and 1.5s to reach userspace. That config is minimal, so it is not comparable to the table above.

These numbers are workload and worker dependent. The important invariants are zero compilation on a full hit, a small object delta for a local config change, and zero or near-zero compilation after seed hydration.

Reproducibility and security

SHA256 pins the source bytes. EPOCH fixes the build timestamp. A committed frontend digest and a TOOLCHAIN ready image pinned by digest fix the remaining build inputs.

The default apt mode favors a one-command first build. Its Ubuntu base image is pinned, but the archive can publish newer compiler and binutils packages, so it is only best-effort reproducible across time. Use a complete, digest-pinned toolchain image for byte reproducibility.

The published frontend image is multi-architecture, signed with keyless cosign, and carries SLSA provenance and an SPDX SBOM. Verification commands are in the Kernelfile reference.

Limits

• Builds sharing one kernel, architecture, toolchain, and patch state serialize on a locked object-tree mount.
• BuildKit garbage collection can evict the tree. Size the daemon for roughly 3 GB per active tree or configure a remote seed.
• arm64 kernels are cross-compiled in linux/amd64 build steps. On an arm64 worker, those steps run under emulation.
• One invocation produces one target architecture, not a multi-platform result.
• The frontend applies in-tree BASE_MAKE targets and one config fragment. It does not implement arbitrary fragment merging or policy.

Documentation

• Kernelfile reference
• Client and Go API
• Operations and design
• Development, testing, and releases
• Security model

Support

Use GitHub issues for bugs and usage questions. Report vulnerabilities privately as described in SECURITY.md.

License

MIT

Documentation

Overview

Package kbuild builds Linux kernels through BuildKit. A Spec describes the build; KernelLLB turns it into an LLB graph; GatewaySolve runs that graph as a gateway frontend (the #syntax= path) or through the client driver (Build). ParseKernelfile reads the Kernelfile format that docker build delegates to the frontend image.

Index

• Constants
• func GatewaySolve(ctx context.Context, c gwclient.Client, spec Spec, opts GatewayOpts) (*gwclient.Result, error)
• func KernelLLB(s Spec) (llb.State, error)
• func ParseCacheEntry(spec, ak, sk string) (client.CacheOptionsEntry, error)
• func ParseKernelfile(r io.Reader, spec *Spec) error
• func Prune(ctx context.Context, addr string) error
• func S3CacheURL(bucketURL, region string) (string, error)
• func SourceExt(name string) (string, error)
• func SourceURLFor(version string) string
• func Timestamp(epoch string) (string, error)
• func VertexLabel(name string) string
• type BuildConfig
• type BuildResult
  • func Build(ctx context.Context, spec Spec, cfg BuildConfig) (*BuildResult, error)
• type GatewayOpts
• type Spec
  • func DefaultSpec() Spec
  • func (s Spec) SeedCfg() []byte
  • func (s Spec) Validate() error
• type VertexTiming

Constants

const HelperPath = "/helper/kbuild-step"

HelperPath is where the compile exec finds the kbuild-step runner: the helper state (frontend image or client-provided dir) is mounted at /helper.

Functions

func GatewaySolve

func GatewaySolve(ctx context.Context, c gwclient.Client, spec Spec, opts GatewayOpts) (*gwclient.Result, error)

GatewaySolve is the one solve path shared by the gateway frontend and the client driver (kbuild.Build runs it inside client.Build). It does what the graph alone cannot: pin the base image by digest through the daemon's resolver (the object tree and the seed are keyed by the Base string, so a moving tag must never reach KernelLLB), confirm that the build context has the files the graph mounts (BuildKit's own failure for a missing selector path is an opaque checksum error), then generate and solve the graph.

func KernelLLB

func KernelLLB(s Spec) (llb.State, error)

KernelLLB builds the LLB graph and returns the captured /out state holding the selected target artifacts (default: vmlinux). This IS the frontend — the build graph is generated in Go, not parsed from a Dockerfile.

Caching has three layers:

1. Coarse, automatic, content-addressed (BuildKit vertex cache): the toolchain vertex and the compile vertex are cache-keyed by content, the source identity (pinned sha256) among the inputs — an identical build is an instant full hit. Exportable to S3/R2 or a registry via the client's cache export options, so a fresh worker gets the full hit too.

2. Fine, object-level incremental (persistent cache mount): the kernel tree lives in a locked per-version cache mount at /build. A config change re-runs the compile vertex, but kbuild's own dependency tracking recompiles only the objects the changed CONFIG symbols touch. No ccache.

3. Remote object-tree seed (ours): BuildKit's cache exporters do NOT cover cache-mount contents, so layer 2 alone dies with the worker. When a seed is configured (via the seed_cfg/seed_access_key/seed_secret_key secrets), a cold mount hydrates from S3-compatible storage before compiling, and a CI build can push the tree back after compiling.

The compile vertex runs the kbuild-step Go binary directly (argv, no shell): stamp/self-heal, seed transfer, fetch, extraction, and patching are Go; the only program it executes is `make`. Cache-key hygiene: everything that affects the output (source identity, config, patches, epoch, CA file) is env or graph structure — in the key. Everything that doesn't (proxy via llb.WithProxy, seed destination + credentials via secrets) — out of the key.

func ParseCacheEntry

func ParseCacheEntry(spec, ak, sk string) (client.CacheOptionsEntry, error)

ParseCacheEntry parses buildctl's cache syntax ("type=registry,ref=...", "type=s3,bucket=...") into a CacheOptionsEntry — the standard BuildKit remote-cache surface, backend-agnostic. For type=s3 entries without explicit credentials, the given creds are injected (the daemon makes the S3 calls and has no env of its own).

func ParseKernelfile

func ParseKernelfile(r io.Reader, spec *Spec) error

ParseKernelfile reads the tiny build-description format that makes `docker build -f Kernelfile` work via the #syntax= directive:

#syntax=ghcr.io/emirb/kernelbuild-buildkit
KERNEL   6.18.20
CONFIG   kernel.config
SHA256   837a5abd...
EPOCH    1785542400
PATCHES  on
PROXY_CA ca-bundle.crt

Lines are KEY VALUE; # starts a comment (whole-line or trailing, after whitespace); unknown keys are an error (a typo must not silently build something else), and so is a key given twice (the second value would win silently, and a stale line left above a new one is the same kind of typo). Values land in the Spec, which is still validated afterwards — this parser adds no trust.

func Prune

func Prune(ctx context.Context, addr string) error

Prune wipes ALL local buildkitd state — vertex cache and cache mounts. The integration suite uses it to simulate a fresh worker.

func S3CacheURL

func S3CacheURL(bucketURL, region string) (string, error)

S3CacheURL expands an https://host/bucket URL into a full s3 cache entry spec in buildctl syntax. region is the bucket's region; empty means "auto", which region-less stores (R2, MinIO) accept and real AWS S3 rejects for SigV4 — so it is a parameter, not a constant.

func SourceExt

func SourceExt(name string) (string, error)

SourceExt returns the tarball extension (".gz", ".xz", ".zst") for a source URL or filename, or an error for an unsupported one.

func SourceURLFor

func SourceURLFor(version string) string

SourceURLFor returns the kernel.org tarball URL for a given version. The directory is keyed by MAJOR version (v6.x, v7.x, ...), derived from the version string — a hardcoded v6.x would silently 404 (or worse, fetch the wrong tree) for 7.x kernels.

The default is .tar.gz, not .tar.xz: gzip decodes in pure Go (klauspost) faster than C xz, so the whole extract path needs no external codec. Both tarballs compress the same tar, so the extracted tree — and the built vmlinux — are byte-identical either way. A .tar.xz SOURCE_URL decodes via ulikunitz/xz (also pure Go); .tar.zst is supported for self-hosted mirrors.

func Timestamp

func Timestamp(epoch string) (string, error)

Timestamp renders SOURCE_DATE_EPOCH exactly as

date -u -d @EPOCH '+%a %b %e %T %Z %Y'

does ("Sat Aug 1 00:00:00 UTC 2026" — %e is space-padded, hence _2). vmlinux embeds this string, so byte-reproducibility depends on it.

func VertexLabel

func VertexLabel(name string) string

VertexLabel compresses a vertex name (which for exec ops is the whole embedded script) to a single readable line.

Types

type BuildConfig

type BuildConfig struct {
	Addr       string // buildkitd address
	ContextDir string // kernel.config (+ patches/, + CA file)
	HelperBin  string // path to the kbuild-step binary ("": next to the executable)
	SrcDir     string // local-source mode: dir with the tarball
	OutDir     string // artifact destination; "" solves WITHOUT exporting (bench: isolates solve+cache from the artifact copy)
	// Remote cache, buildctl syntax ("type=registry,ref=..." / "type=s3,...").
	CacheExports []string
	CacheImports []string
	// AWS-style credentials for the seed secrets and s3 cache entries.
	AccessKey, SecretKey string

	// Progress, when set, receives the LIVE build log stream (vertex output,
	// ">> ..." notes, "KBF-PHASE <name> <ms>ms" markers) as it arrives —
	// what a service streams to its user mid-build. The full transcript is
	// still collected into BuildResult.Logs either way. Writes happen from
	// the solve's status goroutine; the writer must be safe for that.
	Progress io.Writer
	// OnStatus, when set, is called with every raw BuildKit status packet
	// (vertex state changes, log chunks, transfer progress) — the structured
	// feed for consumers that want more than a byte stream. Same goroutine
	// caveat as Progress.
	OnStatus func(*client.SolveStatus)
	// TracerProvider, when set, propagates OpenTelemetry trace context into
	// buildkitd — the solve joins the caller's trace and the daemon's
	// per-vertex spans (cache probe, exec, export) hang off it.
	TracerProvider trace.TracerProvider
}

BuildConfig is everything Build needs beyond the Spec: where the daemon is, which local directories feed the graph, where the artifacts land, and the remote-cache wiring. It is the programmatic form of kbuildctl's flags, and what the integration suite drives directly.

type BuildResult

type BuildResult struct {
	Wall     time.Duration
	Vertices []VertexTiming
	CC       int // "  CC  ..." lines observed — objects compiled
	Logs     string
}

BuildResult reports what the solve did, precisely enough for tests to assert on: wall time, per-vertex timings, and the kbuild activity counted from the captured build logs (no log files, no grep).

func Build

func Build(ctx context.Context, spec Spec, cfg BuildConfig) (*BuildResult, error)

Build solves the Spec's graph against a buildkitd and exports vmlinux (and friends) to cfg.OutDir.

type GatewayOpts

type GatewayOpts struct {
	ResolveMode  string                       // image-resolve-mode ("pull" for docker build --pull)
	CacheImports []gwclient.CacheOptionsEntry // --cache-from / cache-imports
}

GatewayOpts carries the per-invocation knobs the gateway frontend forwards from docker build; the client driver leaves them zero.

type Spec

type Spec struct {
	Base            string // base image, e.g. "docker.io/library/ubuntu:24.04"
	KernelVersion   string // e.g. "6.18.20"
	SourceURL       string // https tarball (used unless SourceLocalName is set)
	SourceSHA256    string // hex sha256 of the tarball; verified in-step and keys the compile vertex
	SourceLocalName string // if set, take the tarball from llb.Local("src")/<name>
	SourceDateEpoch string // reproducible-build epoch (SOURCE_DATE_EPOCH)
	ConfigName      string // config file inside the build context (llb.Local("context"))
	// ExpectName, when set, names a file in the context with post-olddefconfig
	// expectations the step validates BEFORE compiling (see kbuild-step's
	// validateExpectations for the line grammar: "y CONFIG_X", "n CONFIG_X",
	// "= CONFIG_X=val"). olddefconfig silently drops unknown symbols and unmet
	// dependencies; a service composing configs wants that surfaced in seconds,
	// not after a full compile.
	ExpectName string
	// BaseMake, when set, is a space-separated list of make config targets
	// (e.g. "x86_64_defconfig kvm_guest.config") run against the tree FIRST;
	// the context config is then appended as a fragment before olddefconfig.
	// Empty keeps the plain behavior: the context config IS the whole input.
	BaseMake       string
	ApplyPatches   bool   // apply patches/*.patch from the context before building
	HTTPSProxy     string // https_proxy for network steps; cache-neutral (llb.WithProxy)
	HTTPProxy      string // http_proxy for network steps (apt's stock mirrors are http); cache-neutral
	NoProxy        string // extra no_proxy entries (comma list) on top of localhost,127.0.0.1; cache-neutral
	ToolchainReady bool   // Base already has the kernel toolchain (e.g. tuxmake/*): skip the apt vertex
	Arch           string // target architecture ("x86_64" default, "arm64")
	CrossCompile   string // cross prefix override (derived from Arch when empty)
	// Targets selects the artifacts exported to /out. Tokens: "vmlinux",
	// "image" (bzImage on x86_64, Image on arm64), "modules"
	// (modules.tar.zst via modules_install), "config" (the post-olddefconfig
	// .config), "kconfigs" (the tree's Kconfig files bundled as
	// kconfig.txt.gz — symbol-catalog input, no compilation). Empty means the
	// arch default (vmlinux; arm64 also Image).
	Targets     []string
	ProxyCAFile string // CA cert FILE IN THE CONTEXT to trust (MITM-proxy sandbox); "" = none
	HelperRef   string // image ref carrying /kbuild-step; "" = llb.Local("helper") from the client
	NetworkHost bool   // run build steps with host networking (docker build --network=host)
	IgnoreCache bool   // force the build vertices to execute (docker build --no-cache)

	// Remote object-tree seed. BuildKit's cache exporters (registry/s3) cover
	// the vertex/layer cache but NOT the contents of cache mounts, so on a
	// fresh worker the /build mount is empty and a config change would compile
	// cold. The seed closes that gap: after a build the object tree can be
	// pushed to S3-compatible storage (R2), and a cold mount hydrates from it
	// before compiling. Credentials come in as BuildKit secrets (ids "seed_access_key"
	// and "seed_secret_key"), never as opts or env.
	SeedURL    string // bucket base URL on ANY S3-compatible store (AWS, MinIO, R2, Ceph, ...)
	SeedRegion string // bucket region; "auto" (default) suits region-less stores
	SeedPrefix string // key prefix inside the bucket (default "kbuild-seed")
	SeedPush   bool   // push the object tree after a successful build (CI role)
}

Spec describes a kernel build. The frontend turns it into an LLB graph (see KernelLLB). It is deliberately small and JSON/opt-friendly so the same struct drives both the gateway frontend (build.kernel.v0) and the client driver (kbuildctl).

Several fields are interpolated into the build script or into BuildKit identifiers; Validate() MUST pass before the Spec reaches KernelLLB.

func DefaultSpec

func DefaultSpec() Spec

DefaultSpec is the out-of-the-box build: Linux 6.18.20 from kernel.org with a pinned sha256, a fixed epoch, and kernel.config from the context.

func (Spec) SeedCfg

func (s Spec) SeedCfg() []byte

SeedCfg renders the cache-neutral seed configuration that the client passes as the "seed_cfg" BuildKit secret. It lives in a secret, not env or opts, deliberately: whether (and where) the object tree is seeded has no effect on the built vmlinux, so it must not perturb the vertex cache key — and secrets are excluded from cache keys while env vars are not. Returns nil when seeding is disabled.

func (Spec) Validate

func (s Spec) Validate() error

Validate rejects anything that could smuggle shell metacharacters or path traversal into the generated script, a mount identifier, or an object key. Everything KernelLLB interpolates is checked here.

type VertexTiming

type VertexTiming struct {
	Name     string
	Cached   bool
	Duration time.Duration
}

VertexTiming is one solved vertex's observed schedule.