efipack

README

go-coff/efipack

Pure-Go library that compresses PE32+/EFI binaries into a self-extracting PE32+/EFI image — the UPX-equivalent that does not exist anywhere else for this format. Designed to mitigate the EDK2 OVMF amd64 CpuPageTableLib #GP that fires on LoadImage / StartImage of sufficiently large EFI binaries (cloud-boot M6.2 milestone).

Status

The library ships:

• compression API (Pack, Options, PackResult),
• PE32+/EFI envelope assembly (DOS stub + PE signature + COFF + optional header + section table + real per-arch runtime decompressor .stub + .payload body),
• arch detection (amd64, arm64, riscv64, loong64),
• a swappable body compressor — Flate (default), LZFSE, and LZ4, all pure-Go and host-side complete,
• round-trip tests proving the bytes in .payload decompress back to the original input, plus PE-structural tests over the embedded stubs.

The TODO_STUB placeholder is gone: Pack uses a real embedded per-arch self-extracting stub (stub/blobs/<arch>.efi.bin) as the envelope base, so a Flate-packed binary is a genuinely runnable self-extracting EFI. At firmware load time the stub recovers its own on-disk bytes, finds .payload, decompresses, and chain-boots via gBS->LoadImage + gBS->StartImage.

Bootability matrix (Compressor = Flate)

Arch	Runtime stub	Boot status
arm64 (0xaa64)	embedded	OVMF/QEMU-verified — packed EFI decompresses + hands off to the payload
riscv64 (0x5064)	embedded	boots under QEMU+EDK2 smoke; PE-structural in CI
loong64 (0x6264)	embedded	boots under QEMU+EDK2 smoke; PE-structural in CI
amd64 (0x8664)	embedded	not bootable — the stub faults on entry under OVMF with an X64 #UD (Invalid Opcode); a TamaGo-runtime bootstrap defect, tracked for a stub rebuild. Envelope is PE-valid and host-round-trips.

LZFSE and LZ4 are host-side only — see below.

API surface

import "github.com/go-coff/efipack"

// Detect the architecture of an input PE32+ from its COFF header.
arch, err := efipack.InferArch(peBytes)
// arch ∈ {AmdArch, ArmArch, RiscvArch, LoongArch}

// Pack a PE32+ binary into a self-extracting envelope.
in, _ := os.Open("BOOTX64.EFI")
out, _ := os.Create("BOOTX64-packed.EFI")
res, err := efipack.Pack(in, out, efipack.Options{
    Compressor: efipack.Flate, // default; zero stub cost
    Level:      0,             // 0 → codec default
})
// res.OriginalSize, res.CompressedSize, res.PackedSize, res.Arch, res.Compressor

Type / constant	Purpose
Pack(in, out, opts) (PackResult, error)	host-side compress + envelope
Options{Compressor, Level}	knobs; zero value = Flate at default level
PackResult{OriginalSize, CompressedSize, PackedSize, Compressor, Arch}	summary
Compressor (Flate / LZFSE / LZ4)	algorithm switch; all three wired
Arch (AmdArch / ArmArch / RiscvArch / LoongArch)	PE machine
InferArch(pe) (Arch, error)	read COFF.Machine without debug/pe (works on loong64)
ReadPayload(pe) (algo, uncompressedSize, body, err)	inverse of Pack's envelope; used by host tests and by the runtime stub
ErrCompressorNotImplemented	reserved public sentinel; no codec returns it now that all three are wired

.payload wire format

.payload section body:
  magic         [4]byte   "CBP0"   — cloud-boot pack v0
  algo          [4]byte   "FLAT" | "LZFS" | "LZ4 "
  uncompressed  uint64    little-endian — host-input size in bytes
  compressed    uint64    little-endian — body size in bytes
  body          [N]byte   exactly N=compressed bytes; codec-specific stream

The runtime stub reads this header, allocates exactly the right number of EfiBootServicesCode pages, decompresses, then chain-loads via gBS->LoadImage + gBS->StartImage. The shipped stubs dispatch on the FLAT algo tag only (see below).

Why Flate as the default?

The original M6.2 design picked LZFSE on raw ratio (40.13% vs 39.11%). After review we pivoted: cloud-boot binaries already link compress/gzip via M6.1 embeds and M7 OCI manifest handling, so the Flate-based stub adds zero bytes vs LZFSE's ~100-200 KiB cost. The 1-point ratio gap is negligible against the ~5 MiB binaries we are compressing. LZFSE remains pluggable via the Compressor enum for cases where the host has a larger budget than the cloud-boot baseline.

LZFSE & LZ4 — host-side only

Compressor = LZFSE (via go-compressions/lzfse, best ratio) and Compressor = LZ4 (via go-compressions/lz4's pure-Go block format, fastest decompress) both work end-to-end on the host side: Pack produces a structurally valid PE32+ envelope whose .payload decodes back to the original input byte-for-byte, with the matching LZFS / LZ4 algo tag stamped in the CBP0 header.

However, the embedded per-arch runtime decompressor stubs (stub/blobs/<arch>.efi.bin) dispatch on the FLAT tag only — a packed binary produced with LZFSE or LZ4 will NOT boot under firmware because the stub does not decode that tag (verified: an LZ4-packed arm64 EFI loads under OVMF, fails the FLAT check, and exits EFI_ABORTED without handing off). Rebuilding the runtime stubs with an LZFS / LZ4 decode path (and re-embedding them) is gated on a TamaGo toolchain rebuild; meanwhile use Compressor = Flate for runnable packed EFIs.

Dependencies

• github.com/go-coff/peln — PE32+ section-append (used to attach .payload to the envelope).
• github.com/go-compressions/lzfse — pure-Go LZFSE/LZVN codec for the host-side LZFSE compressor.
• github.com/go-compressions/lz4 — pure-Go LZ4 block codec for the host-side LZ4 compressor.

No CGO, no vendoring, stdlib-only otherwise.

License

BSD 3-Clause.

Documentation

Overview

Package efipack compresses PE32+/EFI binaries into a self-extracting PE32+/EFI image. PR1 (this revision) ships the host-side compression API, the PE32+ envelope assembly, and tests; the per-arch runtime decompressor stub lands in PR2 and the pectl CLI integration in PR3.

Index

• Variables
• func ReadPayload(pe []byte) (algo string, uncompressedSize uint64, body []byte, err error)
• type Arch
  • func InferArch(pe []byte) (Arch, error)
  • func (a Arch) String() string
• type Compressor
  • func (c Compressor) String() string
• type Options
• type PackResult
  • func Pack(in io.Reader, out io.Writer, opts Options) (PackResult, error)

Variables

var ErrCompressorNotImplemented = errors.New("efipack: compressor not implemented in this build")

ErrCompressorNotImplemented is the sentinel switchCompressor returns for a valid Compressor constant that has no codec wired in the current build. As of v0.3.0 every defined Compressor (Flate, LZFSE, LZ4) is implemented host-side, so switchCompressor no longer returns it; the sentinel is retained as a stable part of the public API so callers that added a Compressor constant ahead of its codec can keep matching on it with errors.Is.

Functions

func ReadPayload

func ReadPayload(pe []byte) (algo string, uncompressedSize uint64, body []byte, err error)

ReadPayload locates the .payload section in a packed PE32+ image, strips the wire header, and returns the raw compressed body plus the metadata stamped at pack time. It is the inverse of Pack's envelope assembly, used by host tests today and by the runtime stub (PR2) tomorrow. It does NOT decompress — the caller pairs it with a bodyCodec.Decode for the appropriate algorithm.

Types

type Arch

type Arch int

Arch enumerates the four PE32+/EFI target machines efipack ships (or will ship, once PR2 lands the per-arch decompressor stubs).

const (
	// AmdArch is amd64 / x86_64. PE COFF Machine = 0x8664.
	AmdArch Arch = iota
	// ArmArch is arm64 / aarch64. PE COFF Machine = 0xaa64.
	ArmArch
	// RiscvArch is riscv64. PE COFF Machine = 0x5064.
	RiscvArch
	// LoongArch is loongarch64. PE COFF Machine = 0x6264.
	LoongArch
)

func InferArch

func InferArch(pe []byte) (Arch, error)

InferArch reads the PE/COFF header of pe and returns the detected Arch. It does the minimum amount of parsing required to read the Machine field — no debug/pe round-trip, so it accepts machines such as loong64 (0x6264) that debug/pe rejects.

func (Arch) String

func (a Arch) String() string

String returns a short stable name for the architecture. Used in error messages and in the per-arch stub blob name (see bodyCodec.StubBlobName).

type Compressor

type Compressor int

Compressor is the body-compression algorithm used by Pack.

The default is Flate; it costs zero additional bytes in the decompressor stub because cloud-boot binaries already link compress/gzip via M6.1's embedded payloads and M7's OCI manifest handling. Alternatives (LZFSE, LZ4) can be wired in by adding a constant + case in switchCompressor.

const (
	// Flate uses stdlib compress/flate. Default; zero stub cost.
	Flate Compressor = iota
	// LZFSE uses github.com/go-compressions/lzfse. Best raw ratio;
	// host-side wired since v0.2.0. Booting an LZFSE-packed EFI needs
	// an LZFSE-aware runtime stub (the shipped blobs decode FLAT only).
	LZFSE
	// LZ4 uses github.com/go-compressions/lz4's pure-Go block codec.
	// Host-side wired since v0.3.0 — the fastest decompressor of the
	// three, at a lower ratio. Like LZFSE, booting an LZ4-packed EFI
	// needs an LZ4-aware runtime stub (the shipped blobs decode FLAT
	// only); efipack still stamps the LZ4 body + "LZ4 " algo tag so a
	// future LZ4-aware stub — or a host-side unpack — round-trips it.
	LZ4
)

func (Compressor) String

func (c Compressor) String() string

String returns a short stable name for the compressor.

type Options

type Options struct {
	Compressor Compressor // defaults to Flate
	Level      int        // compression level passed to the underlying codec; 0 = codec default
}

Options controls Pack. The zero value is sensible (Flate at the stdlib default level).

type PackResult

type PackResult struct {
	OriginalSize   int64
	CompressedSize int64 // size of the compressed body inside .payload (excludes the 24-byte header)
	PackedSize     int64 // size of the output PE32+ on disk
	Compressor     Compressor
	Arch           Arch
}

PackResult summarises a successful Pack call. Sizes are in bytes.

PR1 leaves StubSize at 0 because the .stub section is the placeholder; PR2 will add a StubSize field carrying the real per-arch blob length.

func Pack

func Pack(in io.Reader, out io.Writer, opts Options) (PackResult, error)

Pack reads a PE32+/EFI image from in, compresses it, and writes a self-extracting PE32+/EFI envelope to out.

PR1 acceptance: the output is structurally a valid PE32+ image (correct DOS stub, PE signature, COFF header with the input arch's Machine field, optional header, section table with `.stub` and `.payload`), but the `.stub` section is a placeholder containing only the sentinel "TODO_STUB". Firmware will load the image, jump to the entry point, and fault — by design. PR2 replaces the .stub body with the real per-arch decompressor blob, after which the output becomes a runnable self-extracting EFI.

Round-trip is guaranteed today: the bytes inside .payload, when fed back through the matching bodyCodec.Decode, reproduce the input byte-for-byte. This is what the host tests exercise.