coverbee

README

CoverBee

Code coverage collection tool for eBPF programs. CoverBee can instrument already compiled eBPF programs by giving it an ELF file. This allows for coverage testing without modifying the existing toolchain.

Installation

go install github.com/cilium/coverbee/cmd/coverbee@latest

Usage CLI

First, instrument and load the programs in a ELF file by using coverbee load. All programs will be pinned in the directory specified by --prog-pin-dir. If --map-pin-dir is specified, all maps with with pinning = LIBBPF_PIN_BY_NAME set will be pinned in the given map. If --map-pin-dir is not specified, a pin location for the cover-map must be specified with --covermap-pin. The block-list will be written as JSON to a location specified by --block-list this file contains translation data and must be passed to coverbee cover afterwards.

Instrument all programs in the given ELF file and load them into the kernel

Usage:
  coverbee load {--elf=ELF path} {--prog-pin-dir=path to dir} {--map-pin-dir=path to dir | --covermap-pin=path to covermap} {--block-list=path to blocklist} [flags]

Flags:
      --block-list string     Path where the block-list is stored (contains coverage data to source code mapping, needed when reading from cover map)
      --covermap-pin string   Path to pin for the covermap (created by coverbee containing coverage information)
      --elf string            Path to the ELF file containing the programs
  -h, --help                  help for load
      --log string            Path for ultra-verbose log output
      --map-pin-dir string    Path to the directory containing map pins
      --prog-pin-dir string   Path the directory where the loaded programs will be pinned
      --prog-type string      Explicitly set the program type

Then attach the programs or test them with BPF_TEST_RUN.

Once done, to inspect the coverage call coverbee cover, pass it the same --map-pin-dir/--covermap-pin and --block-list as was used for coverbee load. Specify a path for the output with --output which is html by default but can also be set to output go-cover for use with other tools by setting --format go-cover

Collect coverage data and output to file

Usage:
  coverbee cover {--map-pin-dir=path to dir | --covermap-pin=path to covermap} {--block-list=path to blocklist} {--output=path to report output} [flags]

Flags:
      --block-list string     Path where the block-list is stored (contains coverage data to source code mapping, needed when reading from cover map)
      --covermap-pin string   Path to pin for the covermap (created by coverbee containing coverage information)
      --format string         Output format (options: html, go-cover) (default "html")
  -h, --help                  help for cover
      --map-pin-dir string    Path to the directory containing map pins
      --output string         Path to the coverage output

Don't forget to clean up the programs by detaching and/or removing the pins.

Usage as library

1. Load the ELF file using cilium/ebpf
2. Perform normal setup(except for loading the programs, maps can be pre-loaded)
3. Call coverbee.InstrumentAndLoadCollection instead of using ebpf.NewCollectionWithOptions
4. Attach the program or run tests
5. Convert the CFG gotten in step 3 to a block-list with coverbee.CFGToBlockList
6. Get the coverbee_covermap from the collection and apply its contents to the block-list with coverbee.ApplyCoverMapToBlockList
7. Convert the block-list into a go-cover or HTML report file with coverbee.BlockListToGoCover or coverbee.BlockListToHTML respectively

How does CoverBee work

CoverBee instruments existing compiled eBPF programs in ELF format and load them into the kernel. This instrumentation will increment numbers in a eBPF map when certain parts of the program are ran. CoverBee uses the kernel verifier logs to find out which registers and stack slots are not used by the program, and uses these for the instrumentation code.

The contents of the cover-map are be mapped back to the source file via the block-list. This block-list is constructed from the control flow graph of the programs and the BTF.ext line information. Then a modified version of go tool cover is used to create HTML reports.

Limitations / Requirements

• CoverBee requires up to 3 stack slots (24 bytes) available on the stack, programs close to the limit might not pass the verifier once instrumented.
• CoverBee adds instructions to the programs, programs close to the instruction or complexity limit of the kernel might not pass the verifier once instrumented.
• CoverBee used BTF.ext information to convert instructions to coverage information, ELF files without BTF will not work
• CoverBee requires the source code of the programs to pre present at the same location as at compile time and to contain the same contents. BTF.ext contains line and column offsets to absolute filepaths, changes in path or file contents between compilation and coverage testing might result in invalid or non-working coverage reports.
• CoverBee will add a map named coverbee_covermap to the collection, so this name can't be used by the program itself.

Documentation

Index

• func ApplyCoverMapToBlockList(coverMap *ebpf.Map, blockList [][]CoverBlock) error
• func BlockListFilePaths(blockList [][]CoverBlock) []string
• func BlockListToGoCover(blockList [][]CoverBlock, out io.Writer, mode string)
• func BlockListToHTML(blockList [][]CoverBlock, out io.Writer, mode string) error
• func CFGToBlockList(cfg []*BasicBlock) [][]CoverBlock
• func HTMLOutput(profiles []*cover.Profile, out io.Writer) error
• func ProfilesToGoCover(profiles []*cover.Profile, out io.Writer, mode string)
• func SourceCodeInterpolation(coverageBlockList [][]CoverBlock, additionalFilePaths []string) ([][]CoverBlock, error)
• type BasicBlock
  • func InstrumentAndLoadCollection(coll *ebpf.CollectionSpec, opts ebpf.CollectionOptions, logWriter io.Writer) (*ebpf.Collection, []*BasicBlock, error)
  • func InstrumentCollection(coll *ebpf.CollectionSpec, logWriter io.Writer) ([]*BasicBlock, error)
  • func ProgramBlocks(prog asm.Instructions) []*BasicBlock
• type CoverBlock
  • func CTranslationUnitToBlocks(tu *cparser.TranslationUnit) ([]*CoverBlock, map[cparser.ASTNode][]*CoverBlock)
  • func (cb CoverBlock) String() string

Functions

func ApplyCoverMapToBlockList

func ApplyCoverMapToBlockList(coverMap *ebpf.Map, blockList [][]CoverBlock) error

ApplyCoverMapToBlockList reads from the coverage map and applies the counts inside the map to the block list. The blocklist can be iterated after this to create a go-cover coverage file.

func BlockListFilePaths

func BlockListFilePaths(blockList [][]CoverBlock) []string

BlockListFilePaths returns a sorted and deduplicateed list of file paths included in the block list

func BlockListToGoCover

func BlockListToGoCover(blockList [][]CoverBlock, out io.Writer, mode string)

BlockListToGoCover convert a block-list into a go-cover file which can be interpreted by `go tool cover`. `mode` value can be `set` or `count`, see `go tool cover -h` for details.

func BlockListToHTML

func BlockListToHTML(blockList [][]CoverBlock, out io.Writer, mode string) error

BlockListToHTML converts a block-list into a HTML coverage report.

func CFGToBlockList

func CFGToBlockList(cfg []*BasicBlock) [][]CoverBlock

CFGToBlockList convert a CFG to a "BlockList", the outer slice indexed by BlockID which maps to an inner slice, each element of which is a reference to a specific block of code inside a source file. Thus the resulting block list can be used to translate blockID's into the pieces of source code to apply coverage mapping.

func HTMLOutput

func HTMLOutput(profiles []*cover.Profile, out io.Writer) error

HTMLOutput generates an HTML page from profile data. coverage report is written to the out writer.

func ProfilesToGoCover

func ProfilesToGoCover(profiles []*cover.Profile, out io.Writer, mode string)

ProfilesToGoCover convert a profile list into a go-cover file which can be interpreted by `go tool cover`. `mode` value can be `set` or `count`, see `go tool cover -h` for details.

func SourceCodeInterpolation

func SourceCodeInterpolation(coverageBlockList [][]CoverBlock, additionalFilePaths []string) ([][]CoverBlock, error)

SourceCodeInterpolation parses all files referenced in the coverage block list and the additional file paths as C code. It then uses the parsed code to construct coverage blocks from the source code instead of the compiled object. This results in a more "complete" negative profile since it also will include lines which the compiler tends to optimize out. It then applies the measured coverage to the source blocks. Lastly it will infer / interpolate which lines of code must also have been evaluated given the AST and the coverage blocklist. The intended goal being a more accurate report.

Types

type BasicBlock

type BasicBlock struct {
	Index int
	// The current block of code
	Block asm.Instructions

	// The next block of we don't branch
	NoBranch *BasicBlock
	// The next block if we do branch
	Branch *BasicBlock
}

BasicBlock is a block of non-branching code, which makes up a node within the CFG.

func InstrumentAndLoadCollection

func InstrumentAndLoadCollection(
	coll *ebpf.CollectionSpec,
	opts ebpf.CollectionOptions,
	logWriter io.Writer,
) (*ebpf.Collection, []*BasicBlock, error)

InstrumentAndLoadCollection will instrument the given collection spec and proceed to load it using the provided options. Please refer to `InstrumentCollection` for more information about the instrumentation process.

func InstrumentCollection

func InstrumentCollection(coll *ebpf.CollectionSpec, logWriter io.Writer) ([]*BasicBlock, error)

InstrumentCollection adds instrumentation instructions to all programs contained within the given collection. This "instrumentation" consists of an additional map with a single key and a value which is an array of 16-bit counters. Each index of the array corresponds to the basic block index. The instrumentation code will increment the counter just before the basic block is executed.

The given spec is modified with this instrumentation. The whole process is logged to the `logWriter` and a list of all the basic blocks are returned and can later be matched to the counters in the map.

Steps of the function:

1. Load the original programs and collect the verbose verifier log
2. Parse the verifier log, which tells us which registers and stack slots are occupied at any given time.
3. Convert the program into a CFG(Control Flow Graph)
4. At the start of each program and bpf-to-bpf function, load the cover-map's index 0 and store the map value in a available slot on the stack.
5. At the start of each block, load an offset into the cover-map value, increment it, write it back. This requires 2 registers which can be clobbered. If only 1 or no registers are unused, store the register values to the stack and restore values afterwards.
6. Move symbols of the original code to the instrumented code so jumps and functions calls first pass by the instrumentation.
7. Load all modified program into the kernel.

func ProgramBlocks

func ProgramBlocks(prog asm.Instructions) []*BasicBlock

ProgramBlocks takes a list of instructions and converts it into a a CFG(Control Flow Graph). Which works as follows:

1. Construct a translation map from RawOffsets to the instructions(since index within the slice doesn't account for LDIMM64 instructions which use two instructions).
2. Apply a label to every jump target and set that label as a reference in the branching instruction. This does two things. First, it makes it easy to find all block boundaries since each block has a function name or jump label. The second is that cilium/ebpf will recalculate the offsets of the jumps based on the symbols when loading, so we can easily add instructions to blocks without fear of breaking offsets.
3. Loop over all instructions, creating a block at each branching instruction or symbol/jump label.
4. Build a translation map from symbol/jump label to block.
5. Loop over all blocks, using the map from step 4 to link blocks together on the branching and non-branching edges.

type CoverBlock

type CoverBlock struct {
	Filename     string
	ProfileBlock cover.ProfileBlock
}

CoverBlock wraps the ProfileBlock, adding a filename so each CoverBlock can be turned into a line on a go coverage file.

func CTranslationUnitToBlocks

func CTranslationUnitToBlocks(tu *cparser.TranslationUnit) ([]*CoverBlock, map[cparser.ASTNode][]*CoverBlock)

CTranslationUnitToBlocks turns a TranslationUnit into a list of cover blocks and a map which can be used to see which AST nodes generated which blocks.

func (CoverBlock) String

func (cb CoverBlock) String() string