flakeguard

README

Flakeguard

Flakeguard helps you detect flaky tests, quarantine them, and guard your CI pipelines from them.

Usage

# Print help output
flakeguard -h

Once you find a flaky test, take a look at our Fixing Flaky Tests Guide for tips and processes to help you debug and narrow down the source of flakes.

There are two modes for running flakeguard, detect and guard.

detect

Run tests over and over to figure out which ones are flaky. Best used in a nightly cron job to regularly checkup on your test suite health.

flakeguard detect -h

guard

Guard your CI pipelines from being affected by flaky tests. This will attempt to re-run any failing tests and make them pass so that your PRs and merge queues aren't decimated by getting unlucky with flakes. It will also look for newly added and modified tests and try to determine if your code changes are introducing new flakes.

flakeguard guard -h

Design

For detailed technical design diagrams and decisions, see the Flakeguard Design Doc. For guiding principles for UX, see the Ideal Flakeguard Developer Experiences page.

Contributing

We use golangci-lint v2 for linting and formatting, and pre-commit for pre-commit and pre-push checks.

pre-commit install # Install our pre-commit scripts

See the Makefile for helpful commands for local development.

make build            # Build binaries, results placed in dist/
make lint             # Lint and format code
make bench            # Run all benchmarks

make test_short       # Run only short tests
make test_unit        # Run only unit tests
make test_integration # Run only integration tests
make test_full        # Run all tests with extensive coverage stats
make test_full_race   # Run all tests with extensive coverage stats and race detection

Cursor

If you use Cursor, you can utilize the Cursor rules included in .cursor/rules to guide suggestions.

Test

• FLAKEGUARD_TEST_LOG_LEVEL Sets the logging level for tests to use, use FLAKEGUARD_TEST_LOG_LEVEL=trace if you're chasing down confusing bugs.
• FLAKEGUARD_GOCOVERDIR Sets the coverage dir for integration tests to utilize. This is handled automatically in most make commands.

A handy debugging setup is available already in the included .vscode folder as "Debug Flakeguard".

Documentation

Overview

Package flakeguard allows you to quarantine tests that have been identified as flaky.

Index

• Constants
• func Quarantine(t *testing.T, quarantineMessage string)

Constants

const RunQuarantinedTestsEnvVar = "FLAKEGUARD_RUN_QUARANTINED_TESTS"

RunQuarantinedTestsEnvVar is the environment variable that controls whether to run quarantined tests. Set to true to run quarantined tests, false to skip them. This is helpful to keep track of tests' flakiness even when they are quarantined.

Functions

func Quarantine

func Quarantine(t *testing.T, quarantineMessage string)

Quarantine a test so that it is skipped during your CI/CD pipelines. You can still make the test run by setting FLAKEGUARD_RUN_QUARANTINED_TESTS to true.