gobenchdata

README

gobenchdata

a tool for inspecting go test -bench data, and a GitHub Action for continuous benchmarking.

• About
• GitHub Action
  • Setup
  • Configuration
    • inputs
    • env
  • Visualisation
• gobenchdata CLI
• Development and Contributions

About

gobenchdata was inspired by the deno.land continuous benchmarks, which aims to display performance improvements and regressions on a continuous basis.

It is available as a GitHub action or a command-line application.

GitHub Action

gobenchdata can be used as GitHub Action for uploading Go benchmark data as JSON to gh-pages and visualizing it with a generated web app (using gobenchdata-web) or your own web application.

Setup

For example, in .github/workflows/push.yml, using the new YAML syntax for workflows:

name: Benchmark
on:
  push:
    branches: [ master ]

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
    - name: checkout
      uses: actions/checkout@v1
      with:
        fetch-depth: 1
    - name: gobenchdata to gh-pages
      uses: bobheadxi/gobenchdata@v0.3.0
      with:
        PRUNE_COUNT: 30
        GO_TEST_FLAGS: -cpu 1,2
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Learn more about GitHub Actions in the official documentation.

Configuration

inputs

Input variables are configured using jobs.<job_id>.steps.with.

Variable	Default	Purpose
SUBDIRECTORY	.	subdirectory of project to run commands from
GO_BENCHMARKS	.	benchmarks to run (argument for -bench)
GO_TEST_FLAGS		additional flags for go test
GO_TEST_PKGS	./...	packages to test (argument for go test)
BENCHMARKS_OUT	benchmarks.json	destination path of benchmark data
PRUNE_COUNT	0	number of past runs to keep (0 keeps everything)
GIT_COMMIT_MESSAGE	"add new benchmark run"	the commit message for the benchmark update

env

Environment variables are configured using jobs.<job_id>.steps.env.

Variable	Recommended	Purpose
GITHUB_TOKEN	${{ secrets.GITHUB_TOKEN }}	token to provide access to repository
GITHUB_ACTOR	set by GitHub	the user to make commits as

Visualisation

The gobenchdata GitHub action eventually generates a JSON file with past benchmarks. You can visualize these continuous benchmarks by creating a web app that reads from the JSON benchmarks file, or by using gobenchdata-web:

go get -u go.bobheadxi.dev/gobenchdata/x/gobenchdata-web
git checkout gh-pages
gobenchdata-web --title "my benchmarks" # generates a web app in your working directory

The generator offers a variety of customization options documented under gobenchdata-web help to configure descriptions, charts, and more. The easiest way to use gobenchdata-web is to set up a Makefile in your gh-pages branch to update the web app using the latest version of gobenchdata-web with your desired configuration - for example:

all: build
	git commit -a -m "regenerate web app"

build:
	gobenchdata-web --title "gobenchdata continuous benchmark demo" --desc "This is a demo for gobenchdata"

You can test the web app locally using a tool like serve:

serve .

The web application generator is a work in progress. An example site published by this repository is available at gobenchdata.bobheaxi.dev (configuration.

Other examples:

• bobheadxi/zapx
• benchx.temporal.cloud by @RTradeLtd

gobenchdata CLI

gobenchdata, which the GitHub Action leverages to manage benchmark data, is also available as a CLI:

go get -u go.bobheadxi.dev/gobenchdata
gobenchdata help

The easiest way to use the CLI is by piping the output of go test -bench to it:

go test -bench . -benchmem ./... | gobenchdata --json bench.json

More detailed usage documentation and examples can be found in the godocs or by running gobenchdata help.

Development and Contributions

Please report bugs and requests in the repository issues!

See CONTRIBUTING.md for more detailed development documentation.

Documentation

Overview

Gobenchdata is a tool for inspecting golang benchmark outputs. To install it, you must have Go installed:

go get -u go.bobheadxi.dev/gobenchdata
gobenchdata help

Then pipe your benchmark into the tool:

go test -bench . -benchmem ./... | gobenchdata --json bench.json

You can create a sort of database of benchmarks by appending new benchmarks to an existing file:

go test -benchtime 10000x -bench . -benchmem ./... | gobenchdata --json benchmarks.json --append

You can also merge results:

gobenchdata merge file1.json file2.json file3.json