hercules

README

Hercules

Fast, insightful and highly customizable Git history analysis.

Overview • How To Use • Installation • Contributions • License

---

Table of Contents

• Overview
• Installation
  • Build from source
  • Build tags and optional dependencies
  • GitHub Action
  • Release and version policy
• Contributions
• License
• Usage
  • Presets
  • Caching
  • GitHub Action
  • Docker image
  • Built-in analyses
    • Project burndown
    • Files
    • People
    • Churn matrix
    • Code ownership
    • Couples
    • Structural hotness
    • Aligned commit series
    • Added vs changed lines through time
    • Efforts through time
    • Sentiment (positive and negative comments)
    • Bus factor
    • Ownership concentration
    • Onboarding ramp
    • Everything in a single pass
  • Plugins
  • Merging
  • Plotting
  • Custom plotting backend
  • Caveats
  • Burndown Out-Of-Memory

Overview

Hercules is a fast and highly customizable Git repository analysis engine written in Go. It builds a dependency-aware DAG of analyses and processes the full commit history in a single run. Powered by go-git.

This fork focuses on:

• tree-sitter-based structural analyses (no legacy parser service path),
• lightweight default builds (no TensorFlow dependency),
• practical report generation and operational tooling.

There are two command-line tools, both written in Go and built from this repository: hercules and labours. The first takes a Git repository and executes a Directed Acyclic Graph (DAG) of analysis tasks over the full commit history. The second renders predefined plots over the collected data (a native drop-in replacement for the retired Python labours package). The two tools can be chained through a pipe, or run as a single step with hercules report, which renders all charts in-process — no Python required. It is possible to write custom analyses using the plugin system. It is also possible to merge several analysis results together - relevant for organizations. The analyzed commit history includes branches, merges, etc. — non-linear histories are a supported, tested path (the pipeline forks and merges analysis state across branches; see the merge-tracking tests in internal/core).

Historical context from the original project is available in blog post 1, blog post 2, and a presentation. Please contribute by testing, fixing bugs, and adding new analyses.

The DAG of burndown and couples analyses. Generated with hercules --burndown --burndown-people --couples --dry-run --dump-dag docs/dag.dot https://github.com/cwbudde/hercules

torvalds/linux line burndown (granularity 30, sampling 30, resampled by year). Generated with hercules --burndown --first-parent --pb https://github.com/torvalds/linux | labours -f pb -m burndown-project in 1h 40min.

Installation

Grab the hercules binary from the Releases page, or build both hercules and labours from source (see below). No Python installation is needed: labours is a statically linked Go binary and hercules report renders charts in-process.

Build from source

You need Go 1.26.5 or newer. The exact minimum is declared in go.mod. For development workflows that regenerate protobuf files or use repo recipes, install protoc and just.

git clone https://github.com/cwbudde/hercules && cd hercules
# builds both binaries (hercules and labours) plus generated assets
CGO_ENABLED=0 just
# or build them directly
CGO_ENABLED=0 go build ./cmd/hercules
CGO_ENABLED=0 go build ./cmd/labours

These exact commands run in CI. CGO_ENABLED=0 is intentional even on machines where Go enables cgo by default: the renderer dependency's native FreeType implementation is selected by cgo and requires a separately prepared native library, while the supported renderer embeds its fonts and is pure Go.

Build tags and optional dependencies

Default build:

CGO_ENABLED=0 go build ./cmd/hercules

• No external parser service dependency.
• No TensorFlow dependency.
• --shotness and --typos-dataset use tree-sitter by default.
• Tree-sitter is the only structural parsing backend.
• The supported default build is fully cgo-free. CGO_ENABLED=0 go build ./cmd/hercules produces a statically linked binary and cross-compiles cleanly to linux/{amd64,arm64}, windows/amd64, and darwin/arm64.
  • LZ4 compression (used for hibernation) is pure-Go via github.com/cwbudde/lz4. The legacy cgo LZ4 path is retained as opt-in via -tags cgo_lz4 (only the internal/rbtree package is affected).
  • Tree-sitter parsing (used by --shotness, --typos-dataset, and the diff refinement pass) runs on the pure-Go runtime github.com/odvcencio/gotreesitter.

Optional TensorFlow build:

CGO_ENABLED=1 TAGS="tensorflow purego" just hercules

• Enables --sentiment (experimental).
• Requires libtensorflow.
• If --sentiment is requested without this tag, Hercules prints a clear rebuild hint.

Optional cgo LZ4 build:

CGO_ENABLED=1 go build -tags "cgo_lz4 purego" ./cmd/hercules

• Restores the legacy cgo-backed LZ4 path for RBTree hibernation.
• Not required for normal releases; the default pure-Go LZ4 path is preferred.

Migration notes (fork-specific)

This fork intentionally removed the legacy UAST/Babelfish surface and does not preserve backward compatibility for it.

• Removed CLI/feature surface:
  • --feature uast
  • --shotness-xpath-*
• Removed internal pipeline items:
  • UAST
  • UASTChanges
  • FileDiffRefiner (UAST-based)
• Replacement mode:
  • --dump-uast-changes is available again as a tree-sitter-based replacement.
  • It writes .src and .ast.json artifacts under --changed-uast-dir.
  • Legacy protobuf messages for UAST dump payloads are still removed in this fork.
• Protobuf schema change:
  • UASTChange and UASTChangesSaverResults were removed from internal/pb/pb.proto.
  • Older payloads containing these messages are not supported by this fork.
• Final decisions:
  • Protobuf UAST* messages were removed, not renamed.
  • --shotness-xpath-* compatibility flags were removed, not kept as ignored aliases.

GitHub Action

It is possible to run Hercules as a GitHub Action: Hercules on GitHub Marketplace. Please refer to the sample workflow which demonstrates how to setup.

Release and version policy

Default release artifacts are built without TensorFlow, without legacy parser services, and without cgo-only dependencies. The supported release build is:

CGO_ENABLED=0 go build ./cmd/hercules
CGO_ENABLED=0 go build ./cmd/labours

hercules version prints the API version derived from the Go module path plus the Git hash embedded at build time. Builds made through just set the Git hash automatically with -ldflags.

See docs/RELEASE.md for the maintainer checklist, version policy, optional build tags, and migration notes from the old upstream.

Contributions

...are welcome! See CONTRIBUTING and code of conduct.

License

Apache 2.0

Usage

The most useful and reliably up-to-date command line reference:

hercules --help
labours --help

The CLI outcome contract, the analysis schema and metric definitions, and the renderer output manifest describe stable user-visible behavior which is not practical to fit in --help.

Some examples (the local equivalent of the first workflow is exercised by the CLI end-to-end test in CI):

# Use "memory" go-git backend and display the burndown plot. "memory" is the fastest but the repository's git data must fit into RAM.
hercules --burndown https://github.com/go-git/go-git | labours -m burndown-project --resample month
# Use "file system" go-git backend and print some basic information about the repository.
hercules /path/to/cloned/go-git
# Use the file-system go-git backend, create a Hercules-managed clone cache at /tmp/repo-cache,
# use Protocol Buffers, and display the burndown plot without resampling.
hercules --burndown --pb https://github.com/git/git /tmp/repo-cache | labours -m burndown-project -f pb --resample raw

# Now something fun
# Get the linear history from git rev-list, reverse it
# Pipe to hercules, produce burndown snapshots for every 30 days grouped by 30 days
# Save the raw data to cache.yaml, so that later is possible to labours -i cache.yaml
# Pipe the raw data to labours, set text font size to 16pt, use Agg matplotlib backend and save the plot to output.png
git rev-list HEAD | tac | hercules --commits - --burndown https://github.com/git/git | tee cache.yaml | labours -m burndown-project --font-size 16 --backend Agg --output git.png

labours -i /path/to/yaml allows to read the output from hercules which was saved on disk.

Presets

--preset <name> applies a curated set of flag defaults so you can get a useful first result without tuning. Explicit flags on the command line always override preset values, so a preset is a starting point you can fine-tune incrementally.

Two presets ship today:

• --preset quick — fastest path to a result. Sets --head, so only the most recent commit is analysed. Use this to validate that a repo is parseable, sanity-check labours rendering, or get a first burndown for a small repo:

  hercules --preset quick --burndown /path/to/repo | labours -m burndown-project
  

• --preset large-repo — for repositories where a default run would otherwise OOM or take hours. Enables first-parent traversal (--first-parent), configures RBTree hibernation with a 200 000-allocation threshold and disk spill (--lines-hibernation-threshold=200000 --lines-hibernation-disk), and reduces output resolution to 30-day buckets (--granularity=30 --sampling=30). Add a positive --hibernation-distance to schedule hibernation:

  hercules --preset large-repo --hibernation-distance=10 --burndown /path/to/big/repo > burndown.yml
  labours -i burndown.yml -m burndown-project
  

  Hibernation periodically compresses RBTree allocators and (with --lines-hibernation-disk) spills them to a temporary file, trading some CPU for a much lower memory ceiling. See docs/HIBERNATION.md for the underlying mechanism.

If you find yourself overriding the same preset flag every run, file an issue — the preset defaults are tunable and we want them to match what users actually need.

Caching

It is possible to store the cloned repository on disk. The subsequent analysis can run on the corresponding directory instead of cloning from scratch:

# First time - cache
hercules https://github.com/git/git /tmp/repo-cache

# Second time - use the cache
hercules --commits-stat /tmp/repo-cache

Hercules clones into a temporary sibling and moves the completed clone into place, so a failed clone does not leave a partial cache. It refuses to overwrite an existing non-empty path by default. To intentionally refresh a cache previously created and marked by Hercules, pass --force-cache-replace:

hercules --force-cache-replace https://github.com/git/git /tmp/repo-cache

The force flag never permits replacing an unrelated directory, a symbolic link, a filesystem root, the current working directory, or the user's home directory. On a platform or filesystem without atomic directory exchange, replacement fails and the existing cache remains untouched.

Exit status, warnings, and failures

Both CLIs return 0 only when command parsing, input, analysis, rendering, and output publication complete without a hard error. Invalid arguments, malformed or unsupported input, analysis failures, unknown or unimplemented renderer modes, and output write failures return nonzero and write a diagnostic to stderr.

Labours treats only a requested mode whose analysis was not collected as a warning. It prints the missing-analysis warning to stderr, continues other requested modes, and still returns 0 if there are no hard failures. hercules report records the same warnings in manifest.json and index.html; warnings do not make the command fail. In non-strict report mode, hard rendering failures are recorded and the completed report is published before the command returns nonzero. With --strict, the first hard rendering failure aborts publication and leaves an existing report untouched. A broken output pipe is always a hard failure.

GitHub Action

The action produces the artifact named hercules_charts. Since it is currently impossible to pack several files in one artifact, all the charts are packed in the inner tar archive.

Docker image

The runtime image is cgo-free, contains both binaries and CA roots, and runs as a non-root user. Its builder image is pinned by multi-architecture digest. Docker Buildx can build the two supported Linux architectures as one image index:

SOURCE_DATE_EPOCH=0 docker buildx build --platform linux/amd64,linux/arm64 \
  --output type=oci,dest=hercules-multiarch.tar .

CI executes that command and verifies both target builds. To publish an image index, replace the --output option with your registry tag and --push.

For a local image on the current architecture:

SOURCE_DATE_EPOCH=0 docker build -t hercules .
docker run --rm hercules hercules --preset quick --burndown --pb https://github.com/git/git | \
  docker run --rm -i -v "$(pwd):/io" hercules labours -f pb -m burndown-project -o /io/git_git.png

The old srcd/hercules Docker image belongs to the upstream project and is not the release artifact for this fork.

Built-in analyses

Project burndown

hercules --burndown
labours -m burndown-project

Line burndown statistics for the whole repository. Exactly the same what git-of-theseus does but much faster. Blaming is performed efficiently and incrementally using a custom RB tree tracking algorithm, and only the last modification date is recorded while running the analysis.

All burndown analyses depend on the values of granularity and sampling. Granularity is the number of days each band in the stack consists of. Sampling is the frequency with which the burnout state is snapshotted. The smaller the value, the more smooth is the plot but the more work is done.

There is an option to resample the bands inside labours, so that you can define a very precise distribution and visualize it different ways. Besides, resampling aligns the bands across periodic boundaries, e.g. months or years. Unresampled bands are apparently not aligned and start from the project's birth date.

Files

hercules --burndown --burndown-files
labours -m burndown-file

Burndown statistics for every file in the repository which is alive in the latest revision.

Note: it will generate separate graph for every file. You don't want to run it on repository with many files.

People

hercules --burndown --burndown-people [--people-dict=/path/to/identities]
labours -m burndown-person

Burndown statistics for the repository's contributors. If --people-dict is not specified, the identities are discovered by the following algorithm:

1. We start from the root commit towards the HEAD. Emails and names are converted to lower case.
2. If we process an unknown email and name, record them as a new developer.
3. If we process a known email but unknown name, match to the developer with the matching email, and add the unknown name to the list of that developer's names.
4. If we process an unknown email but known name, match to the developer with the matching name, and add the unknown email to the list of that developer's emails.

If --people-dict is specified, it should point to a text file with the custom identities. The format is: every line is a single developer, it contains all the matching emails and names separated by |. The case is ignored. Example file contents:

Linus Torvalds|torvalds@linux-foundation.org
Vadim Markovtsev|vadim@sourced.tech|another@one.com

If --people-dict is not specified a .mailmap file will be used if it exists in the latest commit.

Rendered people-based charts use only each identity's canonical name, so their labels do not expose the email aliases stored in analysis reports. Raw JSON output retains the complete identity strings. --people-anonymity remains available when names must also be hidden.

burndown-person is a fan-out mode: labours -m burndown-person -o chart.png writes one chart_<canonical-name>-<identity-hash>.png sibling per contributor instead of chart.png. The names are safe filename slugs and do not contain email aliases. Each chart omits age bands the contributor never occupied and limits its timeline to that contributor's activity. For the common single-chart view of every developer's living code through time, use labours -m ownership -o ownership.png.

Identity discovery can be audited before running people-sensitive analyses:

hercules --identity-audit /path/to/repo > identities.json

The JSON report lists detected identities, automatic merge decisions with confidence values, and ambiguous candidates which should be reviewed manually. The automatic heuristic threshold defaults to 0.92 and can be adjusted:

hercules --identity-audit --identity-merge-threshold=0.98 /path/to/repo > identities.json

To start a manual identity refinement workflow, generate a template in the same format accepted by --people-dict:

hercules --people-dict-template=people.txt /path/to/repo
hercules --burndown --burndown-people --people-dict=people.txt /path/to/repo

Overwrites matrix

Wireshark top 20 devs - overwrites matrix

hercules --burndown --burndown-people [--people-dict=/path/to/identities]
labours -m overwrites-matrix

Beside the burndown information, --burndown-people collects the added and deleted line statistics per developer. Thus it can be visualized how many lines written by developer A are removed by developer B. This indicates collaboration between people and defines expertise teams.

The format is the matrix with N rows and (N+2) columns, where N is the number of developers.

1. First column is the number of lines the developer wrote.
2. Second column is how many lines were written by the developer and deleted by unidentified developers (if --people-dict is not specified, it is always 0).
3. The rest of the columns show how many lines were written by the developer and deleted by identified developers.

The sequence of developers is stored in people_sequence YAML node.

Code ownership

Ember.js top 20 devs - code ownership

hercules --burndown --burndown-people [--people-dict=/path/to/identities]
labours -m ownership

--burndown-people also allows to draw the code share through time stacked area plot. That is, how many lines are alive at the sampled moments in time for each identified developer.

Couples

torvalds/linux files' coupling in Tensorflow Projector

hercules --couples [--people-dict=/path/to/identities]
labours -m couples -o <output-directory> [--tmpdir=/tmp]

The files are coupled if they are changed in the same commit. The developers are coupled if they change the same file. hercules records the number of couples throughout the whole commit history and outputs the two corresponding co-occurrence matrices. labours -m couples renders the file-coupling, people-coupling and (when collected) shotness-coupling charts from those matrices. The shotness chart compares entity co-occurrence profiles: cell (i, j) is the dot product of the counters for entities i and j. Its diagonal is each profile's squared norm; ranked pairs exclude the diagonal and include only positive, distinct pairs. YAML and Protocol Buffer inputs use the same matrix construction. The TensorFlow-based Swivel embedding training and Tensorflow Projector TSV export were features of the retired Python labours package and are not part of the Go renderer.

Structural hotness

      46  jinja2/compiler.py:visit_Template [FunctionDef]
      42  jinja2/compiler.py:visit_For [FunctionDef]
      34  jinja2/compiler.py:visit_Output [FunctionDef]
      29  jinja2/environment.py:compile [FunctionDef]
      27  jinja2/compiler.py:visit_Include [FunctionDef]
      22  jinja2/compiler.py:visit_Macro [FunctionDef]
      22  jinja2/compiler.py:visit_FromImport [FunctionDef]
      21  jinja2/compiler.py:visit_Filter [FunctionDef]
      21  jinja2/runtime.py:__call__ [FunctionDef]
      20  jinja2/compiler.py:visit_Block [FunctionDef]

By default, --shotness is powered by tree-sitter and tracks function-level units for Go, Python, JavaScript and TypeScript. Structural identities combine source path, node kind, and qualified name (including receiver or enclosing scope when available). Coordinate-only moves within a file preserve an entity; qualified renames and individual cross-file moves create a new identity. An explicit whole-file rename migrates the existing identities and their counters to the new path.

hercules --shotness
labours -m shotness

Couples analysis automatically loads "shotness" data if available.

hercules --shotness --pb https://github.com/pallets/jinja | labours -m couples -f pb

Aligned commit series

tensorflow/tensorflow aligned commit series of top 50 developers by commit number.

hercules --devs [--people-dict=/path/to/identities]
labours -m devs -o <name>

We record how many commits made, as well as lines added, removed and changed per day for each developer. We plot the resulting commit time series using a few tricks to show the temporal grouping. In other words, two adjacent commit series should look similar after normalization.

1. We compute the distance matrix of the commit series. Our distance metric is Dynamic Time Warping. We use FastDTW algorithm which has linear complexity proportional to the length of time series. Thus the overall complexity of computing the matrix is quadratic.
2. We compile the linear list of commit series with Seriation technique. Particularly, we solve the Travelling Salesman Problem which is NP-complete. However, given the typical number of developers which is less than 1,000, there is a good chance that the solution does not take much time. We use Google or-tools solver.
3. We find 1-dimensional clusters in the resulting path with HDBSCAN algorithm and assign colors accordingly.
4. Time series are smoothed by convolving with the Slepian window.

This plot allows to discover how the development team evolved through time. It also shows "commit flashmobs" such as Hacktoberfest. For example, here are the revealed insights from the tensorflow/tensorflow plot above:

1. "Tensorflow Gardener" is classified as the only outlier.
2. The "blue" group of developers covers the global maintainers and a few people who left (at the top).
3. The "red" group shows how core developers join the project or become less active.

Added vs changed lines through time

tensorflow/tensorflow added and changed lines through time.

hercules --devs [--people-dict=/path/to/identities]
labours -m old-vs-new -o <name>

--devs from the previous section allows to plot how many lines were added and how many existing changed (deleted or replaced) through time. This plot is smoothed.

Efforts through time

kubernetes/kubernetes efforts through time.

hercules --devs [--people-dict=/path/to/identities]
labours -m devs-efforts -o <name>

Besides, --devs allows to plot how many lines have been changed (added, removed, or modified) by each developer. The plot stacks each developer's cumulative effort over time. Invalid negative daily totals in legacy input are treated as zero, so cumulative effort and the chart's Y axis remain non-negative. There is a difference between the efforts plot and the ownership plot, although changing lines correlate with owning lines.

Sentiment (positive and negative comments)

It can be clearly seen that Django comments were positive/optimistic in the beginning, but later became negative/pessimistic.
hercules --sentiment --pb https://github.com/django/django | labours -m sentiment -f pb

--sentiment is experimental and optional. It is unavailable in default release builds.

We extract new and changed comments from source code on every commit, apply BiDiSentiment general purpose sentiment recurrent neural network and plot the results. This analysis requires libtensorflow and a tensorflow build tag. E.g. sadly, we need to hide the rect from the documentation finder for now is negative and Theano has a built-in optimization for logsumexp (...) so we can just write the expression directly is positive. Don't expect too much though - as was written, the sentiment model is general purpose and the code comments have different nature, so there is no magic (for now).

Hercules must be built with the tensorflow tag - it is not enabled by default:

CGO_ENABLED=1 TAGS="tensorflow purego" just hercules

Such a build requires libtensorflow. If --sentiment is requested in a non-tensorflow build, Hercules exits with a rebuild hint.

Bus factor

hercules --bus-factor [--bus-factor-threshold=0.8] [--people-dict=/path/to/identities]
labours -m bus-factor

The bus factor is the minimum number of developers whose departure would leave the project without sufficient knowledge to maintain it. Hercules computes this over time by finding the smallest set of developers who collectively own at least 80% (configurable via --bus-factor-threshold) of the living code lines. Each occupied tick records ownership after that tick's last commit and before any later tick. Required coverage is rounded up to a whole line, so the configured threshold remains exact for small repositories.

The analysis produces three visualizations:

1. Timeline - a step chart showing how the bus factor evolves over the project's lifetime, with a danger zone highlight at BF=1.
2. Gauge - a summary of the current bus factor value color-coded by risk level (critical/low/moderate/healthy), alongside a pie chart of top code owners.
3. Subsystems - a horizontal bar chart breaking down bus factor by top-level directory, making it easy to spot which parts of the codebase are most at risk.

Ownership concentration

hercules --ownership-concentration [--people-dict=/path/to/identities]
labours -m ownership-concentration

The Gini coefficient and Herfindahl-Hirschman Index quantify how concentrated or distributed code ownership is. Gini=0 means perfectly equal ownership, Gini=1 means one person owns everything. HHI ranges from 1/n (equal) to 1.0 (single author). Both metrics are tracked over time using the same line ownership data as the bus factor analysis. Their occupied-tick and final subsystem snapshots use the same incremental ownership accounting as Bus Factor.

The analysis produces two visualizations:

1. Timeline - a dual-axis step chart showing Gini and HHI evolving over the project's lifetime, with reference lines for moderate and high concentration.
2. Subsystems - a grouped horizontal bar chart comparing Gini and HHI by top-level directory.

Onboarding ramp

hercules --onboarding --pb [--onboarding-windows=7,30,90] [--onboarding-meaningful-threshold=10]
labours -f pb -m onboarding

The onboarding analysis tracks how quickly new contributors ramp up after their first commit. It groups authors by monthly join cohort and measures commits, files, and changed lines at configurable day windows. Cohorts use the local calendar month encoded in each author's first commit timestamp. Windows are exact elapsed 24-hour periods from that timestamp, include commits on the boundary, and never include a later commit merely because it shares a tick.

Render it with labours -f pb -m onboarding (or let hercules report do it — the mode is in the default set). Like every analysis added since the YAML reader was frozen, the onboarding renderer reads protobuf only, so pass --pb to hercules and -f pb to labours. It writes three sibling charts:

1. Cohort heatmap (_rampup) - average meaningful lines by join cohort and days since first commit, reproducing the retired Python renderer's chart (shown below).
2. Time to first meaningful commit (_time-to-first) - distribution across authors, with an explicit bucket for those who made none inside the largest window.
3. Author ramps (_authors) - per-author meaningful-line trajectories for the top contributors in the view.

The analysis is mergeable, so these charts also work on the output of hercules combine. Because window snapshots are anchored at each author's first commit, the merge re-anchors every author onto their earliest commit across all repositories rather than summing per-repository snapshots.

Everything in a single pass

hercules report -o ./report https://github.com/go-git/go-git

This command runs Hercules in Protocol Buffers mode, renders the charts with the built-in in-process renderer (no separate labours process and no Python needed), and transactionally publishes a report directory with generated plots, report.pb, index.html, and a manifest.json inventory. Each run starts in a fresh staging directory, so rerunning with fewer modes removes charts from the previous run. In --strict mode, a rendering failure leaves any previously published report intact. The default report includes the usual project, file, people, ownership, temporal activity, bus factor, knowledge diffusion, hotspot risk, and refactoring proxy views without requiring separate labours flags. An external drop-in renderer can be substituted with --labours-cmd.

Every analysis enabled by the default report links to its precise metric and wire contract:

Default analysis flag	Definition
--burndown, --burndown-files, --burndown-people	Line-age cohorts and snapshot semantics
--couples	File and developer co-occurrence
--devs	Per-developer commit and line-change statistics
--temporal-activity	Author-time activity dimensions
--bus-factor	Exact ownership threshold and snapshot semantics
--ownership-concentration	Gini/HHI ownership snapshots
--knowledge-diffusion	Lifetime and recent file editors
--onboarding	First-commit cohorts and duration windows
--hotspot-risk	Weighted normalized risk factors
--refactoring-proxy	Per-tick rename ratio

The exact files emitted by each mode, including fan-out and companion assets, are listed in the renderer output manifest.

From a source checkout, the same report can be generated with:

just report https://github.com/go-git/go-git

Use --all to request every report mode, including optional or heavier views:

hercules report --all -o ./report https://github.com/go-git/go-git

To customize the report scope, pass explicit analysis flags and modes:

hercules report --analysis burndown --analysis devs --mode burndown-project --mode devs -o ./report <repo>

Manual pipeline chaining is still supported:

hercules --burndown --burndown-files --burndown-people --couples --shotness --devs [--people-dict=/path/to/identities]
labours -m all

Plugins

Hercules has a plugin system and allows to run custom analyses. See PLUGINS.md.

Status: plugins are supported with an important caveat — Go plugins require cgo, while the default build of hercules (just, releases, Docker) is CGO_ENABLED=0 and therefore cannot load any plugin. To use --plugin, build both the plugin and hercules from the same source tree with CGO_ENABLED=1. The compatibility of this path is verified by just test-plugin (test/plugin_smoke/).

Merging

hercules combine joins several analysis results in Protocol Buffers format. Readers and combine enforce the schema-version compatibility matrix; the schema changelog records wire-compatible and semantic compatibility changes which may require regenerating stored results.

hercules --burndown --pb https://github.com/go-git/go-git > go-git.pb
hercules --burndown --pb https://github.com/cwbudde/hercules > hercules.pb
hercules combine go-git.pb hercules.pb | labours -f pb -m burndown-project --resample M

Plotting

These options affects all plots:

labours [--style=white|black] [--size=Y,X] [--theme=default|dark|minimal|vibrant|matplotlib]

--style sets the general style of the plot (see labours --help). --background changes the plot background to be either white or black. --size sets the size of the figure in inches. The default is 12,9. --backend is accepted for compatibility with the retired Python labours CLI.

The Go renderer aims for visual parity with the original matplotlib output; the tracked parity matrix and how to re-run it are documented in docs/RENDER_PARITY.md.

--relative is effective in burndown charts only:

labours [--font-size N] [--relative] [--no-burndown-title]

--font-size changes label and legend size for all charts; --relative activates the stretched burndown layout; --no-burndown-title suppresses the title on burndown and ownership charts without cropping or post-processing the rendered image.

Custom plotting backend

It is possible to output all the information needed to draw the plots in JSON format. Simply append .json to the output (-o) and you are done. The data format is not fully specified and depends on the renderer code which generates it. Each JSON file should contain "type" which reflects the plot kind.

Caveats

1. Processing all the commits may fail in some rare cases. If you get an error similar to https://github.com/cwbudde/hercules/issues/106 please report there and specify --first-parent as a workaround.
2. Burndown collection may fail with an Out-Of-Memory error. See the next session for the workarounds.
3. --sentiment and couples embeddings are optional/experimental TensorFlow-backed paths. They are not enabled in default release builds and should not be treated as release-blocking analyses.
4. Parsing huge YAML outputs is slow and memory-hungry (e.g. the Linux kernel in "couples" mode produces a 1.5 GB YAML document). Most repositories are parsed within a minute, but for big ones prefer Protocol Buffers (hercules --pb and labours -f pb).

Burndown Out-Of-Memory

If the analyzed repository is big and extensively uses branching, the burndown stats collection may fail with an OOM. You should try the following:

1. Read the repo from disk instead of cloning into memory.
2. Use --skip-blacklist to avoid analyzing unwanted files. It is also possible to constrain --languages.
3. Use the hibernation feature: --hibernation-distance 10 --lines-hibernation-threshold=200000. Tune the values to start hibernating before the OOM.
4. Hibernate both line history and burndown state on disk: --lines-hibernation-disk --burndown-hibernation-disk; their optional directories are configured with --lines-hibernation-dir and --burndown-hibernation-dir.
5. --first-parent, you win.

Documentation

Overview

Package hercules contains the functions which are needed to gather various statistics from a Git repository.

The analysis is expressed in a form of the tree: there are nodes - "pipeline items" - which require some other nodes to be executed prior to selves and in turn provide the data for dependent nodes. There are several service items which do not produce any useful statistics but rather provide the requirements for other items. The top-level items include:

- BurndownAnalysis - line burndown statistics for project, files and developers.

- CouplesAnalysis - coupling statistics for files and developers.

- ShotnessAnalysis - structural hotness and couples, powered by tree-sitter in the default build.

The typical API usage is to initialize the Pipeline class:

import "github.com/go-git/go-git/v5"

var repository *git.Repository
// ...initialize repository...
pipeline := hercules.NewPipeline(repository)

Then add the required analysis:

ba := pipeline.DeployItem(&hercules.BurndownAnalysis{}).(hercules.LeafPipelineItem)

This call will add all the needed intermediate pipeline items. Then link and execute the analysis tree:

pipeline.Initialize(nil)
result, err := pipeline.Run(pipeline.Commits(false))

Finally extract the result:

result := result[ba].(hercules.BurndownResult)

The actual usage example is cmd/hercules/root.go - the command line tool's code.

You can provide additional options via `facts` on initialization. For example, to provide your own logger, enable people-tracking, and set a custom tick size:

pipe.Initialize(map[string]interface{}{
  hercules.ConfigLogger:            zap.NewExample().Sugar(),
  hercules.ConfigTickSize:          12,
  leaves.ConfigBurndownTrackPeople: true,
})

Hercules depends heavily on https://github.com/go-git/go-git and leverages the diff algorithm through https://github.com/sergi/go-diff.

Besides, BurndownAnalysis involves File and RBTree. These are low level data structures which enable incremental blaming. File carries an instance of RBTree and the current line burndown state. RBTree implements the red-black balanced binary tree and is based on https://github.com/yasushi-saito/rbtree.

Coupling stats are supposed to be further processed rather than observed directly. labours.py uses Swivel embeddings and visualises them in Tensorflow Projector.

Structural analyses and comment extraction are powered by tree-sitter in the default build.

Index

• Constants
• Variables
• func EnablePathFlagTypeMasquerade()
• func FactValue[T any](facts map[string]any, key string) (T, bool, error)
• func LoadCommitsFromFile(path string, repository *git.Repository) ([]*object.Commit, error)
• func NewLogger() core.Logger
• func PathifyFlagValue(flag *pflag.Flag)
• func QualifyRepositoryPath(repository, path string) string
• func RequiredFactValue[T any](facts map[string]any, key string) (T, error)
• func SafeYamlString(str string) string
• type CachedBlob
• type CommitComponent
• type CommonAnalysisResult
  • func MetadataToCommonAnalysisResult(meta *core.Metadata) *CommonAnalysisResult
• type ConfigurationOption
• type ConfigurationOptionType
• type DisconnectedCommitsError
• type DisposablePipelineItem
• type DuplicateCommitError
• type FactTypeError
• type FeaturedPipelineItem
• type FileDiffData
• type FileIdResolver
• type FlagConfiguration
• type HibernateablePipelineItem
• type IdentityResolver
• type LeafPipelineItem
• type Logger
• type NoopMerger
• type OneShotMergeProcessor
• type Pipeline
  • func NewPipeline(repository *git.Repository) *Pipeline
• type PipelineItem
  • func ForkCopyPipelineItem(origin PipelineItem, n int) []PipelineItem
  • func ForkSamePipelineItem(origin PipelineItem, n int) []PipelineItem
• type PipelineItemRegistry
• type RepositoryQualifiablePipelineItem
• type ResultMergeablePipelineItem

Constants

const (
	// BoolConfigurationOption reflects the boolean value type.
	BoolConfigurationOption = core.BoolConfigurationOption
	// IntConfigurationOption reflects the integer value type.
	IntConfigurationOption = core.IntConfigurationOption
	// StringConfigurationOption reflects the string value type.
	StringConfigurationOption = core.StringConfigurationOption
	// FloatConfigurationOption reflects a floating point value type.
	FloatConfigurationOption = core.FloatConfigurationOption
	// StringsConfigurationOption reflects the array of strings value type.
	StringsConfigurationOption = core.StringsConfigurationOption
	// PathConfigurationOption reflects a filesystem path value type.
	PathConfigurationOption = core.PathConfigurationOption
	// MessageFinalize is the status text reported before calling LeafPipelineItem.Finalize()-s.
	MessageFinalize = core.MessageFinalize
)

const (
	// ConfigPipelineDAGPath is the name of the Pipeline configuration option (Pipeline.Initialize())
	// which enables saving the items DAG to the specified file.
	ConfigPipelineDAGPath = core.ConfigPipelineDAGPath
	// ConfigPipelineDumpPlan is the name of the Pipeline configuration option (Pipeline.Initialize())
	// which outputs the execution plan to stderr.
	ConfigPipelineDumpPlan = core.ConfigPipelineDumpPlan
	// ConfigPipelineDryRun is the name of the Pipeline configuration option (Pipeline.Initialize())
	// which disables Configure() and Initialize() invocation on each PipelineItem during the
	// Pipeline initialization.
	// Subsequent Run() calls are going to fail. Useful with ConfigPipelineDAGPath=true.
	ConfigPipelineDryRun = core.ConfigPipelineDryRun
	// ConfigPipelineCommits is the name of the Pipeline configuration option (Pipeline.Initialize())
	// which allows to specify the custom commit sequence. By default, Pipeline.Commits() is used.
	ConfigPipelineCommits = core.ConfigPipelineCommits
	// ConfigTickSize is the number of hours per 'tick'.
	ConfigTickSize = plumbing.ConfigTicksSinceStartTickSize
	// ConfigLogger is used to set the logger in all pipeline items.
	ConfigLogger = core.ConfigLogger
)

const (
	// DependencyCommit is the name of one of the three items in `deps` supplied to PipelineItem.Consume()
	// which always exists. It corresponds to the currently analyzed commit.
	DependencyCommit = core.DependencyCommit
	// DependencyIndex is the name of one of the three items in `deps` supplied to PipelineItem.Consume()
	// which always exists. It corresponds to the currently analyzed commit's index.
	DependencyIndex = core.DependencyIndex
	// DependencyIsMerge is the name of one of the three items in `deps` supplied to PipelineItem.Consume()
	// which always exists. It indicates whether the analyzed commit is a merge commit.
	// Checking the number of parents is not correct - we remove the back edges during the DAG simplification.
	DependencyIsMerge = core.DependencyIsMerge
	// DependencyAuthor is the name of the dependency provided by identity.PeopleDetector.
	DependencyAuthor = identity.DependencyAuthor
	// DependencyBlobCache identifies the dependency provided by BlobCache.
	DependencyBlobCache = plumbing.DependencyBlobCache
	// DependencyTick is the name of the dependency which TicksSinceStart provides - the number
	// of ticks since the first commit in the analysed sequence.
	DependencyTick = plumbing.DependencyTick
	// DependencyFileDiff is the name of the dependency provided by FileDiff.
	DependencyFileDiff = plumbing.DependencyFileDiff
	// DependencyTreeChanges is the name of the dependency provided by TreeDiff.
	DependencyTreeChanges = plumbing.DependencyTreeChanges
	// FactCommitsByTick contains the mapping between tick indices and the corresponding commits.
	FactCommitsByTick = plumbing.FactCommitsByTick
	// FactIdentityDetectorReversedPeopleDict is the name of the fact which is inserted in
	// identity.PeopleDetector.Configure(). It corresponds to identity.PeopleDetector.ReversedPeopleDict -
	// the mapping from the author indices to the main signature.
	FactIdentityDetectorReversedPeopleDict = identity.FactIdentityDetectorReversedPeopleDict
	// FactIdentityResolver identifies the typed author identity resolver.
	FactIdentityResolver = core.FactIdentityResolver
	// FactLineHistoryResolver identifies the typed file identity resolver.
	FactLineHistoryResolver = core.FactLineHistoryResolver
)

const RepositoryPathSeparator = core.RepositoryPathSeparator

RepositoryPathSeparator separates the repository from the path in a qualified path key.

Variables

var (
	// ErrNoCommits indicates that an explicit commit input or execution plan is empty.
	ErrNoCommits = core.ErrNoCommits
	// ErrNoReferences indicates that a repository does not contain a usable commit reference.
	ErrNoReferences = core.ErrNoReferences
	// ErrInvalidCommit indicates that explicit input contains a nil or zero-hash commit.
	ErrInvalidCommit = core.ErrInvalidCommit
	// ErrDuplicateCommits indicates that explicit input repeats a commit hash.
	ErrDuplicateCommits = core.ErrDuplicateCommits
	// ErrDisconnectedCommits indicates that explicit input has disconnected components.
	ErrDisconnectedCommits = core.ErrDisconnectedCommits
	// ErrInvalidFactType indicates that a configuration or shared fact has an unexpected type.
	ErrInvalidFactType = core.ErrInvalidFactType
	// ErrFactMissing indicates that a required fact is absent.
	ErrFactMissing = core.ErrFactMissing
)

var BinaryGitHash = "<unknown>"

BinaryGitHash is the Git hash of the Hercules binary file which is executing.

var BinaryVersion = detectBinaryVersion()

BinaryVersion is Hercules' API version. It matches the package name.

var Registry = core.Registry

Registry contains all known pipeline item types.

Functions

func EnablePathFlagTypeMasquerade

func EnablePathFlagTypeMasquerade()

EnablePathFlagTypeMasquerade changes the type of all "path" command line arguments from "string" to "path". This operation cannot be canceled and is intended to be used for better --help output.

func FactValue

func FactValue[T any](facts map[string]any, key string) (T, bool, error)

FactValue reads an optional fact with an exact type check.

func LoadCommitsFromFile

func LoadCommitsFromFile(path string, repository *git.Repository) ([]*object.Commit, error)

LoadCommitsFromFile reads the file by the specified FS path and generates the sequence of commits by interpreting each line as a Git commit hash.

func NewLogger

func NewLogger() core.Logger

NewLogger returns an instance of the default Hercules logger.

func PathifyFlagValue

func PathifyFlagValue(flag *pflag.Flag)

PathifyFlagValue changes the type of a string command line argument to "path".

func QualifyRepositoryPath

func QualifyRepositoryPath(repository, path string) string

QualifyRepositoryPath prefixes a repository-local path with the repository which contains it. Implementations of RepositoryQualifiablePipelineItem must build their keys with it rather than concatenating the separator themselves.

func RequiredFactValue

func RequiredFactValue[T any](facts map[string]any, key string) (T, error)

RequiredFactValue reads a required fact with an exact type check.

func SafeYamlString

func SafeYamlString(str string) string

SafeYamlString escapes the string so that it can be reliably used in YAML.

Types

type CachedBlob

type CachedBlob = plumbing.CachedBlob

CachedBlob allows to explicitly cache the binary data associated with the Blob object. Such structs are returned by DependencyBlobCache.

type CommitComponent

type CommitComponent = core.CommitComponent

CommitComponent describes one connected component in explicit commit input.

type CommonAnalysisResult

type CommonAnalysisResult = core.CommonAnalysisResult

CommonAnalysisResult holds the information which is always extracted at Pipeline.Run().

func MetadataToCommonAnalysisResult

func MetadataToCommonAnalysisResult(meta *core.Metadata) *CommonAnalysisResult

MetadataToCommonAnalysisResult copies the data from a Protobuf message.

type ConfigurationOption

type ConfigurationOption = core.ConfigurationOption

ConfigurationOption allows for the unified, retrospective way to setup PipelineItem-s.

type ConfigurationOptionType

type ConfigurationOptionType = core.ConfigurationOptionType

ConfigurationOptionType represents the possible types of a ConfigurationOption's value.

type DisconnectedCommitsError

type DisconnectedCommitsError = core.DisconnectedCommitsError

DisconnectedCommitsError describes disconnected explicit commit input.

type DisposablePipelineItem

type DisposablePipelineItem = core.DisposablePipelineItem

DisposablePipelineItem owns resources which are released after a pipeline run.

type DuplicateCommitError

type DuplicateCommitError = core.DuplicateCommitError

DuplicateCommitError describes a repeated hash in explicit commit input.

type FactTypeError

type FactTypeError = core.FactTypeError

FactTypeError describes a type mismatch in a configuration or shared fact.

type FeaturedPipelineItem

type FeaturedPipelineItem = core.FeaturedPipelineItem

FeaturedPipelineItem enables switching the automatic insertion of pipeline items on or off.

type FileDiffData

type FileDiffData = plumbing.FileDiffData

FileDiffData is the type of the dependency provided by plumbing.FileDiff.

type FileIdResolver

type FileIdResolver = core.FileIdResolver

FileIdResolver provides typed access to configured file identities.

type FlagConfiguration

type FlagConfiguration = core.FlagConfiguration

FlagConfiguration retains typed flag storage until it is snapshotted after parsing.

type HibernateablePipelineItem

type HibernateablePipelineItem = core.HibernateablePipelineItem

HibernateablePipelineItem can compact and restore branch-local run state.

type IdentityResolver

type IdentityResolver = core.IdentityResolver

IdentityResolver provides typed access to configured author identities.

type LeafPipelineItem

type LeafPipelineItem = core.LeafPipelineItem

LeafPipelineItem corresponds to the top level pipeline items which produce the end results.

type Logger

type Logger core.Logger

Logger is the Hercules logging interface.

type NoopMerger

type NoopMerger = core.NoopMerger

NoopMerger provides an empty Merge() method suitable for PipelineItem.

type OneShotMergeProcessor

type OneShotMergeProcessor = core.OneShotMergeProcessor

OneShotMergeProcessor provides the convenience method to consume merges only once.

type Pipeline

type Pipeline = core.Pipeline

Pipeline is the core Hercules entity which carries several PipelineItems and executes them. See the extended example of how a Pipeline works in doc.go.

func NewPipeline

func NewPipeline(repository *git.Repository) *Pipeline

NewPipeline initializes a new instance of Pipeline struct.

type PipelineItem

type PipelineItem = core.PipelineItem

PipelineItem is the interface for all the units in the Git commits analysis pipeline.

func ForkCopyPipelineItem

func ForkCopyPipelineItem(origin PipelineItem, n int) []PipelineItem

ForkCopyPipelineItem clones items by copying them by value from the origin.

func ForkSamePipelineItem

func ForkSamePipelineItem(origin PipelineItem, n int) []PipelineItem

ForkSamePipelineItem clones items by referencing the same origin.

type PipelineItemRegistry

type PipelineItemRegistry = core.PipelineItemRegistry

PipelineItemRegistry contains all the known PipelineItem-s.

type RepositoryQualifiablePipelineItem

type RepositoryQualifiablePipelineItem = core.RepositoryQualifiablePipelineItem

RepositoryQualifiablePipelineItem produces results keyed by repository-local paths.

type ResultMergeablePipelineItem

type ResultMergeablePipelineItem = core.ResultMergeablePipelineItem

ResultMergeablePipelineItem specifies the methods to combine several analysis results together.