godata

README

golang.org/x/build/maintner/godata

Package godata loads the Go project's corpus of Git, Github, and Gerrit activity into memory to allow easy analysis without worrying about APIs and their pagination, quotas, and other nuisances and limitations.

Documentation

Overview

Package godata loads the Go project's corpus of Git, Github, and Gerrit activity into memory to allow easy analysis without worrying about APIs and their pagination, quotas, and other nuisances and limitations.

Index

• Constants
• func Dir() string
• func Get(ctx context.Context) (*maintner.Corpus, error)
• func XdgCacheDir() string

Examples

• Get (NumComments)

Constants

const Server = "https://maintner.golang.org/logs"

Server is the Go project's production maintner log.

Functions

func Dir

func Dir() string

Dir returns the directory containing the cached mutation logs.

func Get

func Get(ctx context.Context) (*maintner.Corpus, error)

Get returns the Go project's corpus, containing all Git commits, Github activity, and Gerrit activity and metadata since the beginning of the project.

Use Corpus.Update to keep the corpus up-to-date. If you do this, you must hold the read lock if reading and updating concurrently.

The initial call to Get will download a few gigabytes of data into a directory "golang-maintner" under your operating system's user cache directory. Subsequent calls will only download what's changed since the previous call.

Even with all the data already cached on local disk, a call to Get takes approximately 15 seconds per gigabyte of mutation log data to load it into memory. For daemons, use Corpus.Update to incrementally update an already-loaded Corpus.

The in-memory representation is about 25% larger than its on-disk size. In April 2022, it's under 4 GB.

See https://pkg.go.dev/golang.org/x/build/maintner#Corpus for how to walk the data structure.

Example (NumComments)

package main

import (
	"context"
	"fmt"
	"log"

	"golang.org/x/build/maintner"
	"golang.org/x/build/maintner/godata"
)

func main() {
	corpus, err := godata.Get(context.Background())
	if err != nil {
		log.Fatal(err)
	}
	num := 0
	corpus.GitHub().ForeachRepo(func(gr *maintner.GitHubRepo) error {
		return gr.ForeachIssue(func(gi *maintner.GitHubIssue) error {
			return gi.ForeachComment(func(*maintner.GitHubComment) error {
				num++
				return nil
			})
		})
	})
	fmt.Printf("%d GitHub comments on Go repos.\n", num)
}

func XdgCacheDir

func XdgCacheDir() string

XdgCacheDir returns the XDG Base Directory Specification cache directory.