bt_tracestore

Documentation

Overview

Package bt_tracestore implements a tracestore backed by BigTable See BIGTABLE.md for an overview of the schema and design.

Index

• Constants
• func InitBT(ctx context.Context, conf BTConfig) error
• type BTConfig
• type BTTraceStore
  • func New(ctx context.Context, conf BTConfig, cache bool) (*BTTraceStore, error)
  • func (b *BTTraceStore) DEBUG_getTracesInRange(ctx context.Context, startTileKey, endTileKey TileKey, ...) (traceMap, paramtools.ParamSet, error)
  • func (b *BTTraceStore) GetDenseTile(ctx context.Context, nCommits int) (*tiling.Tile, []tiling.Commit, error)
  • func (b *BTTraceStore) GetTile(ctx context.Context, nCommits int) (*tiling.Tile, []tiling.Commit, error)
  • func (b *BTTraceStore) Put(ctx context.Context, commitHash string, entries []*tracestore.Entry, ...) error
• type TileKey
  • func GetTileKey(repoIndex int) (TileKey, int)
  • func (t TileKey) OpsRowName() string

Constants

const (

	// This is the size of the tile in Big Table. That is, how many commits do we store in one tile.
	// We can have up to 2^32 tiles in big table, so this would let us store 1 trillion
	// commits worth of data. This tile size does not need to be related to the tile size that
	// Gold operates on (although when tuning, it should be greater than, or an even divisor
	// of the Gold tile size). The first commit in the repo belongs to tile 2^32-1 and tile numbers
	// decrease for newer commits. The columnPad const also depends on the number of digits of
	// DefaultTileSize.
	DefaultTileSize = 256

	// Default number of shards used. A shard splits the traces up on a tile.
	// If a trace exists on shard N in tile A, it will be on shard N for all tiles.
	// Having traces on shards lets BT split up the work more evenly.
	DefaultShards = 32
)

Constants adapted from btts.go See BIGTABLE.md for an overview of how the data is stored in BT.

Functions

func InitBT

func InitBT(ctx context.Context, conf BTConfig) error

InitBT initializes the BT instance for the given configuration. It uses the default way to get auth information from the environment and must be called with an account that has admin rights.

Types

type BTConfig

type BTConfig struct {
	ProjectID  string
	InstanceID string
	TableID    string
	VCS        vcsinfo.VCS
}

BTConfig contains the configuration information for the BigTable-based implementation of TraceStore.

type BTTraceStore

type BTTraceStore struct {
	// contains filtered or unexported fields
}

BTTraceStore implements the TraceStore interface.

func New

func New(ctx context.Context, conf BTConfig, cache bool) (*BTTraceStore, error)

New implements the TraceStore interface backed by BigTable. If cache is true, the OrderedParamSets will be cached based on the row name.

func (*BTTraceStore) DEBUG_getTracesInRange

func (b *BTTraceStore) DEBUG_getTracesInRange(ctx context.Context, startTileKey, endTileKey TileKey, startCommitIndex, endCommitIndex int) (traceMap, paramtools.ParamSet, error)

DEBUG_getTracesInRange exposes getTracesInRange to make it easy to call directly from a helper executable such as trace_tool.

func (*BTTraceStore) GetDenseTile

func (b *BTTraceStore) GetDenseTile(ctx context.Context, nCommits int) (*tiling.Tile, []tiling.Commit, error)

GetDenseTile implements the TraceStore interface. It fetches the most recent tile and sees if there is enough non-empty data, then queries the next oldest tile until it has nCommits non-empty commits.

func (*BTTraceStore) GetTile

func (b *BTTraceStore) GetTile(ctx context.Context, nCommits int) (*tiling.Tile, []tiling.Commit, error)

GetTile implements the TraceStore interface. Of note, due to this request possibly spanning over multiple tiles, the ParamsSet may have a set of params that does not actually correspond to a trace (this shouldn't be a problem, but is worth calling out). For example, suppose a trace with param " device=alpha" abruptly ends on tile 4, commit 7 (where the device was removed from testing). If we are on tile 5 and need to query both tile 4 starting at commit 10 and tile 5 (the whole thing), we'll just merge the paramsets from both tiles, which includes the "device=alpha" params, but they don't exist in any traces seen in the tile (since it ended prior to our cutoff point).

func (*BTTraceStore) Put

func (b *BTTraceStore) Put(ctx context.Context, commitHash string, entries []*tracestore.Entry, ts time.Time) error

Put implements the TraceStore interface.

type TileKey

type TileKey int32

TileKey is the identifier for each tile held in BigTable.

Note that tile keys are in the opposite order of tile offset, that is, the first commit in a repo goes in the first tile, which has key 2^32-1. We do this so more recent tiles come first in sort order.

func GetTileKey

func GetTileKey(repoIndex int) (TileKey, int)

GetTileKey retrieves the tile key and the index of the commit in the given tile (commitIndex) given the index of a commit in the repo (repoIndex). commitIndex starts at 0 for the oldest commit in the tile.

func (TileKey) OpsRowName

func (t TileKey) OpsRowName() string

OpsRowName returns the name of the BigTable row which stores the OrderedParamSet for this tile.