README

The grailbio/base project includes all the packages used by many other grailbio Go packages:

Directories

Path Synopsis
Package admit contains utilities for admission control.
Package admit contains utilities for admission control.
Package backgroundcontext manages the singleton v23 context.
Package backgroundcontext manages the singleton v23 context.
Package bitset provides support for treating a []uintptr as a bitset.
Package bitset provides support for treating a []uintptr as a bitset.
cloud
awssession
Package awssession provides a simple way to obtain AWS session.Session using GRAIL tickets.
Package awssession provides a simple way to obtain AWS session.Session using GRAIL tickets.
cmd
gofat
Command gofat is a simple utility to make fat binaries in the fatbin format (see github.com/grailbio/base/fatbin).
Command gofat is a simple utility to make fat binaries in the fatbin format (see github.com/grailbio/base/fatbin).
grail-access
Command grail-access creates Vanadium credentials (also called principals) using either Google ID tokens (the default) or the AWS IAM role attached to an EC2 instance (requested using the '-ec2' flag).
Command grail-access creates Vanadium credentials (also called principals) using either Google ID tokens (the default) or the AWS IAM role attached to an EC2 instance (requested using the '-ec2' flag).
grail-fuse/gfs
Package gfs implements FUSE on top oh grailfile.
Package gfs implements FUSE on top oh grailfile.
grail-role
Command role creates Vanadium principals for a Vanadium role account.
Command role creates Vanadium principals for a Vanadium role account.
grail-role-group
Usage: role-group [flags] <command> The role-group commands are: list List all the role groups create Create a new role group update Update an existing role group help Display help for commands or topics The global flags are: -alsologtostderr=false log to standard error as well as files -block-profile= filename prefix for block profiles -block-profile-rate=1 rate for runtime.
Usage: role-group [flags] <command> The role-group commands are: list List all the role groups create Create a new role group update Update an existing role group help Display help for commands or topics The global flags are: -alsologtostderr=false log to standard error as well as files -block-profile= filename prefix for block profiles -block-profile-rate=1 rate for runtime.
grail-ssh
The following enables go generate to generate the doc.go file.
The following enables go generate to generate the doc.go file.
grail-ticket
Command grail-ticket retrieves a ticket from a ticket-server.
Command grail-ticket retrieves a ticket from a ticket-server.
oom
ticket-server
Command ticket-server runs a Vanadium server that provides restricted access to tickets.
Command ticket-server runs a Vanadium server that provides restricted access to tickets.
Package cmdutil provides utility routines for implementing command line tools.
Package cmdutil provides utility routines for implementing command line tools.
interactive
Package interactive is switching defaults for logging to not output anything to stderr.
Package interactive is switching defaults for logging to not output anything to stderr.
Package compress provides convenience functions for creating compressors and uncompressors based on filenames.
Package compress provides convenience functions for creating compressors and uncompressors based on filenames.
Package config is used to configure software systems.
Package config is used to configure software systems.
aws
http
Package http defines profile providers for local HTTP servers.
Package http defines profile providers for local HTTP servers.
crypto
encryption
Package encryption provides support for encrypting and decrypting data and files with granular key management.
Package encryption provides support for encrypting and decrypting data and files with granular key management.
Package data provides functionality for measuring and displaying data quantities.
Package data provides functionality for measuring and displaying data quantities.
diagnostic
dump
Package dump provides the endpoint "debug/dump", registered with http.DefaultServeMux, which returns a dump of useful diagnostic information as a tarball.
Package dump provides the endpoint "debug/dump", registered with http.DefaultServeMux, which returns a dump of useful diagnostic information as a tarball.
Package digest provides a generalized representation for digests computed with cryptographic hash functions.
Package digest provides a generalized representation for digests computed with cryptographic hash functions.
Package errors implements an error type that defines standard interpretable error codes for common error conditions.
Package errors implements an error type that defines standard interpretable error codes for common error conditions.
Package eventlog provides logging of semi-structured events, particularly in service of downstream analysis, e.g.
Package eventlog provides logging of semi-structured events, particularly in service of downstream analysis, e.g.
Package fatbin implements a simple fat binary format, and provides facilities for creating fat binaries and accessing its variants.
Package fatbin implements a simple fat binary format, and provides facilities for creating fat binaries and accessing its variants.
Package file provides basic file operations across multiple file-system types.
Package file provides basic file operations across multiple file-system types.
s3file
Package s3file implements grail file interface for S3.
Package s3file implements grail file interface for S3.
Package flock implements a simple POSIX file-based advisory lock.
Package flock implements a simple POSIX file-based advisory lock.
Package grail contains the Init function that all programs are expected to call.
Package grail contains the Init function that all programs are expected to call.
gtl
tests
Code generated by generate_randomized_freepool.py.
Code generated by generate_randomized_freepool.py.
Package intervalmap stores a set of (potentially overlapping) intervals.
Package intervalmap stores a set of (potentially overlapping) intervals.
Package iofmt contains various IO utilities used for formatting output.
Package iofmt contains various IO utilities used for formatting output.
Package limiter implements a concurrency limiter with support for contexts.
Package limiter implements a concurrency limiter with support for contexts.
Package log provides simple level logging.
Package log provides simple level logging.
Package logio implements a failure-tolerant log, typically used as a write-ahead log.
Package logio implements a failure-tolerant log, typically used as a write-ahead log.
Package mapio implements a sorted, on-disk map, similar to the SSTable data structure used in Bigtable [1], Cassandra [2], and others.
Package mapio implements a sorted, on-disk map, similar to the SSTable data structure used in Bigtable [1], Cassandra [2], and others.
Package must provides a handful of functions to express fatal assertions in Go programs.
Package must provides a handful of functions to express fatal assertions in Go programs.
Package psort includes functions for parallel sorting.
Package psort includes functions for parallel sorting.
Package recordio implements the recordio file format.
Package recordio implements the recordio file format.
recordioflate
Package recordioflate provides the "flate" transformer.
Package recordioflate provides the "flate" transformer.
recordioiov
Package recordioiov provides utility functions for dealing with [][]bytes, used by recordio transformers.
Package recordioiov provides utility functions for dealing with [][]bytes, used by recordio transformers.
Package retry contains utilities for implementing retry logic.
Package retry contains utilities for implementing retry logic.
security
identity
Package identity defines interfaces for Vanadium identity providers.
Package identity defines interfaces for Vanadium identity providers.
keycrypt
Package keycrypt implements an API for storing and retrieving opaque blobs of data stored in a secure fashion.
Package keycrypt implements an API for storing and retrieving opaque blobs of data stored in a secure fashion.
keycrypt/file
Package file implements a file-based keycrypt.
Package file implements a file-based keycrypt.
keycrypt/kms
Package kms implements a Keycrypt using AWS's KMS service and S3.
Package kms implements a Keycrypt using AWS's KMS service and S3.
ssh/certificateauthority
Package certificateauthority implements an x509 certificate authority.
Package certificateauthority implements an x509 certificate authority.
tls/certificateauthority
Package certificateauthority implements an x509 certificate authority.
Package certificateauthority implements an x509 certificate authority.
Package shutdown implements a global process shutdown mechanism.
Package shutdown implements a global process shutdown mechanism.
Package simd provides access to SIMD-based implementations of several common operations on byte arrays which the compiler cannot be trusted to autovectorize within the next several years.
Package simd provides access to SIMD-based implementations of several common operations on byte arrays which the compiler cannot be trusted to autovectorize within the next several years.
Package state implements atomic file-based state management with support for advisory locking.
Package state implements atomic file-based state management with support for advisory locking.
Package stateio implements persistent state mechanism based on log files that interleave indexed state snapshots with state updates.
Package stateio implements persistent state mechanism based on log files that interleave indexed state snapshots with state updates.
Package status provides facilities for reporting statuses from a number of tasks working towards a common goal.
Package status provides facilities for reporting statuses from a number of tasks working towards a common goal.
stress
oom
Package oom contains a single function to trigger Linux kernel OOMs.
Package oom contains a single function to trigger Linux kernel OOMs.
sync
once
Package once contains utilities for managing actions that must be performed exactly once.
Package once contains utilities for managing actions that must be performed exactly once.
Package traverse provides primitives for concurrent and parallel traversal of slices or user-defined collections.
Package traverse provides primitives for concurrent and parallel traversal of slices or user-defined collections.
Package tsv provides a simple TSV writer which takes care of number->string conversions and tabs, and is far more performant than fmt.Fprintf (thanks to use of strconv.Append{Uint,Float}).
Package tsv provides a simple TSV writer which takes care of number->string conversions and tabs, and is far more performant than fmt.Fprintf (thanks to use of strconv.Append{Uint,Float}).
Package vcontext creates a singleton vanadium Context object.
Package vcontext creates a singleton vanadium Context object.
web
webutil
Package webutil implements utility routines for web-related operations and applications.
Package webutil implements utility routines for web-related operations and applications.
Package writehash provides a set of utility functions to hash common types into hashes.
Package writehash provides a set of utility functions to hash common types into hashes.