bigquery

README

bigquery

Run Google BigQuery queries and inspect datasets, table schemas, and SQL runbooks, for the gollem LLM agent framework.

github.com/gollem-dev/tools/bigquery

Tools

Name	Description
bigquery_list_dataset	List available BigQuery datasets, tables, and partial schema.
bigquery_query	Execute a SQL query with scan-limit enforcement.
bigquery_result	Get the results of a previously executed query.
bigquery_table_summary	Get a summary of available BigQuery tables.
bigquery_schema	Get the detailed schema for a specific table.
get_runbook_entry	Get a SQL runbook entry by ID.

bigquery_query writes results to a GCS bucket and bigquery_result reads them back, so those two tools require WithStorageBucket. Queries are capped by WithScanLimit (bytes scanned) to guard against runaway costs.

Usage

ts, err := bigquery.New("my-project",
	bigquery.WithStorageBucket("my-result-bucket"), // needed for query/result
)
if err != nil {
	return err
}
if err := ts.Ping(ctx); err != nil { // optional preflight
	return err
}

Credentials come from Application Default Credentials unless overridden with WithCredentials or WithImpersonateServiceAccount.

Options

Option	Required	Default
WithStorageBucket(string)	for query/result	—
WithStoragePrefix(string)	no	—
WithCredentials(string)	no	Application Default Credentials
WithImpersonateServiceAccount(string)	no	—
WithConfigFiles([]string)	no	—
WithRunbookPaths([]string)	no	—
WithTimeout(time.Duration)	no	5m
WithScanLimit(string)	no	"10GB"
WithLogger(*slog.Logger)	no	slog.Default()

Testing

Mock tests run unconditionally. The live-service test runs only when the TEST_BIGQUERY_* variables are set:

TEST_BIGQUERY_PROJECT_ID=... TEST_BIGQUERY_STORAGE_BUCKET=... \
	TEST_BIGQUERY_DATASET=... TEST_BIGQUERY_TABLE=... \
	TEST_BIGQUERY_QUERY="SELECT 1" go test ./...

Optional: TEST_BIGQUERY_CREDENTIALS, TEST_BIGQUERY_CONFIG.

Documentation

Overview

Package bigquery provides a gollem.ToolSet for querying Google Cloud BigQuery, inspecting table schemas, and retrieving SQL runbook entries.

Package bigquery provides a gollem.ToolSet for querying Google Cloud BigQuery and inspecting table schemas and runbooks.

Index

• type ColumnConfig
• type Config
• type Option
  • func WithConfigFiles(paths []string) Option
  • func WithCredentials(path string) Option
  • func WithImpersonateServiceAccount(sa string) Option
  • func WithLogger(logger *slog.Logger) Option
  • func WithRunbookPaths(paths []string) Option
  • func WithScanLimit(s string) Option
  • func WithStorageBucket(bucket string) Option
  • func WithStoragePrefix(prefix string) Option
  • func WithTimeout(d time.Duration) Option
• type PartitioningConfig
• type RunbookEntries
• type RunbookEntry
• type RunbookID
  • func (x RunbookID) String() string
• type ToolSet
  • func New(projectID string, opts ...Option) (*ToolSet, error)
  • func (t *ToolSet) Ping(ctx context.Context) error
  • func (t *ToolSet) Run(ctx context.Context, name string, args map[string]any) (map[string]any, error)
  • func (t *ToolSet) Specs(_ context.Context) ([]gollem.ToolSpec, error)

Types

type ColumnConfig

type ColumnConfig struct {
	Name         string `yaml:"name"          json:"name"`
	Description  string `yaml:"description"   json:"description"`
	ValueExample string `yaml:"value_example" json:"value_example"`
	// Type is the BigQuery field type (STRING, INTEGER, FLOAT, BOOLEAN,
	// TIMESTAMP, DATE, TIME, DATETIME, BYTES, RECORD).
	Type string `yaml:"type"          json:"type"`
	// Fields holds nested column definitions for RECORD type columns.
	Fields []ColumnConfig `yaml:"fields"        json:"fields"`
}

ColumnConfig describes a single BigQuery table column.

type Config

type Config struct {
	DatasetID    string             `yaml:"dataset_id"    json:"dataset_id"`
	TableID      string             `yaml:"table_id"      json:"table_id"`
	Description  string             `yaml:"description"   json:"description"`
	Columns      []ColumnConfig     `yaml:"columns"       json:"columns"`
	Partitioning PartitioningConfig `yaml:"partitioning" json:"partitioning"`

	// RunbookPaths is an optional per-config list of SQL runbook file paths or
	// directories. These are merged with the top-level WithRunbookPaths option.
	RunbookPaths []string `yaml:"runbook_paths" json:"runbook_paths"`
}

Config describes a BigQuery table's metadata for the tool. It is loaded from YAML configuration files at construction time.

type Option

type Option func(*ToolSet)

Option configures a ToolSet.

func WithConfigFiles

func WithConfigFiles(paths []string) Option

WithConfigFiles sets YAML configuration file or directory paths. Each entry may be a file (*.yaml / *.yml) or a directory that is walked recursively. At least one valid config file is required for most tool operations.

func WithCredentials

func WithCredentials(path string) Option

WithCredentials sets the path to a Google Cloud service account credentials JSON file. When empty, Application Default Credentials are used.

func WithImpersonateServiceAccount

func WithImpersonateServiceAccount(sa string) Option

WithImpersonateServiceAccount sets a service account email for impersonation. When set, the tool obtains short-lived credentials for that account.

func WithLogger

func WithLogger(logger *slog.Logger) Option

WithLogger sets the structured logger. A nil logger keeps slog.Default().

func WithRunbookPaths

func WithRunbookPaths(paths []string) Option

WithRunbookPaths sets file or directory paths from which SQL runbook entries are loaded (*.sql files). Runbooks are loaded once during New.

func WithScanLimit

func WithScanLimit(s string) Option

WithScanLimit sets the maximum bytes a query may scan (parsed via go-humanize, e.g. "10GB"). Defaults to "10GB".

func WithStorageBucket

func WithStorageBucket(bucket string) Option

WithStorageBucket sets the GCS bucket name used for storing intermediate query results. Required for bigquery_query / bigquery_result.

func WithStoragePrefix

func WithStoragePrefix(prefix string) Option

WithStoragePrefix sets an optional path prefix for GCS objects. For example, "tools/bq/" results in objects like "tools/bq/bigquery/<queryID>/data.json".

func WithTimeout

func WithTimeout(d time.Duration) Option

WithTimeout sets the maximum time to wait for a BigQuery job to complete when calling bigquery_result. Defaults to 5 minutes.

type PartitioningConfig

type PartitioningConfig struct {
	Field    string `yaml:"field"     json:"field"`
	Type     string `yaml:"type"      json:"type"`
	TimeUnit string `yaml:"time_unit" json:"time_unit"`
}

PartitioningConfig describes BigQuery table partitioning settings.

type RunbookEntries

type RunbookEntries []*RunbookEntry

RunbookEntries is a slice of RunbookEntry pointers.

type RunbookEntry

type RunbookEntry struct {
	ID          RunbookID `json:"id"`
	Title       string    `json:"title"`
	Description string    `json:"description"`
	SQLContent  string    `json:"sql_content"`
}

RunbookEntry represents a single SQL runbook loaded from a file.

type RunbookID

type RunbookID string

RunbookID is the unique identifier for a runbook entry.

func (RunbookID) String

func (x RunbookID) String() string

String returns the string representation of the RunbookID.

type ToolSet

type ToolSet struct {
	// contains filtered or unexported fields
}

ToolSet implements gollem.ToolSet for Google Cloud BigQuery. All fields are unexported; configure via Option.

func New

func New(projectID string, opts ...Option) (*ToolSet, error)

New constructs the ToolSet with the required projectID, applies opts, loads config files from disk, and loads runbook SQL files. No network I/O is performed; use Ping to verify connectivity.

func (*ToolSet) Ping

func (t *ToolSet) Ping(ctx context.Context) error

Ping verifies connectivity by creating a BigQuery client. If the underlying client is a real *bigquery.Client, it calls Datasets to validate credentials and network access (iterating even a single entry proves the connection works; iterator.Done — no datasets — is also a success). If WithStorageBucket is set it also creates a GCS client.

func (*ToolSet) Run

func (t *ToolSet) Run(ctx context.Context, name string, args map[string]any) (map[string]any, error)

Run dispatches to the appropriate handler based on name.

func (*ToolSet) Specs

func (t *ToolSet) Specs(_ context.Context) ([]gollem.ToolSpec, error)

Specs returns the tool specifications for the BigQuery tool set.