truegrain

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 20, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

README

truegrain (Go)

The Go client for truegrain: governed metrics for agents, notebooks and applications.

Go Reference

go get github.com/rk-chavali/truegrain-go

No dependencies. Only the standard library, and a CI job proves it rather than the README asserting it. That is why this is a separate module from the engine: importing the engine would pull in a BigQuery client, an OIDC verifier and an MCP server, none of which a caller asking for a number needs.

There is no method that sends SQL, because there is no endpoint that accepts it. The only expressible request is a semantic one.

Use it

client, err := truegrain.FromEnv() // SEMANTIC_URL, SEMANTIC_TOKEN
if err != nil {
    return err
}

result, err := client.Query(ctx, truegrain.Request{
    Metrics:    []string{"sales.order_revenue", "marketing.campaign_spend"},
    Dimensions: []string{"sales.orders.order_date"},
    Grain:      "month",
    Filters:    []truegrain.Filter{truegrain.Eq("sales.orders.status", "shipped")},
})
if err != nil {
    return err
}

for _, row := range result.Maps() {
    fmt.Println(row)
}

Every Result carries CompiledSQL and ModelVersion. That is how a disagreement about a number gets settled: two results with the same digest were produced by exactly the same definitions.

Queries that outlive an HTTP request

A warehouse query can run longer than a connection should be held open.

result, err := client.Run(ctx, truegrain.Request{
    Metrics: []string{"sales.order_revenue"},
})

Run submits, polls with backoff, collects every page and returns one result, so no caller writes that loop. Cancelling ctx cancels the job on the engine too, rather than leaving the warehouse spending on an answer nobody is waiting for.

Read the refusal

The engine refuses questions it cannot answer correctly rather than returning a plausible wrong number. A refusal says what to do about itself, and that is what to branch on:

result, err := client.Run(ctx, req)

var refusal *truegrain.Refused
if errors.As(err, &refusal) {
    switch {
    case refusal.ShouldModify():
        // Answerable, but not as written. refusal.Hint names the metrics
        // defined at the grain where the question is well defined.
    case refusal.ShouldWait():
        // Nothing about the request is wrong. The same call may work shortly.
    case refusal.IsFinal():
        // A denial. Say so rather than substituting a different metric that
        // answers a different question.
    }
}

That distinction is the difference between a self-correcting agent loop and an infinite one.

Know what is actually enforced

Health reports two different things, and they are not the same:

health, _ := client.Health(ctx)

health.Governance.ColumnLevel      // what this engine enforces before emitting SQL
health.Dialect.ColumnLevelSecurity // what the warehouse enforces by itself
health.EnforcementNotes            // the gaps, in plain language

When both are false the engine's gate is the only control, and it applies only to queries that go through the engine. A caller with direct warehouse credentials is unaffected by it. Reading EnforcementNotes before trusting a deployment with anything sensitive is worth the two lines.

Reference documentation

Every exported type and method is documented on pkg.go.dev, generated from the doc comments in this repository.

How this stays in step with the engine

The engine's OpenAPI contract is vendored at spec/openapi.yaml, pinned to an engine commit in spec/PINNED_AT.

covers_spec_test.go checks this client against it in both directions: an operation the engine exposes with no method here fails the build, and a method claiming an operation the spec does not define fails it too. A scheduled job compares the pin against the engine daily and opens a pull request when the contract moves.

The client is hand-written rather than generated. A generator produces a transport and a flat error type; the parts worth having are the ones it cannot produce, namely the refusal semantics above and a Run that submits a job and polls it.

Licence

Apache 2.0, matching the engine.

Documentation

Overview

Package truegrain is the Go client for a truegrain semantic engine.

truegrain compiles an Apache Ossie semantic model into governed warehouse SQL. It answers questions phrased as metrics, dimensions and structured filters, and refuses the ones that would return a plausible wrong number. See https://github.com/rk-chavali/truegrain.

This package depends only on the standard library. It is deliberately a separate module from the engine: importing the engine would pull in the BigQuery client, an OIDC verifier and an MCP server, none of which a caller asking for a number needs.

Getting started 

client, err := truegrain.FromEnv()   // SEMANTIC_URL, SEMANTIC_TOKEN
if err != nil {
	return err
}

result, err := client.Query(ctx, truegrain.Request{
	Metrics:    []string{"sales.order_revenue"},
	Dimensions: []string{"sales.customers.region"},
	Filters:    []truegrain.Filter{truegrain.Eq("sales.orders.status", "shipped")},
})

Every Result carries CompiledSQL and ModelVersion. That is how a disagreement about a number gets settled: two results with the same digest were produced by exactly the same definitions.

There is no way to send SQL

No method on Client accepts a SQL string, because no endpoint on the engine accepts one. The only expressible request is a semantic one, which is what makes an ungoverned query impossible rather than merely discouraged.

Refusals are answers

The engine refuses questions it cannot answer correctly. A refusal is not a failure of the request so much as a statement about it, and it says what to do next. Branch on that rather than on the message: 

result, err := client.Run(ctx, req)
var refusal *truegrain.Refused
if errors.As(err, &refusal) {
	switch {
	case refusal.ShouldModify():
		// Answerable, but not as written. refusal.Hint names the metrics
		// defined at the grain where the question is well defined.
	case refusal.ShouldWait():
		// Nothing about the request is wrong. The same call may work shortly.
	case refusal.IsFinal():
		// A denial. Say so rather than substituting a different metric that
		// answers a different question.
	}
}

That distinction is the difference between a self-correcting agent loop and an infinite one.

Slow warehouses

A warehouse query can run longer than an HTTP request should be held open. Client.Run submits the query, polls with backoff, collects every page and returns one Result, so no caller writes that loop: 

result, err := client.Run(ctx, truegrain.Request{
	Metrics: []string{"sales.order_revenue"},
})

Cancelling the context cancels the job on the engine as well, rather than leaving the warehouse spending on an answer nobody is waiting for.

Index

Constants

View Source 
const (
	// ReloadReading means the engine has started re-reading its source.
	ReloadReading = "reading"
	// ReloadAlreadyRunning means a sync was already in flight. It is not an
	// error: the caller is getting what they asked for.
	ReloadAlreadyRunning = "already running"
)

What a Client.Reload was told.

View Source 
const DefaultTimeout = 60 * time.Second

DefaultTimeout bounds a single HTTP call.

Variables

View Source 
var Operations = map[string]string{
	"getHealth":       "Health",
	"getModelVersion": "ModelVersion",
	"listNamespaces":  "Namespaces",
	"listMetrics":     "Metrics",
	"describeMetric":  "Metric",
	"listDimensions":  "Dimensions",
	"query":           "Query",
	"compile":         "Compile",
	"submitJob":       "Submit",
	"getJob":          "Job",
	"cancelJob":       "CancelJob",
	"listAudit":       "Audit",
	"doctor":          "Doctor",
	"doctorHistory":   "DoctorHistory",
	"runTests":        "RunTests",
	"policy":          "Policy",
	"explainPolicy":   "ExplainPolicy",
	"diff":            "Diff",
	"reload":          "Reload",
}

Operations maps each OpenAPI operationId to the Client method implementing it.

TestClientCoversTheSpec asserts this covers every operation in spec/openapi.yaml, so an endpoint the engine gains cannot quietly go unsupported here.

Functions

This section is empty.

Types

type AuditEvent added in v1.1.0 

type AuditEvent struct {
	Time         time.Time `json:"time"`
	Identity     string    `json:"identity,omitempty"`
	ModelName    string    `json:"model_name,omitempty"`
	ModelVersion string    `json:"model_version,omitempty"`
	Namespace    string    `json:"namespace,omitempty"`
	Metrics      []string  `json:"metrics,omitempty"`
	Dimensions   []string  `json:"dimensions,omitempty"`
	Decision     Decision  `json:"decision"`
	// RefusalCode names which refusal, for a Decision of DecisionRefused. The
	// same codes [Refused] carries, so one switch serves both.
	RefusalCode string `json:"refusal_code,omitempty"`
	// Retry is what the caller was told to do.
	Retry  Retry  `json:"retry,omitempty"`
	Reason string `json:"reason,omitempty"`
	Hint   string `json:"hint,omitempty"`
	// DeniedFields are the semantic fields withheld, on a denial. Never
	// returned to the denied caller, only to a reader of the record.
	DeniedFields []string `json:"denied_fields,omitempty"`
	SQLHash      string   `json:"sql_hash,omitempty"`
	Dialect      string   `json:"dialect,omitempty"`
	// JobID is the warehouse's own identifier, so two logs can be joined.
	JobID    string `json:"job_id,omitempty"`
	RowCount int    `json:"row_count,omitempty"`
	// BytesBilled is what the warehouse says the query cost. Zero means not
	// reported rather than free: DuckDB bills nobody and reports nothing.
	BytesBilled int64  `json:"bytes_billed,omitempty"`
	DurationMS  int64  `json:"duration_ms,omitempty"`
	Error       string `json:"error,omitempty"`
}

AuditEvent is one decision the engine recorded.

Decision is the field to branch on, and Refused and Denied are deliberately separate: denied is access, meaning this caller may not read something, and refused is correctness, meaning nobody can be told this accurately. Counting them together would report every fan-out as an access incident.

Carries no filter values, no compiled SQL and no result rows. SQLHash identifies the statement without disclosing it.

type Cell 

type Cell any

Cell is one value in a result row. The engine returns JSON, so a cell is whatever JSON gave: a string, a float64, a bool, or nil.

type Change added in v1.1.0 

type Change struct {
	Before string `json:"before"`
	After  string `json:"after"`
}

Change is what one request compiled to before and after a reload.

type Client 

type Client struct {
	// BaseURL is where the engine is served, without a trailing slash.
	BaseURL string
	// HTTPClient is used for every request. Replace it to add tracing, a
	// proxy, or a different timeout.
	HTTPClient *http.Client
	// UserAgent identifies the caller. The engine's audit log records it,
	// which is how one agent is told from another.
	UserAgent string
	// contains filtered or unexported fields
}

Client is a connection to one truegrain engine.

It is safe for concurrent use. Build one with New or FromEnv.

func FromEnv 

func FromEnv(opts ...Option) (*Client, error)

FromEnv builds a client from SEMANTIC_URL and SEMANTIC_TOKEN.

Keeping the token in the environment keeps it out of source and out of the process list.

func New 

func New(baseURL string, opts ...Option) (*Client, error)

New builds a client for the engine at baseURL.

func (*Client) Audit added in v1.1.0 

func (c *Client) Audit(ctx context.Context, decision Decision, limit int) ([]AuditEvent, error)

Audit reads the decisions this engine recently made, newest first.

The window an operator reads to answer "why did that agent give up". Pass DecisionRefused for the usual case; an empty decision returns every kind. limit is clamped to the engine's maximum of 200; pass 0 for the default.

Not served unless an operator has named who may read it, because the record discloses what other teams query and which fields are protected. An engine with no reader configured answers 404 and a caller who is authenticated but not a named reader gets 403, both of which arrive here as a *Refused.

func (*Client) CancelJob 

func (c *Client) CancelJob(ctx context.Context, id string) (*Job, error)

CancelJob stops a running query. It is idempotent: cancelling a finished job returns its existing state rather than an error.

func (*Client) Compile 

func (c *Client) Compile(ctx context.Context, req Request) (*Compiled, error)

Compile returns the SQL a request compiles to, without running it.

A dry run is still governed: a request you may not run returns a refusal and no SQL, and the inspection is audited. It is a way to see what a query would do, not a way around the gate.

func (*Client) Diff added in v1.1.0 

func (c *Client) Diff(ctx context.Context) (*Diff, error)

Diff reports whether the last model reload moved a number.

This compares the model being served against the one served before it, which is the comparison nobody can make from outside the process. An engine that has served only one model answers 404 as a *Refused, because that is a different answer from nothing having changed and only one of them is reassuring.

func (*Client) Dimensions 

func (c *Client) Dimensions(ctx context.Context, metric string) ([]Dimension, error)

Dimensions lists what is available for grouping and filtering.

Pass a metric name to get only the dimensions valid for it, which is almost always what you want.

func (*Client) Doctor added in v1.1.0 

func (c *Client) Doctor(ctx context.Context) (*Diagnosis, error)

Doctor asks the warehouse whether the model is still true.

Everything else validates the model against itself: the YAML parses, the joins resolve, the expressions compile. None of that notices that somebody dropped a column last Tuesday, and the first sign of that is usually a caller getting an error, which is the most expensive place to find it.

Reports rather than refuses, so a non-nil error here means the check could not run. Read Diagnosis.OK for the verdict and Diagnosis.Skipped for the case where nothing could be checked at all.

func (*Client) DoctorHistory added in v1.1.0 

func (c *Client) DoctorHistory(ctx context.Context) (*DoctorHistory, error)

DoctorHistory returns what the scheduled check has seen, oldest first.

Drift is found by looking regularly, not by looking once, which is how "when did this start" stays answerable. An engine started without -doctor-every has nothing scheduled and answers 404 as a *Refused.

func (*Client) ExplainPolicy added in v1.1.0 

func (c *Client) ExplainPolicy(ctx context.Context, metric string) (*PolicyExplanation, error)

ExplainPolicy says what the caller may read of a metric, and why.

Answers "why can I not group by that column" without running a query and being denied. For the calling identity only, which is a security property rather than a limitation: an endpoint that reported another identity's access would publish the policy it was configured to enforce.

metric is the qualified name, as Client.Metrics reports it.

func (*Client) Health 

func (c *Client) Health(ctx context.Context) (*Health, error)

Health reports what this deployment enforces, including what it does not.

Worth reading before trusting the deployment with anything sensitive: Governance.Note and Executor state the gaps in plain language.

func (*Client) Job 

func (c *Client) Job(ctx context.Context, id, cursor string, pageSize int) (*Job, error)

Job polls a job and reads one page of its rows.

cursor comes from a previous page's NextCursor; pass an empty string for the first page. pageSize of zero uses the engine's default.

func (*Client) Metric 

func (c *Client) Metric(ctx context.Context, name string) (*Metric, error)

Metric returns one metric's definition and the dimensions legal for it.

name is qualified as `namespace.metric`, or bare when unambiguous across the workspace.

func (*Client) Metrics 

func (c *Client) Metrics(ctx context.Context, search string) ([]Metric, error)

Metrics lists every metric this identity may read.

search is a case-insensitive substring over name and description; pass an empty string for all of them.

func (*Client) ModelVersion 

func (c *Client) ModelVersion(ctx context.Context) (string, error)

ModelVersion returns the workspace digest currently served.

Every Result carries this too. Two results with the same digest were produced by exactly the same definitions.

func (*Client) Namespaces 

func (c *Client) Namespaces(ctx context.Context) ([]Namespace, error)

Namespaces lists the namespaces in the workspace, their owners and whether each one loaded.

func (*Client) Policy added in v1.1.0 

func (c *Client) Policy(ctx context.Context) (*Policy, error)

Policy reports what this engine enforces, and what it does not.

Says nothing about who is allowed what. For that, and only about yourself, use Client.ExplainPolicy.

func (*Client) Query 

func (c *Client) Query(ctx context.Context, req Request) (*Result, error)

Query answers a question and returns rows.

It holds the connection open for the whole query. Use Client.Run when the warehouse might be slow.

A refusal comes back as a *Refused; check it with errors.As and branch on ShouldModify, ShouldWait and IsFinal rather than on the message.

func (*Client) Reload added in v1.1.0 

func (c *Client) Reload(ctx context.Context) (string, error)

Reload tells the engine to re-read its model source, and returns ReloadReading or ReloadAlreadyRunning.

It carries no model, deliberately: this means "look now", not "install this". The engine already follows git on a timer, so the only thing this changes is the wait. A method that accepted a model would be a second way into production, one that skips the pull request, the checks and the diff that reports which numbers move.

Needs the deploy:model scope, which read:model and run:query never imply.

Returning without error does not mean the new model is serving. A sync is not instant, and reporting a commit before the swap happened would be a claim a pipeline then asserts as fact. Poll Client.Health and read Origin.Commit to know when the new model is the one answering.

An engine reading from a path has nothing to re-read and answers 404. A model that fails to load is not a failure of this call either: the engine keeps serving the previous one and says so in a 502. Both are *Refused.

func (*Client) Run 

func (c *Client) Run(ctx context.Context, req Request, opts ...RunOptions) (*Result, error)

Run executes a query that may take longer than an HTTP request should.

It submits the query, polls until it finishes, collects every page and returns one Result, so no caller writes that loop. A large warehouse scan routinely outlives the default timeout of an HTTP client, and a synchronous call that times out leaves the query running and billable with nobody reading it.

The governance gate runs during submission, so a request this identity may not make returns a refusal immediately rather than after a wait.

Cancelling ctx cancels the job on the engine too, rather than leaving the warehouse spending on an answer nobody is waiting for.

func (*Client) RunTests added in v1.1.0 

func (c *Client) RunTests(ctx context.Context) (*TestReport, error)

RunTests asserts what this model answers.

validate says the model holds together and Client.Diff says a number changed. Neither says a number was ever right.

Check TestReport.Withheld as well as OK. A credential without run:query cannot cause warehouse execution, so cases that would are withheld and counted rather than run or silently dropped, and a caller reading only OK would conclude a suite passed when half of it never ran.

func (*Client) Submit 

func (c *Client) Submit(ctx context.Context, req Request) (*Job, error)

Submit starts a query in the background and returns immediately.

The returned Job already carries CompiledSQL, because compilation happened synchronously. A job coming back at all means the query was authorized, not merely that it was accepted.

Prefer Client.Run unless you genuinely need to do something else while the query runs.

func (*Client) Wait 

func (c *Client) Wait(ctx context.Context, id string, opts ...RunOptions) (*Job, error)

Wait polls until a job reaches a terminal state.

It backs off from a fast first poll to a two second ceiling, so a query that finishes quickly is not delayed and a slow one is not hammered.

If ctx is cancelled the job is cancelled on the engine before returning, so giving up here also stops the warehouse work.

type Compiled 

type Compiled struct {
	SQL          string   `json:"compiled_sql"`
	Columns      []string `json:"columns"`
	Dialect      string   `json:"dialect"`
	Namespace    string   `json:"namespace"`
	ModelVersion string   `json:"model_version"`
	// Parts is how many facts were aggregated separately and joined. More than
	// one means the query spanned grains or namespaces.
	Parts int `json:"parts"`
}

Compiled is the SQL a request compiles to, without having run it.

type Decision added in v1.1.0 

type Decision string

Decision is what the engine did about one request. Named rather than a bare string so a caller switching on it cannot pass a value the engine never produces, the same reason Retry is a type. 

const (
	DecisionAllowed  Decision = "allowed"
	DecisionRefused  Decision = "refused"
	DecisionDenied   Decision = "denied"
	DecisionCompiled Decision = "compiled"
	DecisionError    Decision = "error"
)

The decisions an AuditEvent can carry. Refused and Denied are separate on purpose: see AuditEvent.

type Diagnosis added in v1.1.0 

type Diagnosis struct {
	// OK is false when a finding would break a query. It is not the same as
	// having no findings.
	OK            bool      `json:"ok"`
	TablesChecked int       `json:"tables_checked"`
	Findings      []Finding `json:"findings"`
	// Skipped explains why nothing was checked, when nothing was. An empty
	// Diagnosis with Skipped set is the engine saying it could not look, not
	// saying everything is fine.
	Skipped string `json:"skipped,omitempty"`
}

Diagnosis is what the warehouse says about the model right now.

It reports rather than refuses. A model can be wrong in ways that do not matter yet, and which of those to act on is a person's decision.

type DialectSecurity 

type DialectSecurity struct {
	// Dialect names the compilation target, for example bigquery or duckdb.
	Dialect string `json:"dialect"`
	// ColumnLevelSecurity reports whether the warehouse restricts columns on
	// its own, such as through BigQuery policy tags.
	ColumnLevelSecurity bool `json:"column_level_security"`
	// RowLevelSecurity reports whether the warehouse filters rows on its own.
	RowLevelSecurity bool `json:"row_level_security"`
	// Note states the limits in plain language.
	Note string `json:"note,omitempty"`
}

DialectSecurity reports what the target warehouse enforces by itself, independently of this engine.

When both flags are false, the engine's own gate is the only control, and it applies only to queries that go through the engine.

type Diff added in v1.1.0 

type Diff struct {
	Changed bool `json:"changed"`
	// From and To are the model versions either side of the reload.
	From    string   `json:"from"`
	To      string   `json:"to"`
	Added   []string `json:"added,omitempty"`
	Removed []string `json:"removed,omitempty"`
	// Altered maps a request label to its SQL before and after.
	Altered map[string]Change `json:"altered,omitempty"`
}

Diff is what the last reload moved.

Compared on compiled SQL rather than on model text, because that is where a silent correctness incident lives: the model still validates, the tests still pass, and every dashboard quietly moves. Renaming a description does not appear here; changing a join, a grain or an expression does.

type Dimension 

type Dimension struct {
	Name        string   `json:"name"`
	Namespace   string   `json:"namespace,omitempty"`
	Datatype    string   `json:"datatype,omitempty"`
	Description string   `json:"description,omitempty"`
	Synonyms    []string `json:"synonyms,omitempty"`
	// IsTime reports whether this dimension accepts a Grain.
	IsTime bool `json:"is_time,omitempty"`
	// Grains are the legal time buckets. Populated only when IsTime.
	Grains []string `json:"grains,omitempty"`
}

Dimension is something a metric can be grouped or filtered by.

type DoctorHistory added in v1.1.0 

type DoctorHistory struct {
	// EverySeconds is the configured interval, which is how a reader tells a
	// gap from a check that has simply not come round yet.
	EverySeconds int `json:"every_seconds"`
	// Drifted counts runs that completed and found the warehouse changed.
	Drifted int         `json:"drifted"`
	Runs    []DoctorRun `json:"runs"`
}

DoctorHistory is what the scheduled check has seen, oldest first.

type DoctorRun added in v1.1.0 

type DoctorRun struct {
	At            time.Time `json:"at"`
	OK            bool      `json:"ok"`
	TablesChecked int       `json:"tables_checked"`
	Findings      int       `json:"findings"`
	// Error is set when the check could not run at all, which is a different
	// thing from running and finding something wrong.
	Error string `json:"error,omitempty"`
	// ModelVersion ties the result to what was being served, so a run from
	// before a reload is not read as evidence about the model after it.
	ModelVersion string `json:"model_version,omitempty"`
}

DoctorRun is one scheduled check.

type Filter 

type Filter struct {
	// Dimension is `dataset.field`, or `namespace.dataset.field`.
	Dimension string `json:"dimension"`
	// Op is the comparison. Use the constructors rather than setting it.
	Op string `json:"op"`
	// Values holds one value for the comparisons, two for between, one or more
	// for in and not_in, and none for the null checks.
	Values []any `json:"values,omitempty"`
}

Filter is a structured predicate.

It is never a SQL fragment. That is what keeps an ungoverned predicate inexpressible, and it is why these are constructors rather than a string builder: a misspelled operator is caught at the call site rather than as a refusal from the server.

func Between 

func Between(dimension string, low, high any) Filter

Between matches rows where the dimension falls within the inclusive range.

The engine refuses a reversed range rather than matching no rows, because an empty result that looks like a real answer is worse than an error.

func Eq 

func Eq(dimension string, value any) Filter

Eq matches rows where the dimension equals value.

func Gt 

func Gt(dimension string, value any) Filter

Gt matches rows where the dimension is greater than value.

func Gte 

func Gte(dimension string, value any) Filter

Gte matches rows where the dimension is greater than or equal to value.

func In 

func In(dimension string, values ...any) Filter

In matches rows where the dimension is any of values.

func IsNotNull 

func IsNotNull(dimension string) Filter

IsNotNull matches rows where the dimension has a value.

func IsNull 

func IsNull(dimension string) Filter

IsNull matches rows where the dimension has no value.

func Lt 

func Lt(dimension string, value any) Filter

Lt matches rows where the dimension is less than value.

func Lte 

func Lte(dimension string, value any) Filter

Lte matches rows where the dimension is less than or equal to value.

func Ne 

func Ne(dimension string, value any) Filter

Ne matches rows where the dimension does not equal value.

func NotIn 

func NotIn(dimension string, values ...any) Filter

NotIn matches rows where the dimension is none of values.

type Finding added in v1.1.0 

type Finding struct {
	Severity Severity `json:"severity"`
	// Dataset and Field name it in model terms, Source in warehouse terms,
	// because the person reading this has to look in both places.
	Dataset string `json:"dataset"`
	Field   string `json:"field,omitempty"`
	Source  string `json:"source,omitempty"`
	Message string `json:"message"`
	// Hint says what to do, when there is something to do.
	Hint string `json:"hint,omitempty"`
}

Finding is one thing the warehouse disagrees with the model about.

type Governance 

type Governance struct {
	// Resolver names the implementation, for example bigquery-policy-tags,
	// file, or allow-all.
	Resolver string `json:"resolver"`
	// ColumnLevel reports whether real column-level decisions are made. False
	// means every caller may read every column.
	ColumnLevel bool `json:"column_level"`
	// Note states the limits in plain language.
	Note string `json:"note"`
}

Governance describes what this engine's policy resolver enforces.

It is deliberately honest about a resolver that enforces nothing: a deployment that grants everything says so here, because overstating a guarantee is worse than not offering one.

type Health 

type Health struct {
	// Workspace names the namespaces served.
	Workspace string `json:"workspace"`
	// WorkspaceDigest identifies the exact set of definitions in force. Two
	// results carrying the same digest came from the same model.
	WorkspaceDigest string `json:"workspace_digest"`
	// OssieSpecVersion is the Apache Ossie spec version the models declare.
	OssieSpecVersion string `json:"ossie_spec_version,omitempty"`
	// Namespaces reports each namespace and whether it loaded.
	Namespaces []Namespace `json:"namespaces,omitempty"`
	// Dialect describes the compilation target and what it secures itself.
	Dialect DialectSecurity `json:"dialect"`
	// Governance describes the policy resolver this engine applies.
	Governance Governance `json:"governance"`
	// Executor names the warehouse driver and states its limits, including
	// whether queries run as the caller.
	Executor string `json:"executor"`
	// MetricCount and DimensionCount are what this identity may see.
	MetricCount    int `json:"metric_count"`
	DimensionCount int `json:"dimension_count"`
	// SupportedFilterOps are the operators [Filter] may use.
	SupportedFilterOps []string `json:"supported_filter_ops,omitempty"`
	// SupportedGrains are the legal values for Request.Grain.
	SupportedGrains []string `json:"supported_grains,omitempty"`
	// EnforcementNotes states the gaps in plain language. Read these.
	EnforcementNotes []string `json:"enforcement_notes,omitempty"`
	// Origin is where the served model came from, and is nil when the engine
	// read it from a path rather than following a repository. Nil is the
	// answer to "is this deployment under version control", so check it
	// before reading through it.
	Origin *Origin `json:"origin,omitempty"`
}

Health describes what a deployment enforces, including what it does not.

Worth reading before trusting a deployment with anything sensitive. Overstating a governance guarantee is worse than not offering one, so the engine reports its gaps here in plain language.

Two separate things are reported and they are not the same: Governance says what this engine enforces before it emits SQL, and DialectSecurity says what the warehouse enforces on its own. A caller with direct warehouse credentials is subject only to the second.

type Job 

type Job struct {
	ID    string   `json:"job_id"`
	State JobState `json:"state"`
	// CompiledSQL is available from the moment of submission, because
	// compilation and the governance gate run before the job exists.
	CompiledSQL string   `json:"compiled_sql,omitempty"`
	Columns     []string `json:"columns,omitempty"`
	// Rows is one page. Use [Client.Run] to collect every page.
	Rows [][]Cell `json:"rows,omitempty"`
	// RowCount is the size of the whole result, not of this page.
	RowCount int `json:"row_count,omitempty"`
	// NextCursor is present when more rows remain. Pass it back to read on.
	NextCursor   string `json:"next_cursor,omitempty"`
	ModelVersion string `json:"model_version,omitempty"`
	Namespace    string `json:"namespace,omitempty"`
	Dialect      string `json:"dialect,omitempty"`

	// The refusal fields, set only when State is StateFailed. They carry the
	// same vocabulary a synchronous refusal does.
	Code   string `json:"code,omitempty"`
	Reason string `json:"reason,omitempty"`
	Hint   string `json:"hint,omitempty"`
	Retry  Retry  `json:"retry,omitempty"`
}

Job is one asynchronously executing query.

func (*Job) Done 

func (j *Job) Done() bool

Done reports whether the job reached any terminal state, including failure.

func (*Job) Err 

func (j *Job) Err() error

Err returns a non-nil error when the job did not succeed.

A failure is returned as a *Refused carrying the same code, hint and retry class a synchronous refusal would have, so a caller branches on it identically.

func (*Job) Result 

func (j *Job) Result(rows [][]Cell) *Result

Result renders a finished job's rows as a Result.

rows overrides the single page held on the job, which is how Client.Run returns every page as one result.

type JobState 

type JobState string

JobState is where an asynchronous query has got to.

StateRunning is the only non-terminal one, and StateCancelled is distinct from StateFailed because the caller stopped it rather than the warehouse failing. 

const (
	StateRunning   JobState = "running"
	StateSucceeded JobState = "succeeded"
	StateFailed    JobState = "failed"
	StateCancelled JobState = "cancelled"
)

The states a job can be in.

type Metric 

type Metric struct {
	// Name is qualified as `namespace.metric`.
	Name      string `json:"name"`
	Namespace string `json:"namespace"`
	// Description is the grounding text an agent reads to decide whether this
	// metric answers the question asked. It is not decoration.
	Description string   `json:"description"`
	Datatype    string   `json:"datatype,omitempty"`
	Synonyms    []string `json:"synonyms,omitempty"`
	// Definition is the metric's expression. Present on [Client.Metric],
	// absent from a listing.
	Definition string `json:"definition,omitempty"`
	// Dimensions lists only the dimensions this identity may group by.
	Dimensions []Dimension `json:"-"`
}

Metric is something the engine can measure.

type Namespace 

type Namespace struct {
	Name        string   `json:"name"`
	Available   bool     `json:"available"`
	MetricCount int      `json:"metric_count"`
	Digest      string   `json:"digest,omitempty"`
	Owners      []string `json:"owners,omitempty"`
	// Error is set only when the namespace failed to load. An unavailable
	// namespace is reported rather than omitted, so absence is never mistaken
	// for a model that simply had no such metrics.
	Error string `json:"error,omitempty"`
}

Namespace is one team's independently owned set of definitions.

type Option 

type Option func(*Client)

Option configures a Client.

func WithHTTPClient 

func WithHTTPClient(hc *http.Client) Option

WithHTTPClient replaces the HTTP client, which is how to add a proxy, a tracing transport, or a different timeout.

func WithToken 

func WithToken(token string) Option

WithToken sets the bearer token identifying the workload.

Read it from the environment; never hard-code one. A token passed as a command-line flag ends up in the process list.

func WithUserAgent 

func WithUserAgent(ua string) Option

WithUserAgent overrides the identifying header, which is useful when you want the engine's audit log to distinguish one agent from another.

type Order 

type Order struct {
	// Field is a metric or dimension name from the same request.
	Field string `json:"field"`
	// Desc sorts descending.
	Desc bool `json:"desc,omitempty"`
}

Order is one sort key.

type Origin added in v1.1.0 

type Origin struct {
	// Repository is the clone URL, without credentials.
	Repository string `json:"repository,omitempty"`
	// Ref is the branch, tag or commit asked for, absent when the remote's
	// default branch is followed. A ref moves; read Commit to know what
	// actually answered.
	Ref string `json:"ref,omitempty"`
	// Commit is the revision serving right now.
	Commit string `json:"commit,omitempty"`
	// Subdirectory is the directory inside the repository holding the
	// workspace.
	Subdirectory string `json:"subdirectory,omitempty"`
}

Origin identifies the commit a served model came from.

This is how Client.Reload is confirmed. Reload names no commit, because a sync is not instant and reporting one before the swap happened would be a claim a pipeline then asserts as fact. So a deploy is finished when Health reports the commit you merged, and not before.

Carries no credential. A repository URL with one in it is refused at startup rather than stored here and redacted on the way out.

type Policy added in v1.1.0 

type Policy struct {
	Governance Governance `json:"governance"`
	// EnforcementNotes state the gaps in plain language. Read these: an
	// engine running allow-all says so here rather than letting a reader
	// assume a gate exists because the product has one.
	EnforcementNotes []string `json:"enforcement_notes,omitempty"`
}

Policy is what the engine enforces. It says nothing about who is allowed what, deliberately.

type PolicyExplanation added in v1.1.0 

type PolicyExplanation struct {
	Metric   string `json:"metric"`
	Identity string `json:"identity,omitempty"`
	// Readable are the dimensions this caller may group the metric by,
	// qualified and sorted.
	Readable   []string   `json:"readable"`
	Governance Governance `json:"governance"`
}

PolicyExplanation answers "why can I not group by that column".

For the calling identity only. An engine that reported what somebody else can see would publish the policy it was configured to enforce, so there is no field here for another identity and no endpoint that takes one.

type Refused 

type Refused struct {
	// Code is the machine-readable reason. Branch on this, never on the text.
	Code string `json:"code"`
	// Reason is one sentence stating what was wrong.
	Reason string `json:"reason"`
	// Hint is what to do instead. For a fan-out refusal it names the metrics
	// defined at the grain where the question is well defined.
	Hint string `json:"hint,omitempty"`
	// Retry classifies the refusal. See [Retry].
	Retry Retry `json:"retry"`
	// Status is the HTTP status that carried the refusal, or 0 when the
	// refusal came from a failed job rather than from a response.
	Status int `json:"-"`
}

Refused is a structured denial from the engine.

The engine understood the request and declined it, and said what to do about that. Use errors.As to recover it: 

var refusal *truegrain.Refused
if errors.As(err, &refusal) && refusal.ShouldModify() {
	// adjust the request and try again
}

func (*Refused) Error 

func (r *Refused) Error() string

Error renders the refusal, including what to do about it.

func (*Refused) IsFinal 

func (r *Refused) IsFinal() bool

IsFinal reports that no version of this request from this caller will succeed.

Usually a denial. Say so rather than substituting a different metric that answers a different question.

func (*Refused) ShouldModify 

func (r *Refused) ShouldModify() bool

ShouldModify reports that the request is answerable, but not as written.

Change the arguments and try again. Repeating it unchanged will not work. Hint usually names what to change.

func (*Refused) ShouldWait 

func (r *Refused) ShouldWait() bool

ShouldWait reports that nothing about the request is wrong.

Something outside it failed, such as the policy source being unreachable. The same request may succeed shortly.

type Request 

type Request struct {
	// Metrics names what to measure. Metrics at different grains, or in
	// different namespaces, are aggregated separately and joined on the shared
	// dimensions rather than refused.
	Metrics []string `json:"metrics"`
	// Dimensions group the result: `dataset.field`, or
	// `namespace.dataset.field` when a bare name would be ambiguous.
	Dimensions []string `json:"dimensions,omitempty"`
	// Filters are structured predicates, never SQL text. Build them with [Eq],
	// [In], [Between] and the rest.
	Filters []Filter `json:"filters,omitempty"`
	// Grain buckets the selected time dimension: day, week, month, quarter,
	// year and the sub-day units.
	Grain string `json:"grain,omitempty"`
	// Limit caps the rows returned. Zero applies the engine's own default.
	Limit int `json:"limit,omitempty"`
	// OrderBy sorts the result.
	OrderBy []Order `json:"order_by,omitempty"`
}

Request is a semantic question.

It is the same shape for Client.Query, Client.Compile, Client.Submit and Client.Run. There is no field for SQL, by design.

type Result 

type Result struct {
	Columns []string `json:"columns"`
	Rows    [][]Cell `json:"rows"`
	// CompiledSQL is the exact statement executed. Carried on every response
	// on purpose: it is how a disagreement about a number gets settled.
	CompiledSQL  string `json:"compiled_sql"`
	ModelVersion string `json:"model_version"`
	Namespace    string `json:"namespace"`
	Dialect      string `json:"dialect"`
}

Result is rows, plus the provenance needed to defend the numbers in them.

func (*Result) Maps 

func (r *Result) Maps() []map[string]Cell

Maps returns every row keyed by column name, which is usually what a caller wants to range over.

func (*Result) RowCount 

func (r *Result) RowCount() int

RowCount is the number of rows in the result.

type Retry 

type Retry string

Retry is what a caller should do about a refusal.

It is the field an agent branches on. Without it a caller either gives up on a typo or loops forever on a denial. 

const (
	// RetryModify means the request was understood and cannot be answered as
	// written. Changing the arguments may work; repeating it will not.
	RetryModify Retry = "modify"
	// RetryLater means the same request may succeed later. Something outside
	// the request failed, such as the policy source being unreachable, or this
	// caller is at their concurrency limit.
	RetryLater Retry = "later"
	// RetryNever means no version of this request from this caller will
	// succeed. A denial, or something the model itself has to change.
	RetryNever Retry = "never"
)

type RunOptions 

type RunOptions struct {
	// PollInterval is the first gap between polls. It backs off to two seconds.
	// Zero uses a sensible default.
	PollInterval time.Duration
	// PageSize is rows per page while collecting. Lower it when rows are wide.
	PageSize int
}

RunOptions tunes Client.Run.

type Severity added in v1.1.0 

type Severity string

Severity grades a Finding. 

const (
	// SeverityError is something that will break a query.
	SeverityError Severity = "error"
	// SeverityWarning is something that will not break today.
	SeverityWarning Severity = "warning"
)

type TestCase added in v1.1.0 

type TestCase struct {
	Name    string `json:"name"`
	Passed  bool   `json:"passed"`
	Skipped bool   `json:"skipped"`
	// Reason says why, for a case that failed or was skipped.
	Reason     string `json:"reason,omitempty"`
	DurationMS int64  `json:"duration_ms"`
}

TestCase is one assertion and what became of it.

type TestReport added in v1.1.0 

type TestReport struct {
	OK      bool `json:"ok"`
	Passed  int  `json:"passed"`
	Failed  int  `json:"failed"`
	Skipped int  `json:"skipped"`
	// Withheld counts cases this credential may not run, because they would
	// execute against the warehouse. Reported rather than dropped: a suite
	// that quietly checked half of what it claimed is worse than one that
	// failed. A caller reading OK without reading this is being misled.
	Withheld int        `json:"withheld"`
	Results  []TestCase `json:"results"`
}

TestReport is the result of every case in the suite.

type TransportError 

type TransportError struct {
	// URL is what was being called.
	URL string
	// Err is the underlying cause.
	Err error
}

TransportError means the engine could not be reached, or answered with something unparseable.

It is a network or deployment problem, never a statement about the request, so retrying the same request is reasonable.

func (*TransportError) Error 

func (e *TransportError) Error() string

func (*TransportError) Unwrap 

func (e *TransportError) Unwrap() error

Unwrap returns the underlying cause.

type UnauthorizedError 

type UnauthorizedError struct {
	// Message is what the engine said. It never distinguishes an unknown
	// credential from a malformed one, which would let a caller probe for
	// valid tokens.
	Message string
}

UnauthorizedError means credentials are missing or were not recognised.

func (*UnauthorizedError) Error 

func (e *UnauthorizedError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.