README
¶
semantic-go
A small, dependency-light semantic layer compiler for Go: declare your business metrics, dimensions, entities, and join graph once in YAML, then compile a typed semantic query into fan-out / chasm-safe SQL for any dialect.
It's the contract between business meaning and SQL — the thing you point an LLM at instead of 1,200 raw tables, so "revenue" means one governed thing everywhere.
Pure stdlib + gopkg.in/yaml.v3. No database, no LLM, no network.
Why
Valid SQL ≠ correct SQL. Summing an order total after joining order line items multiplies it by the line count — a clean run, a green check, a silently wrong number. semantic-go makes that class of bug impossible by construction:
- Three nouns — every question is a metric (what), a dimension (how to slice), and an entity (who/what, with a primary key). Typed, so invalid combinations are refused, not hallucinated.
- Declared join graph — edges carry keys + cardinality. The compiler only traverses declared, many-to-one edges; a missing edge fails to compile rather than inventing a join. "A refused join is a feature."
- Aggregate-to-grain-then-join — each measure is aggregated to the requested grain in its own CTE, then the CTEs are outer-joined on shared dimensions. This neutralizes both fan-out and chasm traps.
Install
go get github.com/liliang-cn/semantic-go
Example
package main
import (
"fmt"
semantic "github.com/liliang-cn/semantic-go"
)
func main() {
m, err := semantic.LoadFile("model.yaml")
if err != nil {
panic(err)
}
compiled, err := semantic.Compile(m, semantic.Query{
Metrics: []string{"net_revenue"},
GroupBy: []string{"store_region"},
}, semantic.Postgres{})
if err != nil {
panic(err) // e.g. "cannot group net_revenue by product_category: not reachable"
}
fmt.Println(compiled.SQL) // run with compiled.Args against your warehouse
}
A model is plain YAML — entities (with keys), a join graph (edges with cardinality), dimensions, and metrics:
entities:
- {name: order, table: orders, primary_key: order_id}
- {name: order_item, table: order_items, primary_key: item_id}
joins:
- {from: order, to: order_item, cardinality: one_to_many, from_key: order_id, to_key: order_id}
dimensions:
- {name: store_region, entity: store, column: region, type: categorical}
metrics:
- name: total_revenue
entity: order_item
agg: sum
expr: "quantity * unit_price"
- name: net_revenue
formula: "total_revenue - refund_total" # derived
- name: revenue_cumulative
of: total_revenue
window: "cumulative" # time intelligence
Metric types
- simple —
entity+agg(sum/count/count_distinct/avg/…) +expr - ratio / derived —
formulaover other metric names (computed in the outer SELECT) - window / time intelligence —
of+window:rolling:N·cumulative·delta:N·prior:N(require a time dimension ingroup_by)
API
| Symbol | Purpose |
|---|---|
Load([]byte) / LoadFile(path) |
parse + validate a model |
Compile(*Model, Query, Dialect) (Compiled, error) |
semantic query → SQL + args |
Query{Metrics, GroupBy, Where, TimeGrain, OrderBy, Limit} |
the typed intent |
Postgres{} / ANSI{} |
dialects (Dialect interface) |
Model.DimensionsFor(metric) |
which dimensions a metric can legally be sliced by |
Model.MetricNames() / Metric() / Dimension() / Entity() |
catalog access |
Status
The compiler covers entities/dimensions/metrics (simple/ratio/derived/window), the many-to-one join graph, aggregate-then-join, and reachability-based refusal. Not yet built: bridge (many-to-many) tables, role-playing dimensions, grain-to-date period reset. Test coverage is light — contributions welcome.
License
MIT
Documentation
¶
Overview ¶
Package semantic is a dependency-light semantic layer for Go: a model (entities, dimensions, metrics, join graph) plus a compiler that turns a semantic Query — "this metric by these dimensions" — into fanout/chasm-safe SQL. It speaks no LLM, opens no database; it only produces SQL strings.
The core technique (the reason this exists): aggregate each measure to its base grain inside a CTE first, THEN join dimensions. That single move makes fan-out and chasm joins impossible by construction.
Index ¶
- Constants
- type ANSI
- type Compiled
- type Databricks
- type Dialect
- type Dimension
- type DuckDB
- type Entity
- type Filter
- type Issue
- type Join
- type Metric
- type Model
- func (m *Model) Additivity(name string) string
- func (m *Model) Dimension(name string) *Dimension
- func (m *Model) DimensionsFor(metric string) ([]string, error)
- func (m *Model) Entity(name string) *Entity
- func (m *Model) Index() error
- func (m *Model) Metric(name string) *Metric
- func (m *Model) MetricNames() []string
- func (m *Model) ReachableEntities(base string) map[string]bool
- type Postgres
- type Query
- type Snowflake
- type UnknownMetricError
Constants ¶
const ( Additive = "additive" SemiAdditive = "semi_additive" NonAdditive = "non_additive" )
Additivity classes.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ANSI ¶
type ANSI struct{}
ANSI is a portable fallback (SQLite/DuckDB-ish): ? placeholders, no date_trunc.
func (ANSI) DistinctFrom ¶
func (ANSI) Placeholder ¶
func (ANSI) QuoteIdent ¶
type Compiled ¶
Compiled is the output: SQL plus ordered bind arguments.
func Compile ¶
Compile turns a semantic Query into fanout/chasm-safe SQL for the dialect.
Algorithm: each base metric is aggregated to the requested dimension grain in its OWN CTE (joining only upward via many-to-one edges, so the leaf grain is never multiplied); the CTEs are then null-safe outer-joined on the shared dimensions and combined (derived formulas computed in the outer SELECT). This is what makes fan-out and chasm joins impossible by construction.
type Databricks ¶ added in v0.1.2
type Databricks struct{}
Databricks (Spark SQL) dialect: backtick-quoted identifiers, ? binds, and the null-safe equality operator <=> for outer joins.
func (Databricks) DateTrunc ¶ added in v0.1.2
func (Databricks) DateTrunc(grain, expr string) string
func (Databricks) DistinctFrom ¶ added in v0.1.2
func (Databricks) DistinctFrom(l, r string) string
func (Databricks) Name ¶ added in v0.1.2
func (Databricks) Name() string
func (Databricks) Placeholder ¶ added in v0.1.2
func (Databricks) Placeholder(int) string
func (Databricks) QuoteIdent ¶ added in v0.1.2
func (Databricks) QuoteIdent(id string) string
type Dialect ¶
type Dialect interface {
Name() string
QuoteIdent(string) string // quote a table/column identifier
DateTrunc(grain, expr string) string // truncate a date/timestamp to a grain
Placeholder(i int) string // bind placeholder for the i-th arg (1-based)
DistinctFrom(left, right string) string // null-safe equality for outer joins
}
Dialect captures the per-engine SQL differences the compiler needs. It only shapes SQL text — it never opens a connection. Postgres ships here; Snowflake/Databricks/DuckDB are added the same way.
func DialectByName ¶ added in v0.1.2
DialectByName resolves a dialect by its Name() (case-insensitive). The bool is false for an unknown name, so callers can fail loudly instead of guessing.
type Dimension ¶
type Dimension struct {
Name string `yaml:"name"`
Entity string `yaml:"entity"`
Column string `yaml:"column"`
Type string `yaml:"type"` // categorical | time
Mask string `yaml:"mask"`
}
Dimension is a typed attribute to group/filter by, named in business words. Mask is the SQL expression returned when the caller may not see the raw value.
type DuckDB ¶ added in v0.1.4
type DuckDB struct{}
DuckDB dialect: largely Postgres-compatible (double-quoted identifiers, positional $N binds, native DATE_TRUNC and IS NOT DISTINCT FROM) — its own type so routing and any future divergence stay explicit rather than falling back to the generic ANSI shape.
func (DuckDB) DistinctFrom ¶ added in v0.1.4
func (DuckDB) Placeholder ¶ added in v0.1.4
func (DuckDB) QuoteIdent ¶ added in v0.1.4
type Entity ¶
type Entity struct {
Name string `yaml:"name"`
Table string `yaml:"table"`
PrimaryKey string `yaml:"primary_key"`
}
Entity is a real business thing with a primary key the layer joins on — the key is declared, never guessed.
type Filter ¶
type Filter struct {
Dimension string `json:"dimension"`
Op string `json:"op"` // = | != | > | >= | < | <= | in
Values []any `json:"values"` // one value for scalar ops; many for "in"
}
Filter is one predicate against a dimension.
type Issue ¶ added in v0.1.1
type Issue struct {
Severity string // "error" | "warn"
Target string // metric/dimension name
Message string
}
Issue is one finding from Lint. Error issues should fail a CI gate; Warn issues are advisory (the model still compiles).
func Lint ¶ added in v0.1.1
Lint enforces the metadata contract a grounding agent relies on: every metric must describe what it means and offer at least one synonym to route to, and any roll-up that an agent could get wrong must be classified. Metadata is the agent's only map; an undescribed metric is an invitation to guess.
Returns issues in model order. Callers gate on len(errors) > 0.
func LintErrors ¶ added in v0.1.1
LintErrors returns only the error-severity issues from Lint.
type Join ¶
type Join struct {
From string `yaml:"from"`
To string `yaml:"to"`
FromKey string `yaml:"from_key"`
ToKey string `yaml:"to_key"`
Cardinality string `yaml:"cardinality"` // many_to_one | one_to_many | many_to_many
}
Join is one declared edge of the join graph: keys + cardinality. The compiler only ever traverses declared edges in the safe (many-to-one) direction; a missing edge is refused, never invented.
type Metric ¶
type Metric struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Synonyms []string `yaml:"synonyms"`
// base metric
Entity string `yaml:"entity"`
Agg string `yaml:"agg"` // sum | count | count_distinct | avg | min | max
Expr string `yaml:"expr"` // SQL expr at base grain, e.g. "quantity * unit_price"
// derived metric
Formula string `yaml:"formula"` // expression over metric names
// time-window metric: a transform of metric Of over the time dimension.
// Window is one of: rolling:N | cumulative | prior:N | delta:N
Of string `yaml:"of"`
Window string `yaml:"window"`
// Reset gives a window metric grain-to-date semantics: the accumulation
// restarts at each boundary of this period (e.g. reset: year → YTD). Empty
// means the window runs unbroken across the whole series.
// One of: day | week | month | quarter | year.
Reset string `yaml:"reset"`
// Additivity declares how the measure may be rolled up, so the compiler can
// refuse a roll-up that would produce a silent wrong number:
// additive — safe to sum across any dimension (revenue, units).
// semi_additive — summable across non-time dims only; over time use a
// point-in-time pick, never SUM (inventory, balances).
// non_additive — never summable (ratios, distinct counts).
// Empty means "infer from agg/formula" (see EffectiveAdditivity).
Additivity string `yaml:"additivity"`
// governance
Roles []string `yaml:"roles"` // if set, only these roles may resolve the metric
}
Metric is an aggregated number with grain + aggregation locked in. A simple metric aggregates Expr over its base Entity; a derived metric is a formula over other metric names (e.g. "total_revenue - refund_total") — which is how chasm traps are avoided: each base metric aggregates in its own CTE.
type Model ¶
type Model struct {
Entities []Entity `yaml:"entities"`
Joins []Join `yaml:"joins"`
Dimensions []Dimension `yaml:"dimensions"`
Metrics []Metric `yaml:"metrics"`
// contains filtered or unexported fields
}
Model is the single source of truth: business meaning compiled to SQL once.
func (*Model) Additivity ¶ added in v0.1.1
Additivity resolves how a metric may be rolled up, honoring an explicit `additivity:` and otherwise inferring it from the metric's shape:
- base: sum/count/min/max → additive; count_distinct/avg → non_additive.
- derived: a ratio formula (uses / or %) → non_additive; otherwise the least-additive of its parts (non_additive < semi_additive < additive).
- window: rolling/cumulative re-sum their input → non_additive; prior/delta are differences → non_additive. (A window result is never re-summable.)
func (*Model) DimensionsFor ¶
DimensionsFor returns the dimensions that can slice the metric WITHOUT a fanout — i.e. dimensions whose entity is reachable from every base metric the metric depends on. (This is why "net_revenue by product_category" is rejected: refunds can't reach product.) Powers the get_dimensions tool so agents self-correct.
func (*Model) MetricNames ¶
MetricNames returns metric names in definition order (for list_metrics tools).
type Postgres ¶
type Postgres struct{}
Postgres dialect.
func (Postgres) DistinctFrom ¶
func (Postgres) Placeholder ¶
func (Postgres) QuoteIdent ¶
type Query ¶
type Query struct {
Metrics []string // metric names to compute
GroupBy []string // dimension names to slice by
Where []Filter // filters expressed against dimensions, not raw columns
TimeGrain string // day | week | month | quarter | year — applied to time dimensions in GroupBy
OrderBy string // a metric or dimension name (presentation only)
Descending bool
Limit int
}
Query is a semantic query: pure intent, zero table names, zero join keywords. "These metrics, by these dimensions, filtered like so, at this time grain."
type Snowflake ¶ added in v0.1.2
type Snowflake struct{}
Snowflake dialect: double-quoted identifiers, positional :N binds, native DATE_TRUNC and IS NOT DISTINCT FROM.
func (Snowflake) DistinctFrom ¶ added in v0.1.2
func (Snowflake) Placeholder ¶ added in v0.1.2
func (Snowflake) QuoteIdent ¶ added in v0.1.2
type UnknownMetricError ¶
type UnknownMetricError struct{ Name string }
UnknownMetricError is returned when a metric name doesn't exist.
func (*UnknownMetricError) Error ¶
func (e *UnknownMetricError) Error() string
Source Files
Directories