README
¶
semantic-go
A small, dependency-light semantic layer compiler for Go: declare your business metrics, dimensions, entities, and join graph once in YAML, then compile a typed semantic query into fan-out / chasm-safe SQL for any dialect.
It's the contract between business meaning and SQL — the thing you point an LLM at instead of 1,200 raw tables, so "revenue" means one governed thing everywhere.
Pure stdlib + gopkg.in/yaml.v3. No database, no LLM, no network.
Why
Valid SQL ≠ correct SQL. Summing an order total after joining order line items multiplies it by the line count — a clean run, a green check, a silently wrong number. semantic-go makes that class of bug impossible by construction:
- Three nouns — every question is a metric (what), a dimension (how to slice), and an entity (who/what, with a primary key). Typed, so invalid combinations are refused, not hallucinated.
- Declared join graph — edges carry keys + cardinality. The compiler only traverses declared, many-to-one edges; a missing edge fails to compile rather than inventing a join. "A refused join is a feature."
- Aggregate-to-grain-then-join — each measure is aggregated to the requested grain in its own CTE, then the CTEs are outer-joined on shared dimensions. This neutralizes both fan-out and chasm traps.
Install
go get github.com/liliang-cn/semantic-go
Example
package main
import (
"fmt"
semantic "github.com/liliang-cn/semantic-go"
)
func main() {
m, err := semantic.LoadFile("model.yaml")
if err != nil {
panic(err)
}
compiled, err := semantic.Compile(m, semantic.Query{
Metrics: []string{"net_revenue"},
GroupBy: []string{"store_region"},
}, semantic.Postgres{})
if err != nil {
panic(err) // e.g. "cannot group net_revenue by product_category: not reachable"
}
fmt.Println(compiled.SQL) // run with compiled.Args against your warehouse
}
A model is plain YAML — entities (with keys), a join graph (edges with cardinality), dimensions, and metrics:
entities:
- {name: order, table: orders, primary_key: order_id}
- {name: order_item, table: order_items, primary_key: item_id}
joins:
- {from: order, to: order_item, cardinality: one_to_many, from_key: order_id, to_key: order_id}
dimensions:
- {name: store_region, entity: store, column: region, type: categorical}
metrics:
- name: total_revenue
entity: order_item
agg: sum
expr: "quantity * unit_price"
- name: net_revenue
formula: "total_revenue - refund_total" # derived
- name: revenue_cumulative
of: total_revenue
window: "cumulative" # time intelligence
CLI: semc
semc compiles a semantic query to SQL and prints it. It opens no database —
it only emits SQL (pipe it into psql to run).
go install github.com/liliang-cn/semantic-go/cmd/semc@latest
Flags:
-model— path to the semantic model YAML (defaulttestdata/meridian.yaml)-metrics— comma-separated metric names (required)-by— comma-separated group-by dimensions-grain— time grain for time dimensions (day|month|quarter|year)-limit— row limit (0 = none)
# revenue by region
semc -model testdata/meridian.yaml -metrics total_revenue -by store_region
# monthly net revenue, last 12 rows
semc -model testdata/meridian.yaml -metrics net_revenue -by order_date -grain month -limit 12
Compiled query args, if any, are printed to stderr as a -- args: comment.
Metric types
- simple —
entity+agg(sum/count/count_distinct/avg/…) +expr - ratio / derived —
formulaover other metric names (computed in the outer SELECT) - window / time intelligence —
of+window:rolling:N·cumulative·delta:N·prior:N(require a time dimension ingroup_by)
API
| Symbol | Purpose |
|---|---|
Load([]byte) / LoadFile(path) |
parse + validate a model |
Compile(*Model, Query, Dialect) (Compiled, error) |
semantic query → SQL + args |
Query{Metrics, GroupBy, Where, TimeGrain, OrderBy, Limit} |
the typed intent |
Postgres{} / ANSI{} |
dialects (Dialect interface) |
Model.DimensionsFor(metric) |
which dimensions a metric can legally be sliced by |
Model.MetricNames() / Metric() / Dimension() / Entity() |
catalog access |
Status
The compiler covers entities/dimensions/metrics (simple/ratio/derived/window), the many-to-one join graph, aggregate-then-join, and reachability-based refusal. Not yet built: bridge (many-to-many) tables, role-playing dimensions, grain-to-date period reset. Test coverage is light — contributions welcome.
License
MIT
Documentation
¶
Overview ¶
Package semantic is a dependency-light semantic layer for Go: a model (entities, dimensions, metrics, join graph) plus a compiler that turns a semantic Query — "this metric by these dimensions" — into fanout/chasm-safe SQL. It speaks no LLM, opens no database; it only produces SQL strings.
The core technique (the reason this exists): aggregate each measure to its base grain inside a CTE first, THEN join dimensions. That single move makes fan-out and chasm joins impossible by construction.
Index ¶
- Constants
- type ANSI
- type Compiled
- type Databricks
- type Dialect
- type Dimension
- type DuckDB
- type Entity
- type Filter
- type Issue
- type Join
- type Metric
- type Model
- func (m *Model) Additivity(name string) string
- func (m *Model) Dimension(name string) *Dimension
- func (m *Model) DimensionsFor(metric string) ([]string, error)
- func (m *Model) Entity(name string) *Entity
- func (m *Model) Index() error
- func (m *Model) Metric(name string) *Metric
- func (m *Model) MetricNames() []string
- func (m *Model) ReachableEntities(base string) map[string]bool
- func (m *Model) ResolveDimensionName(name string) (string, bool)
- func (m *Model) ResolveGroupBy(q *Query)
- func (m *Model) ResolveMetricName(name string) (string, bool)
- func (m *Model) ResolveMetrics(q *Query) error
- func (m *Model) SuggestMetricNames(name string, n int) []string
- type MySQL
- type Postgres
- type Query
- type SQLServer
- type SQLite
- type Snowflake
- type UnknownMetricError
Constants ¶
const ( Additive = "additive" SemiAdditive = "semi_additive" NonAdditive = "non_additive" )
Additivity classes.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ANSI ¶
type ANSI struct{}
ANSI is a portable fallback (SQLite/DuckDB-ish): ? placeholders, no date_trunc.
func (ANSI) DistinctFrom ¶
func (ANSI) Placeholder ¶
func (ANSI) QuoteIdent ¶
type Compiled ¶
Compiled is the output: SQL plus ordered bind arguments.
func Compile ¶
Compile turns a semantic Query into fanout/chasm-safe SQL for the dialect.
Algorithm: each base metric is aggregated to the requested dimension grain in its OWN CTE (joining only upward via many-to-one edges, so the leaf grain is never multiplied); the CTEs are then null-safe outer-joined on the shared dimensions and combined (derived formulas computed in the outer SELECT). This is what makes fan-out and chasm joins impossible by construction.
type Databricks ¶ added in v0.1.2
type Databricks struct{}
Databricks (Spark SQL) dialect: backtick-quoted identifiers, ? binds, and the null-safe equality operator <=> for outer joins.
func (Databricks) DateTrunc ¶ added in v0.1.2
func (Databricks) DateTrunc(grain, expr string) string
func (Databricks) DistinctFrom ¶ added in v0.1.2
func (Databricks) DistinctFrom(l, r string) string
func (Databricks) Name ¶ added in v0.1.2
func (Databricks) Name() string
func (Databricks) Placeholder ¶ added in v0.1.2
func (Databricks) Placeholder(int) string
func (Databricks) QuoteIdent ¶ added in v0.1.2
func (Databricks) QuoteIdent(id string) string
type Dialect ¶
type Dialect interface {
Name() string
QuoteIdent(string) string // quote a table/column identifier
DateTrunc(grain, expr string) string // truncate a date/timestamp to a grain
Placeholder(i int) string // bind placeholder for the i-th arg (1-based)
DistinctFrom(left, right string) string // null-safe equality for outer joins
}
Dialect captures the per-engine SQL differences the compiler needs. It only shapes SQL text — it never opens a connection. Postgres ships here; Snowflake/Databricks/DuckDB are added the same way.
func DialectByName ¶ added in v0.1.2
DialectByName resolves a dialect by its Name() (case-insensitive). The bool is false for an unknown name, so callers can fail loudly instead of guessing.
type Dimension ¶
type Dimension struct {
Name string `yaml:"name"`
Entity string `yaml:"entity"`
Column string `yaml:"column"`
Type string `yaml:"type"` // categorical | time
Synonyms []string `yaml:"synonyms"`
Mask string `yaml:"mask"`
}
Dimension is a typed attribute to group/filter by, named in business words. Mask is the SQL expression returned when the caller may not see the raw value.
type DuckDB ¶ added in v0.1.4
type DuckDB struct{}
DuckDB dialect: largely Postgres-compatible (double-quoted identifiers, positional $N binds, native DATE_TRUNC and IS NOT DISTINCT FROM) — its own type so routing and any future divergence stay explicit rather than falling back to the generic ANSI shape.
func (DuckDB) DistinctFrom ¶ added in v0.1.4
func (DuckDB) Placeholder ¶ added in v0.1.4
func (DuckDB) QuoteIdent ¶ added in v0.1.4
type Entity ¶
type Entity struct {
Name string `yaml:"name"`
Table string `yaml:"table"`
PrimaryKey string `yaml:"primary_key"`
}
Entity is a real business thing with a primary key the layer joins on — the key is declared, never guessed.
type Filter ¶
type Filter struct {
Dimension string `json:"dimension"`
Op string `json:"op"` // = | != | > | >= | < | <= | in
Values []any `json:"values"` // one value for scalar ops; many for "in"
}
Filter is one predicate against a dimension.
type Issue ¶ added in v0.1.1
type Issue struct {
Severity string // "error" | "warn"
Target string // metric/dimension name
Message string
}
Issue is one finding from Lint. Error issues should fail a CI gate; Warn issues are advisory (the model still compiles).
func Lint ¶ added in v0.1.1
Lint enforces the metadata contract a grounding agent relies on: every metric must describe what it means and offer at least one synonym to route to, and any roll-up that an agent could get wrong must be classified. Metadata is the agent's only map; an undescribed metric is an invitation to guess.
Returns issues in model order. Callers gate on len(errors) > 0.
func LintErrors ¶ added in v0.1.1
LintErrors returns only the error-severity issues from Lint.
type Join ¶
type Join struct {
From string `yaml:"from"`
To string `yaml:"to"`
FromKey string `yaml:"from_key"`
ToKey string `yaml:"to_key"`
Cardinality string `yaml:"cardinality"` // many_to_one | one_to_many | many_to_many
}
Join is one declared edge of the join graph: keys + cardinality. The compiler only ever traverses declared edges in the safe (many-to-one) direction; a missing edge is refused, never invented.
type Metric ¶
type Metric struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Synonyms []string `yaml:"synonyms"`
// base metric
Entity string `yaml:"entity"`
Agg string `yaml:"agg"` // sum | count | count_distinct | avg | min | max
Expr string `yaml:"expr"` // SQL expr at base grain, e.g. "quantity * unit_price"
// derived metric
Formula string `yaml:"formula"` // expression over metric names
// time-window metric: a transform of metric Of over the time dimension.
// Window is one of: rolling:N | cumulative | prior:N | delta:N
Of string `yaml:"of"`
Window string `yaml:"window"`
// Reset gives a window metric grain-to-date semantics: the accumulation
// restarts at each boundary of this period (e.g. reset: year → YTD). Empty
// means the window runs unbroken across the whole series.
// One of: day | week | month | quarter | year.
Reset string `yaml:"reset"`
// Additivity declares how the measure may be rolled up, so the compiler can
// refuse a roll-up that would produce a silent wrong number:
// additive — safe to sum across any dimension (revenue, units).
// semi_additive — summable across non-time dims only; over time use a
// point-in-time pick, never SUM (inventory, balances).
// non_additive — never summable (ratios, distinct counts).
// Empty means "infer from agg/formula" (see EffectiveAdditivity).
Additivity string `yaml:"additivity"`
// governance
Roles []string `yaml:"roles"` // if set, only these roles may resolve the metric
}
Metric is an aggregated number with grain + aggregation locked in. A simple metric aggregates Expr over its base Entity; a derived metric is a formula over other metric names (e.g. "total_revenue - refund_total") — which is how chasm traps are avoided: each base metric aggregates in its own CTE.
type Model ¶
type Model struct {
Entities []Entity `yaml:"entities"`
Joins []Join `yaml:"joins"`
Dimensions []Dimension `yaml:"dimensions"`
Metrics []Metric `yaml:"metrics"`
// contains filtered or unexported fields
}
Model is the single source of truth: business meaning compiled to SQL once.
func (*Model) Additivity ¶ added in v0.1.1
Additivity resolves how a metric may be rolled up, honoring an explicit `additivity:` and otherwise inferring it from the metric's shape:
- base: sum/count/min/max → additive; count_distinct/avg → non_additive.
- derived: a ratio formula (uses / or %) → non_additive; otherwise the least-additive of its parts (non_additive < semi_additive < additive).
- window: rolling/cumulative re-sum their input → non_additive; prior/delta are differences → non_additive. (A window result is never re-summable.)
func (*Model) DimensionsFor ¶
DimensionsFor returns the dimensions that can slice the metric WITHOUT a fanout — i.e. dimensions whose entity is reachable from every base metric the metric depends on. (This is why "net_revenue by product_category" is rejected: refunds can't reach product.) Powers the get_dimensions tool so agents self-correct.
func (*Model) MetricNames ¶
MetricNames returns metric names in definition order (for list_metrics tools).
func (*Model) ReachableEntities ¶
ReachableEntities returns the set of entities reachable from base by following only many-to-one edges (the joins that don't fan out the base grain).
func (*Model) ResolveDimensionName ¶ added in v0.1.5
ResolveDimensionName maps a dimension name or a declared synonym to its canonical dimension name. An exact canonical name resolves to itself; ok is false when nothing matches.
func (*Model) ResolveGroupBy ¶ added in v0.1.5
ResolveGroupBy rewrites q.GroupBy from dimension synonyms to canonical names in place (canonical names pass through unchanged). Unknown names are left as is so the compiler reports them with its own dimension diagnostics. A fresh slice is allocated, so the caller's original is not mutated.
func (*Model) ResolveMetricName ¶ added in v0.1.5
ResolveMetricName maps a metric name or a declared synonym to its canonical metric name. An exact canonical name always resolves to itself unchanged; otherwise matching is case-insensitive over names and synonyms. ok is false when nothing matches.
func (*Model) ResolveMetrics ¶ added in v0.1.5
ResolveMetrics rewrites q.Metrics from synonyms to canonical names in place (canonical names pass through unchanged). An unknown name returns an error naming the closest known metrics. Dimensions are untouched — they carry no synonyms. A fresh slice is allocated, so the caller's original is not mutated.
func (*Model) SuggestMetricNames ¶ added in v0.1.5
SuggestMetricNames returns up to n canonical metric names whose name or synonyms are closest to the given (unknown) name — for a helpful error. Substring matches rank first, then smallest edit distance.
type MySQL ¶ added in v0.1.6
type MySQL struct{}
MySQL dialect (also MariaDB): backtick identifiers, ? binds, and the null-safe equality operator <=>.
MySQL has no DATE_TRUNC — not in 8.x, not in MariaDB — so each grain is emulated with date arithmetic that returns a DATE. Returning a DATE rather than a formatted string matters: a bucket label has to sort chronologically and compare against date literals in a WHERE clause, and '2024-10' as text does neither.
func (MySQL) DistinctFrom ¶ added in v0.1.6
func (MySQL) Placeholder ¶ added in v0.1.6
func (MySQL) QuoteIdent ¶ added in v0.1.6
type Postgres ¶
type Postgres struct{}
Postgres dialect.
func (Postgres) DistinctFrom ¶
func (Postgres) Placeholder ¶
func (Postgres) QuoteIdent ¶
type Query ¶
type Query struct {
Metrics []string // metric names to compute
GroupBy []string // dimension names to slice by
Where []Filter // filters expressed against dimensions, not raw columns
TimeGrain string // day | week | month | quarter | year — applied to time dimensions in GroupBy
OrderBy string // a metric or dimension name (presentation only)
Descending bool
Limit int
}
Query is a semantic query: pure intent, zero table names, zero join keywords. "These metrics, by these dimensions, filtered like so, at this time grain."
type SQLServer ¶ added in v0.1.6
type SQLServer struct{}
SQLServer (T-SQL) dialect: bracketed identifiers, @pN binds.
DATETRUNC exists only from SQL Server 2022, so truncation uses the DATEADD/DATEDIFF idiom that every supported version understands. Its epoch (0 = 1900-01-01) is a Monday, so the week grain agrees with Postgres. IS NOT DISTINCT FROM is likewise 2022-only, hence the explicit null pair.
func (SQLServer) DistinctFrom ¶ added in v0.1.6
func (SQLServer) Placeholder ¶ added in v0.1.6
func (SQLServer) QuoteIdent ¶ added in v0.1.6
type SQLite ¶ added in v0.1.6
type SQLite struct{}
SQLite dialect: double-quoted identifiers, ? binds, and IS as null-safe equality.
SQLite has no date_trunc and no date type — dates are text, and the modifiers on date() are the only truncation available. Routing "sqlite" to the generic ANSI dialect (as this package once did) emitted date_trunc('month', …) against an engine with no such function, so every time-grained query failed at execution. Buckets come back as 'YYYY-MM-DD' text, which is the one text date format that sorts chronologically.
func (SQLite) DistinctFrom ¶ added in v0.1.6
DistinctFrom uses SQLite's IS, which compares NULLs as equal.
func (SQLite) Placeholder ¶ added in v0.1.6
func (SQLite) QuoteIdent ¶ added in v0.1.6
type Snowflake ¶ added in v0.1.2
type Snowflake struct{}
Snowflake dialect: double-quoted identifiers, positional :N binds, native DATE_TRUNC and IS NOT DISTINCT FROM.
func (Snowflake) DistinctFrom ¶ added in v0.1.2
func (Snowflake) Placeholder ¶ added in v0.1.2
func (Snowflake) QuoteIdent ¶ added in v0.1.2
type UnknownMetricError ¶
type UnknownMetricError struct{ Name string }
UnknownMetricError is returned when a metric name doesn't exist.
func (*UnknownMetricError) Error ¶
func (e *UnknownMetricError) Error() string
Source Files
Directories