semantic

README

semantic-go

A small, dependency-light semantic layer compiler for Go: declare your business metrics, dimensions, entities, and join graph once in YAML, then compile a typed semantic query into fan-out / chasm-safe SQL for any dialect.

It's the contract between business meaning and SQL — the thing you point an LLM at instead of 1,200 raw tables, so "revenue" means one governed thing everywhere.

Pure stdlib + gopkg.in/yaml.v3. No database, no LLM, no network.

Why

Valid SQL ≠ correct SQL. Summing an order total after joining order line items multiplies it by the line count — a clean run, a green check, a silently wrong number. semantic-go makes that class of bug impossible by construction:

• Three nouns — every question is a metric (what), a dimension (how to slice), and an entity (who/what, with a primary key). Typed, so invalid combinations are refused, not hallucinated.
• Declared join graph — edges carry keys + cardinality. The compiler only traverses declared, many-to-one edges; a missing edge fails to compile rather than inventing a join. "A refused join is a feature."
• Aggregate-to-grain-then-join — each measure is aggregated to the requested grain in its own CTE, then the CTEs are outer-joined on shared dimensions. This neutralizes both fan-out and chasm traps.

Install

go get github.com/liliang-cn/semantic-go

Example

package main

import (
	"fmt"

	semantic "github.com/liliang-cn/semantic-go"
)

func main() {
	m, err := semantic.LoadFile("model.yaml")
	if err != nil {
		panic(err)
	}
	compiled, err := semantic.Compile(m, semantic.Query{
		Metrics: []string{"net_revenue"},
		GroupBy: []string{"store_region"},
	}, semantic.Postgres{})
	if err != nil {
		panic(err) // e.g. "cannot group net_revenue by product_category: not reachable"
	}
	fmt.Println(compiled.SQL)  // run with compiled.Args against your warehouse
}

A model is plain YAML — entities (with keys), a join graph (edges with cardinality), dimensions, and metrics:

entities:
  - {name: order,      table: orders,      primary_key: order_id}
  - {name: order_item, table: order_items, primary_key: item_id}

joins:
  - {from: order, to: order_item, cardinality: one_to_many, from_key: order_id, to_key: order_id}

dimensions:
  - {name: store_region, entity: store, column: region, type: categorical}

metrics:
  - name: total_revenue
    entity: order_item
    agg: sum
    expr: "quantity * unit_price"
  - name: net_revenue
    formula: "total_revenue - refund_total"   # derived
  - name: revenue_cumulative
    of: total_revenue
    window: "cumulative"                       # time intelligence

Metric types

• simple — entity + agg (sum/count/count_distinct/avg/…) + expr
• ratio / derived — formula over other metric names (computed in the outer SELECT)
• window / time intelligence — of + window: rolling:N · cumulative · delta:N · prior:N (require a time dimension in group_by)

API

Symbol	Purpose
Load([]byte) / LoadFile(path)	parse + validate a model
Compile(*Model, Query, Dialect) (Compiled, error)	semantic query → SQL + args
Query{Metrics, GroupBy, Where, TimeGrain, OrderBy, Limit}	the typed intent
Postgres{} / ANSI{}	dialects (Dialect interface)
Model.DimensionsFor(metric)	which dimensions a metric can legally be sliced by
Model.MetricNames() / Metric() / Dimension() / Entity()	catalog access

Status

The compiler covers entities/dimensions/metrics (simple/ratio/derived/window), the many-to-one join graph, aggregate-then-join, and reachability-based refusal. Not yet built: bridge (many-to-many) tables, role-playing dimensions, grain-to-date period reset. Test coverage is light — contributions welcome.

License

MIT

Documentation

Overview

Package semantic is a dependency-light semantic layer for Go: a model (entities, dimensions, metrics, join graph) plus a compiler that turns a semantic Query — "this metric by these dimensions" — into fanout/chasm-safe SQL. It speaks no LLM, opens no database; it only produces SQL strings.

The core technique (the reason this exists): aggregate each measure to its base grain inside a CTE first, THEN join dimensions. That single move makes fan-out and chasm joins impossible by construction.

Index

• type ANSI
  • func (ANSI) DateTrunc(grain, expr string) string
  • func (ANSI) DistinctFrom(l, r string) string
  • func (ANSI) Name() string
  • func (ANSI) Placeholder(int) string
  • func (ANSI) QuoteIdent(id string) string
• type Compiled
  • func Compile(m *Model, q Query, d Dialect) (Compiled, error)
• type Dialect
• type Dimension
• type Entity
• type Filter
• type Join
• type Metric
  • func (m *Metric) IsDerived() bool
  • func (m *Metric) IsWindow() bool
• type Model
  • func Load(data []byte) (*Model, error)
  • func LoadFile(path string) (*Model, error)
  • func (m *Model) Dimension(name string) *Dimension
  • func (m *Model) DimensionsFor(metric string) ([]string, error)
  • func (m *Model) Entity(name string) *Entity
  • func (m *Model) Index() error
  • func (m *Model) Metric(name string) *Metric
  • func (m *Model) MetricNames() []string
  • func (m *Model) ReachableEntities(base string) map[string]bool
• type Postgres
  • func (Postgres) DateTrunc(grain, expr string) string
  • func (Postgres) DistinctFrom(l, r string) string
  • func (Postgres) Name() string
  • func (Postgres) Placeholder(i int) string
  • func (Postgres) QuoteIdent(id string) string
• type Query
• type UnknownMetricError
  • func (e *UnknownMetricError) Error() string

Types

type ANSI

type ANSI struct{}

ANSI is a portable fallback (SQLite/DuckDB-ish): ? placeholders, no date_trunc.

func (ANSI) DateTrunc

func (ANSI) DateTrunc(grain, expr string) string

func (ANSI) DistinctFrom

func (ANSI) DistinctFrom(l, r string) string

func (ANSI) Name

func (ANSI) Name() string

func (ANSI) Placeholder

func (ANSI) Placeholder(int) string

func (ANSI) QuoteIdent

func (ANSI) QuoteIdent(id string) string

type Compiled

type Compiled struct {
	SQL  string
	Args []any
}

Compiled is the output: SQL plus ordered bind arguments.

func Compile

func Compile(m *Model, q Query, d Dialect) (Compiled, error)

Compile turns a semantic Query into fanout/chasm-safe SQL for the dialect.

Algorithm: each base metric is aggregated to the requested dimension grain in its OWN CTE (joining only upward via many-to-one edges, so the leaf grain is never multiplied); the CTEs are then null-safe outer-joined on the shared dimensions and combined (derived formulas computed in the outer SELECT). This is what makes fan-out and chasm joins impossible by construction.

type Dialect

type Dialect interface {
	Name() string
	QuoteIdent(string) string               // quote a table/column identifier
	DateTrunc(grain, expr string) string    // truncate a date/timestamp to a grain
	Placeholder(i int) string               // bind placeholder for the i-th arg (1-based)
	DistinctFrom(left, right string) string // null-safe equality for outer joins
}

Dialect captures the per-engine SQL differences the compiler needs. It only shapes SQL text — it never opens a connection. Postgres ships here; Snowflake/Databricks/DuckDB are added the same way.

type Dimension

type Dimension struct {
	Name   string `yaml:"name"`
	Entity string `yaml:"entity"`
	Column string `yaml:"column"`
	Type   string `yaml:"type"` // categorical | time
	Mask   string `yaml:"mask"`
}

Dimension is a typed attribute to group/filter by, named in business words. Mask is the SQL expression returned when the caller may not see the raw value.

type Entity

type Entity struct {
	Name       string `yaml:"name"`
	Table      string `yaml:"table"`
	PrimaryKey string `yaml:"primary_key"`
}

Entity is a real business thing with a primary key the layer joins on — the key is declared, never guessed.

type Filter

type Filter struct {
	Dimension string `json:"dimension"`
	Op        string `json:"op"`     // = | != | > | >= | < | <= | in
	Values    []any  `json:"values"` // one value for scalar ops; many for "in"
}

Filter is one predicate against a dimension.

type Join

type Join struct {
	From        string `yaml:"from"`
	To          string `yaml:"to"`
	FromKey     string `yaml:"from_key"`
	ToKey       string `yaml:"to_key"`
	Cardinality string `yaml:"cardinality"` // many_to_one | one_to_many | many_to_many
}

Join is one declared edge of the join graph: keys + cardinality. The compiler only ever traverses declared edges in the safe (many-to-one) direction; a missing edge is refused, never invented.

type Metric

type Metric struct {
	Name        string   `yaml:"name"`
	Description string   `yaml:"description"`
	Synonyms    []string `yaml:"synonyms"`

	// base metric
	Entity string `yaml:"entity"`
	Agg    string `yaml:"agg"`  // sum | count | count_distinct | avg | min | max
	Expr   string `yaml:"expr"` // SQL expr at base grain, e.g. "quantity * unit_price"

	// derived metric
	Formula string `yaml:"formula"` // expression over metric names

	// time-window metric: a transform of metric Of over the time dimension.
	// Window is one of: rolling:N | cumulative | prior:N | delta:N
	Of     string `yaml:"of"`
	Window string `yaml:"window"`

	// governance
	Roles []string `yaml:"roles"` // if set, only these roles may resolve the metric
}

Metric is an aggregated number with grain + aggregation locked in. A simple metric aggregates Expr over its base Entity; a derived metric is a formula over other metric names (e.g. "total_revenue - refund_total") — which is how chasm traps are avoided: each base metric aggregates in its own CTE.

func (*Metric) IsDerived

func (m *Metric) IsDerived() bool

func (*Metric) IsWindow

func (m *Metric) IsWindow() bool

type Model

type Model struct {
	Entities   []Entity    `yaml:"entities"`
	Joins      []Join      `yaml:"joins"`
	Dimensions []Dimension `yaml:"dimensions"`
	Metrics    []Metric    `yaml:"metrics"`
	// contains filtered or unexported fields
}

Model is the single source of truth: business meaning compiled to SQL once.

func Load

func Load(data []byte) (*Model, error)

Load parses a YAML model and validates it.

func LoadFile

func LoadFile(path string) (*Model, error)

LoadFile reads and parses a YAML model from disk.

func (*Model) Dimension

func (m *Model) Dimension(name string) *Dimension

func (*Model) DimensionsFor

func (m *Model) DimensionsFor(metric string) ([]string, error)

DimensionsFor returns the dimensions that can slice the metric WITHOUT a fanout — i.e. dimensions whose entity is reachable from every base metric the metric depends on. (This is why "net_revenue by product_category" is rejected: refunds can't reach product.) Powers the get_dimensions tool so agents self-correct.

func (*Model) Entity

func (m *Model) Entity(name string) *Entity

func (*Model) Index

func (m *Model) Index() error

Index builds lookups and validates references. Must be called after loading.

func (*Model) Metric

func (m *Model) Metric(name string) *Metric

func (*Model) MetricNames

func (m *Model) MetricNames() []string

MetricNames returns metric names in definition order (for list_metrics tools).

func (*Model) ReachableEntities

func (m *Model) ReachableEntities(base string) map[string]bool

ReachableEntities returns the set of entities reachable from base by following only many-to-one edges (the joins that don't fan out the base grain).

type Postgres

type Postgres struct{}

Postgres dialect.

func (Postgres) DateTrunc

func (Postgres) DateTrunc(grain, expr string) string

func (Postgres) DistinctFrom

func (Postgres) DistinctFrom(l, r string) string

func (Postgres) Name

func (Postgres) Name() string

func (Postgres) Placeholder

func (Postgres) Placeholder(i int) string

func (Postgres) QuoteIdent

func (Postgres) QuoteIdent(id string) string

type Query

type Query struct {
	Metrics    []string // metric names to compute
	GroupBy    []string // dimension names to slice by
	Where      []Filter // filters expressed against dimensions, not raw columns
	TimeGrain  string   // day | week | month | quarter | year — applied to time dimensions in GroupBy
	OrderBy    string   // a metric or dimension name (presentation only)
	Descending bool
	Limit      int
}

Query is a semantic query: pure intent, zero table names, zero join keywords. "These metrics, by these dimensions, filtered like so, at this time grain."

type UnknownMetricError

type UnknownMetricError struct{ Name string }

UnknownMetricError is returned when a metric name doesn't exist.

func (*UnknownMetricError) Error

func (e *UnknownMetricError) Error() string