ondatrasql

README

OndatraSQL

The data runtime for teams that ship
Build pipelines without infrastructure.

One binary replaces ingestion, transformation, and orchestration.
No setup. No services. No stack.

Documentation · Discord · Blueprints

---

No Kafka. No Airflow. No dbt. No warehouse setup.

Just SQL files, one binary, and your data.

OndatraSQL is a data runtime that runs directly on DuckDB and DuckLake. You write SQL. OndatraSQL handles:

• Execution order (no DAGs to define)
• Change detection (no incremental logic)
• Schema evolution (no migrations)
• Validation and lineage (built-in)

Why

Most data tools assume you already have a stack. Kafka. Airflow. dbt. A warehouse.

OndatraSQL removes that assumption.

It runs on a single machine, requires no services, and works in minutes.

Install

curl -fsSL https://ondatra.sh/install.sh | sh

Windows: WSL2. From source: go install github.com/ondatra-labs/ondatrasql/cmd/ondatrasql@latest (Go 1.25+ and gcc/clang).

Quick Start

mkdir my-pipeline && cd my-pipeline
ondatrasql init

Create models/staging/orders.sql:

-- @kind: merge
-- @unique_key: order_id

SELECT * FROM (VALUES
    (1, 'Alice', 100, '2026-01-15'),
    (2, 'Bob',   200, '2026-02-20'),
    (3, 'Charlie', 150, '2026-03-10')
) AS t(order_id, customer, amount, order_date)

Run it:

ondatrasql run

[OK] staging.orders (merge, backfill, 3 rows, 180ms)

You now have a versioned table in DuckLake with automatic change tracking and a reproducible pipeline. No setup beyond this.

Principles

• No DAGs — dependencies are inferred from SQL
• No incremental logic — change detection is automatic
• No infrastructure — runs on one machine
• No blind runs — preview everything with sandbox

What You Don't Have to Do

No incremental logic — only changed data is processed automatically.

No migrations — schema updates itself when your query changes.

No blind runs — preview every change before committing.

No event infrastructure — POST events directly to an HTTP endpoint.

No separate tools for quality or lineage — built into execution.

Three Ways to Write Models

SQL — transformations:

-- @kind: table

SELECT date, SUM(total) AS revenue
FROM staging.orders GROUP BY date

Starlark — API ingestion (no Python):

# @kind: append

resp = http.get("https://api.example.com/users")
for user in resp.json:
    save.row(user)

YAML — declarative config:

kind: append
source: api_fetch
config:
  base_url: https://api.example.com

Mental Model

Files are tables. SQL is the pipeline. Runs are deterministic.

You don't build pipelines. You run data.

Compared to the Modern Data Stack

Traditional:

• dbt for transforms
• Airbyte for ingestion
• Airflow for orchestration
• Kafka for events
• Snowflake for storage

OndatraSQL:

• One binary
• One runtime
• One execution model

When Not to Use OndatraSQL

• You need a distributed system
• You process petabytes across clusters
• You require real-time streaming

OndatraSQL is designed for simplicity over horizontal scale.

Commands

run [model]          Run pipeline or specific model
sandbox [model]      Preview changes safely
daemon               Start event collection
sql "SELECT ..."     Query your data
lineage overview     See all dependencies

Full CLI reference →

Documentation

ondatra.sh

Philosophy

The best data system is the one that runs now.

Not the one you finish setting up next week.

License

GNU AGPL v3