tq

README

docs

Documentation

Overview

Package tq is the canonical engine of tq — the query language of the tsvsheet ecosystem: a `|`-separated pipeline of relational verbs (select, drop, where, derive, rename, sort, distinct, limit, offset, group) over a TSV or .tsvt table, with every embedded expression written in the tsvsheet formula language. TSV in, TSV out; between cut and jq.

Parse turns query text into an immutable Program; NewProgram and the per-verb stage constructors (Select, Where, Derive, …) build the identical Program programmatically. Program.Run plans the pipeline against the input table's header (or arity, headerless) — resolving every column reference, rejecting raw A1 references, and compiling each stage expression once through go-tsvsheet's expression seam — then executes the verbs over the table model. ReadTable and WriteTable delegate grid I/O (comment semantics included) to go-tsvsheet, so the two engines never disagree; when the input holds formula cells and IsRaw is unset, Run computes the sheet first and queries the computed values (ADR 0003).

The engine is filesystem- and network-free by construction: the compute pass's clock, limits, sheet loader, and import fetcher arrive injected via Options. Errors returned to callers are the errs.Const sentinels re-exported from errors.go — the closed program-error set of SPECIFICATION §8 — matchable with errors.Is; expression evaluation failures are error values in the data, exactly as in a sheet.

Example

Example runs the canonical pipeline from SPECIFICATION §3: filter, derive, project, sort, and limit — TSV in, TSV out.

package main

import (
	"os"
	"strings"

	tq "github.com/tsvsheet/go-tq"
)

func main() {
	input := "name\tstars\tforks\n" +
		"alpha\t1500\t3\n" +
		"beta\t900\t2\n" +
		"gamma\t2000\t8\n"

	program, err := tq.Parse(
		"where [stars] > 1000 | derive ratio = round([stars] / [forks], 2) | select name, ratio | sort -ratio",
	)
	if err != nil {
		panic(err)
	}
	table, err := tq.ReadTable(strings.NewReader(input), tq.Options{})
	if err != nil {
		panic(err)
	}
	out, err := program.Run(table, tq.Options{})
	if err != nil {
		panic(err)
	}
	if err := tq.WriteTable(os.Stdout, out); err != nil {
		panic(err)
	}
}

Output:
name	ratio
alpha	500
gamma	250

Index

• Constants
• func WriteTable(w io.Writer, t Table) error
• type Assignment
• type Column
• type ColumnName
• type Expression
• type Options
• type Program
  • func NewProgram(stages ...Stage) Program
  • func Parse(q Query) (Program, error)
  • func (p Program) Run(t Table, opts Options) (Table, error)
• type Query
• type RenamePair
• type RowCount
• type SortKey
• type Stage
  • func Derive(assignments ...Assignment) Stage
  • func Distinct(cols ...Column) Stage
  • func Drop(cols ...Column) Stage
  • func GroupBy(keys []Column, aggs ...Assignment) Stage
  • func Limit(n RowCount) Stage
  • func Offset(n RowCount) Stage
  • func Rename(pairs ...RenamePair) Stage
  • func Select(cols ...Column) Stage
  • func SortBy(keys ...SortKey) Stage
  • func Where(expr Expression) Stage
• type Table
  • func ReadTable(r io.Reader, opts Options) (Table, error)

Examples

• Package

Constants

const (
	// ErrSyntax: the program does not parse (line/column attached) — including
	// a fractional limit/offset and an embedded-expression failure at compile.
	ErrSyntax = constants.ErrSyntax
	// ErrUnknownColumn: plan-time — a reference names no column or an index is
	// out of range.
	ErrUnknownColumn = constants.ErrUnknownColumn
	// ErrCellRef: plan-time — an expression contains a raw A1 reference or
	// sheet qualifier; tq addresses columns, never cells.
	ErrCellRef = constants.ErrCellRef
	// ErrHeaderless: plan-time — rename (or another name-requiring form) in
	// headerless mode (IsHeaderless).
	ErrHeaderless = constants.ErrHeaderless
	// ErrStrict: run-time under IsStrict — an expression produced an error
	// value, named with its column and 1-based input row.
	ErrStrict = constants.ErrStrict
	// ErrLimit: the input exceeded Options.MaxCells (raised by ReadTable).
	ErrLimit = constants.ErrLimit
)

The closed program-error set (SPECIFICATION §8), matchable with errors.Is. Each sentinel is distinct from go-tsvsheet's; an embedded-expression compile failure is tq's ErrSyntax with the go-tsvsheet detail attached via With, so errors.Is reaches both. Everything else — coercion surprises, lookup misses, division by zero — is an error value in the data, never a Go error.

Functions

func WriteTable

func WriteTable(w io.Writer, t Table) error

WriteTable writes the table as TSV — header row first when present, each row newline-terminated — via go-tsvsheet's grid writer. A write failure passes through as go-tsvsheet's own sentinel.

Types

type Assignment

type Assignment struct {
	Name ColumnName
	Expr Expression
}

Assignment is one `name = expr` (derive or group aggregate).

type Column

type Column string

Column is a stage-position column reference, written as it would appear in a program: a bare header name (`stars`), a bare 1-based index (`2`), or the bracketed form (`[full title]`, `[2]`) — brackets are stripped, and digits-only content is an index (SPECIFICATION §4). Unlike the parser, the builder accepts any name unbracketed.

type ColumnName

type ColumnName string

ColumnName is a column name being introduced — a derive target, a rename target, or a group aggregate name — taken verbatim (digits-only text introduces a digits-only name, never an index).

type Expression

type Expression string

Expression is one tq expression in tsvsheet formula syntax (the pipe sugar excluded — `|` belongs to stages), exactly as it would appear in a program. A builder expression is validated at Run's plan step (ErrSyntax).

type Options

type Options struct {
	// At is the compute-pass clock volatile functions read.
	At time.Time
	// Fetcher serves IMPORT* functions; nil disables them (#IMPORT!).
	Fetcher tsvsheet.Fetcher
	// Loader resolves SHEET(...) embeds; nil disables them (#REF!).
	Loader tsvsheet.Loader
	// Limits bounds engine allocations during the compute pass and expression
	// evaluation; the zero value selects the engine's defaults.
	Limits tsvsheet.Limits
	// MaxCells is the input ceiling ReadTable admits — rows × columns; 0 is
	// unbounded. Exceeding it is ErrLimit.
	MaxCells int
	// IsHeaderless treats every row as data: column references are positional
	// only, rename is ErrHeaderless, and no header is emitted.
	IsHeaderless bool
	// IsStrict aborts the run with ErrStrict on the first expression
	// evaluation that produces an error value.
	IsStrict bool
	// IsRaw skips the compute-first pass: every cell is verbatim text,
	// formula cells included.
	IsRaw bool
}

Options is the per-run configuration. The zero value means: header-first, lenient, compute-first, unbounded input, default engine limits, and embeds and imports disabled (nil Loader/Fetcher).

type Program

type Program struct {
	// contains filtered or unexported fields
}

Program is a parsed, unplanned pipeline of stages: an immutable value, reusable and safe for concurrent Runs. Parse and the builder (NewProgram with the per-verb stage constructors) produce identical Programs.

func NewProgram

func NewProgram(stages ...Stage) Program

NewProgram builds a Program from stages — identical in value and behavior to parsing the equivalent query text. With no stages the program is the identity, a builder-only form: the grammar requires at least one stage (Parse of empty text is ErrSyntax).

func Parse

func Parse(q Query) (Program, error)

Parse parses and validates a whole pipeline; a failure is ErrSyntax with line/column detail. There are no partial programs.

func (Program) Run

func (p Program) Run(t Table, opts Options) (Table, error)

Run plans the program against t's header (or arity, headerless) and executes every stage. Plan-time failures — ErrUnknownColumn, ErrCellRef, ErrHeaderless, a builder expression's ErrSyntax — surface before any row is processed. When the input holds formula cells and IsRaw is unset, the sheet is computed first and the query runs over the computed values (ADR 0003). The returned Table shares no storage with t: mutating one never affects the other.

type Query

type Query string

Query is tq source text — a pipeline of stages separated by `|`.

type RenamePair

type RenamePair struct {
	Col Column
	As  ColumnName
}

RenamePair is one `col as name`.

type RowCount

type RowCount int

RowCount is a limit/offset row count; a negative count is rejected at Run's plan step (ErrSyntax), matching the grammar's non-negative literals.

type SortKey

type SortKey struct {
	Col          Column
	IsDescending bool
}

SortKey is one sort key; IsDescending reverses the key's entire order (`-col`).

type Stage

type Stage struct {
	// contains filtered or unexported fields
}

Stage is one built pipeline stage, produced by the per-verb constructors.

func Derive

func Derive(assignments ...Assignment) Stage

Derive appends computed columns, or replaces in place on name collision, left to right. With no assignments the stage is the identity, a builder-only form the grammar (which requires at least one assignment) cannot express.

func Distinct

func Distinct(cols ...Column) Stage

Distinct keeps the first row per key — whole row when no columns are given — comparing raw cell text.

func Drop

func Drop(cols ...Column) Stage

Drop removes the listed columns; everything else keeps its order.

func GroupBy

func GroupBy(keys []Column, aggs ...Assignment) Stage

GroupBy partitions by the key columns and emits one row per group — key columns first, then one column per aggregate — in first-appearance order. With no keys every row falls in one group, emitting a single whole-table aggregation row — a builder-only form the grammar (which requires at least one key) cannot express.

func Limit

func Limit(n RowCount) Stage

Limit keeps the first n rows.

func Offset

func Offset(n RowCount) Stage

Offset skips the first n rows.

func Rename

func Rename(pairs ...RenamePair) Stage

Rename relabels columns without moving them.

func Select

func Select(cols ...Column) Stage

Select projects and reorders to exactly the listed columns (duplicates allowed).

func SortBy

func SortBy(keys ...SortKey) Stage

SortBy sorts stably by the keys, in the total order of SPECIFICATION §5.

func Where

func Where(expr Expression) Stage

Where keeps rows whose predicate value is exactly TRUE.

type Table

type Table struct {
	Header []string
	Rows   [][]string
}

Table is the data model: an optional header row plus data rows of raw cell texts. A nil Header means headerless. Rows may be ragged; a reference to a column a row lacks reads an empty cell. Constructed by ReadTable, emitted by WriteTable.

func ReadTable

func ReadTable(r io.Reader, opts Options) (Table, error)

ReadTable reads a TSV grid with tsvsheet's comment semantics (a leading `#!` first line and any `# ` line are skipped), splits the header (all rows are data under IsHeaderless), and enforces opts.MaxCells (ErrLimit). A read failure passes through as go-tsvsheet's own sentinel.