parquetquery

Documentation

Overview

Package parquetquery implements a DuckDB-backed query engine for Parquet archive files. It reads Parquet files written by bintrail rotate --archive-dir (local) or stored in S3, and returns results in the same format as internal/query.

Index

• func Fetch(ctx context.Context, opts query.Options, source string) ([]query.ResultRow, error)
• func FetchWithTuning(ctx context.Context, opts query.Options, source string, ...) ([]query.ResultRow, error)

Functions

func Fetch

func Fetch(ctx context.Context, opts query.Options, source string) ([]query.ResultRow, error)

Fetch queries Parquet archive files (local or S3) using DuckDB and returns matching events. source is either a local directory path or an S3 URL prefix (s3://bucket/prefix/). Archives follow the Hive-partitioned layout written by bintrail rotate (event_date=YYYY-MM-DD/event_hour=HH/events.parquet).

It applies bintrail's conservative, container-safe DuckDB budget (duckdbutil.DefaultTuning). Long-lived/shared callers (shim, console, agent) use this. The offline CLI commands that can afford more resources call FetchWithTuning to lift the cap (#510).

func FetchWithTuning

func FetchWithTuning(ctx context.Context, opts query.Options, source string, tuning duckdbutil.Tuning) ([]query.ResultRow, error)

FetchWithTuning is Fetch with an explicit DuckDB resource budget. See Tuning: the conservative DefaultTuning (threads=2, memory_limit=4GB) is for small containers; duckdbutil.Ultrafast lets DuckDB self-tune to the host.