dbsqlrows

README

dbsqlrows

Experimental — APIs may change before a stable release.

Package documentation: pkg.go.dev/github.com/apstndb/spanvalue/dbsqlrows

Streams database/sql query results into writer using the GenericColumnValue slice export path, or into custom sinks via SQLRowsHooks. The package owns the *sql.Rows loop (metadata pseudo-row → data rows → optional stats pseudo-row) and keeps database/sql drivers out of the root go.mod. See the package godoc for the API contract; this README covers driver integration and application patterns.

go-sql-spanner integration

Option A (driver-agnostic): configure ExecOptions at query time, then pass open *sql.Rows to WriteRows or RunRows:

opts := spannerdriver.ExecOptions{
    DecodeOption:            spannerdriver.DecodeOptionProto,
    ReturnResultSetMetadata: true,
    ReturnResultSetStats:    false,
}
rows, err := db.QueryContext(ctx, q, opts)
// ...
result, err := dbsqlrows.WriteRows(rows, w, dbsqlrows.SQLRowsConfig{})

Option B: nested module gospanner/ provides DefaultExecOptions and QueryExport for one-shot query → csv/jsonl export when the app already depends on go-sql-spanner. It is a thin reference integration (ExecOptions + QueryContext + WriteRows); root go.mod still has no go-sql-spanner. Interactive shells, metadata-first batches, EXPLAIN, and per-query driver options (QueryMode, DirectExecuteQuery) should use Option A with app-owned ExecOptions instead — validated by spannersh.

Stats: driver vs export

A common REPL pattern: set ReturnResultSetStats true on the driver at QueryContext, keep SQLRowsConfig.ReadResultSetStats false during csv/jsonl/table export, then read the stats pseudo-row in application code after render. dbsqlrows leaves the cursor on the data result set until export completes or ReadResultSetStats is enabled.

Application patterns

• Metadata-first batch: ReadMetadataAndAdvanceToData, render (table via RunRowsAtData + hooks), then read stats from rows or set ReadResultSetStats.
• Table sink: RunRowsAtData with SQLRowsHooks.WithPrepareMetadata, WithWriteDataRow, and WithFinish — apps own layout libraries and cell formatting.
• csv/jsonl: WriteRowsAtData with writer.DelimitedGCVExportOptions or writer.JSONLGCVExportOptions at writer construction.
• EXPLAIN / drain: RunRowsAtData with NewSQLRowsHooks and SQLRowsConfig.WithReadResultSetStats(true).

Testing

go test ./dbsqlrows/...
# or from repo root: make check

Optional nested module gospanner/ (QueryExport, DefaultExecOptions).

Documentation

Overview

Package dbsqlrows is experimental: APIs may change before a stable release.

It streams database/sql query results into github.com/apstndb/spanvalue/writer using the GenericColumnValue slice export path, or into custom sinks via SQLRowsHooks.

Callers that use a Spanner database/sql driver (for example github.com/googleapis/go-sql-spanner) configure driver-specific options themselves; this package only iterates *sql.Rows once they are open.

Extended documentation, go-sql-spanner integration recipes (ExecOptions), and application patterns (REPL stats flow, table sinks): https://github.com/apstndb/spanvalue/blob/main/dbsqlrows/README.md

Naming

The name combines db (standard library database/sql) and sqlrows (*sql.Rows as input). For the native-client export path, use github.com/apstndb/spanvalue/writer (github.com/apstndb/spanvalue/writer.WriteRowIterator on *cloud.google.com/go/spanner.RowIterator).

writer vs dbsqlrows

Three export paths, by iterator and row shape:

• Native client: *spanner.RowIterator yielding *spanner.Row; spanvalue entry github.com/apstndb/spanvalue/writer.WriteRowIterator.
• database/sql driver: *sql.Rows yielding []spanner.GenericColumnValue; spanvalue entry github.com/apstndb/spanvalue/writer.DelimitedWriter.WriteGCVs.
• dbsqlrows: *sql.Rows (caller-owned) yielding []spanner.GenericColumnValue; spanvalue entry RunRowsAtData / WriteRowsAtData.

dbsqlrows does not convert GCV slices to *spanner.Row for github.com/apstndb/spanvalue/writer.Writer.WriteRow.

Module layout

Package path github.com/apstndb/spanvalue/dbsqlrows is part of the single github.com/apstndb/spanvalue module. The package does not import go-sql-spanner (or any database/sql driver). Optional one-shot helpers live in nested module github.com/apstndb/spanvalue/dbsqlrows/gospanner.

Goals

• Own the *sql.Rows loop: metadata pseudo-row → data rows → optional stats pseudo-row.
• Delegate csv/jsonl formatting to GCVStreamWriter.WriteGCVs / GCVStreamWriter.Flush.
• Expose SQLRowsHooks for custom sinks (table layout, drain-only) parallel to github.com/apstndb/spanvalue/writer.RowIteratorHooks.
• Keep database/sql drivers out of spanvalue go.mod.

Non-goals

• Native *spanner.RowIterator export (github.com/apstndb/spanvalue/writer.WriteRowIterator).
• String → GCV parsing, PostgreSQL table cells, or built-in ASCII table layout.
• Batch orchestration, SQL INSERT export, or owning db.QueryContext / driver ExecOptions.

API overview

• WriteRows: open *sql.Rows at the metadata pseudo-row; csv/jsonl via GCVStreamWriter.
• RunRows / RunRowsAtData: custom sinks via SQLRowsHooks.
• ReadMetadataAndAdvanceToData: metadata-first apps; advances cursor to data rows.
• WriteRowsAtData: RunRowsAtData + SQLRowsHooksFromGCVWriter.

Symmetry with writer:

• github.com/apstndb/spanvalue/writer.RunRowIterator ↔ RunRows / RunRowsAtData
• github.com/apstndb/spanvalue/writer.RowIteratorHooks ↔ SQLRowsHooks
• github.com/apstndb/spanvalue/writer.RowIteratorHooksFromWriter ↔ SQLRowsHooksFromGCVWriter
• github.com/apstndb/spanvalue/writer.RowIteratorResult ↔ SQLRowsResult

SQLRowsResult carries Metadata when known on error paths (partial-result contract matching github.com/apstndb/spanvalue/writer.RowIteratorResult). Stats are not consumed unless SQLRowsConfig.ReadResultSetStats is true; the iterator then advances with NextResultSet for multi-statement batches. SQLRowsResult.RowsRead follows github.com/apstndb/spanvalue/writer.RowIteratorResult RowsRead semantics: it counts data rows for which WriteDataRow returned nil and stays zero when WriteDataRow is nil, even though rows are still drained.

SQLRowsConfig.ReadResultSetStats requires the driver to produce a stats pseudo result set (ReturnResultSetStats: true at QueryContext); otherwise the run fails with ErrMissingStatsResultSet. With driver stats disabled in a multi-statement batch, NextResultSet would otherwise land on the next statement's metadata result set and consume its pseudo-row before the scan fails, corrupting the batch cursor position.

An empty SQLRowsHooks value advances past data rows without per-row decode when WriteDataRow is nil (EXPLAIN / drain before stats; RowsRead stays zero). When WriteDataRow is set, the GCV slice passed to it is reused each row — copy or format before returning if the sink retains row data.

Index

• Variables
• func ReadMetadataAndAdvanceToData(rows *sql.Rows) (*sppb.ResultSetMetadata, bool, error)
• type GCVStreamWriter
• type SQLRowsConfig
  • func (cfg SQLRowsConfig) WithReadResultSetStats(read bool) SQLRowsConfig
• type SQLRowsHooks
  • func NewSQLRowsHooks() SQLRowsHooks
  • func SQLRowsHooksFromGCVWriter(w GCVStreamWriter) SQLRowsHooks
  • func (h SQLRowsHooks) WithFinish(fn func(*SQLRowsResult) error) SQLRowsHooks
  • func (h SQLRowsHooks) WithPrepareMetadata(fn func(*sppb.ResultSetMetadata) error) SQLRowsHooks
  • func (h SQLRowsHooks) WithWriteDataRow(fn func([]spanner.GenericColumnValue) error) SQLRowsHooks
• type SQLRowsResult
  • func RunRows(rows *sql.Rows, hooks SQLRowsHooks, cfg SQLRowsConfig) (*SQLRowsResult, error)
  • func RunRowsAtData(rows *sql.Rows, metadata *sppb.ResultSetMetadata, hooks SQLRowsHooks, ...) (*SQLRowsResult, error)
  • func WriteRows(rows *sql.Rows, w GCVStreamWriter, cfg SQLRowsConfig) (*SQLRowsResult, error)
  • func WriteRowsAtData(rows *sql.Rows, metadata *sppb.ResultSetMetadata, w GCVStreamWriter, ...) (*SQLRowsResult, error)

Variables

var (
	// ErrNilRows reports that a dbsqlrows entry point was called with a nil *sql.Rows
	// (for example [WriteRows], [RunRows], [RunRowsAtData], [WriteRowsAtData], or
	// [ReadMetadataAndAdvanceToData]).
	ErrNilRows = errors.New("nil sql.Rows")
	// ErrNilWriter reports that an export entry point was called with a nil [GCVStreamWriter]
	// (for example [WriteRows] or [WriteRowsAtData]).
	ErrNilWriter = errors.New("nil GCV stream writer")
	// ErrNilMetadata reports that a data-phase entry point was called with nil metadata
	// (for example [WriteRowsAtData] or [RunRowsAtData]).
	ErrNilMetadata = errors.New("nil result set metadata")
	// ErrMissingMetadataRow reports that the iterator produced no metadata
	// pseudo-row when WriteRows expected one.
	ErrMissingMetadataRow = errors.New("missing result set metadata row")
	// ErrMissingDataResultSet reports that NextResultSet did not advance to the
	// data rows result set after the metadata pseudo-row.
	ErrMissingDataResultSet = errors.New("missing data rows result set after metadata")
	// ErrMissingStatsRow reports that the stats result set had no stats pseudo-row.
	ErrMissingStatsRow = errors.New("missing result set stats row")
	// ErrMissingStatsResultSet reports that NextResultSet did not advance to the
	// stats result set when [SQLRowsConfig.ReadResultSetStats] was requested.
	// This typically means the driver was not opened with ReturnResultSetStats
	// enabled (see the precondition on [SQLRowsConfig.ReadResultSetStats]).
	ErrMissingStatsResultSet = errors.New("missing result set stats result set")
)

Functions

func ReadMetadataAndAdvanceToData

func ReadMetadataAndAdvanceToData(rows *sql.Rows) (*sppb.ResultSetMetadata, bool, error)

ReadMetadataAndAdvanceToData reads the metadata pseudo-row from rows and advances to the data result set. Use before WriteRowsAtData or custom rendering when metadata is consumed outside export.

If there is no metadata row, returns ok=false and err=rows.Err() (nil on clean EOF).

Types

type GCVStreamWriter

type GCVStreamWriter interface {
	WriteGCVs([]spanner.GenericColumnValue) error
	Flush() error
}

GCVStreamWriter is the subset of github.com/apstndb/spanvalue/writer types that dbsqlrows drives. Built-in writers also implement PrepareRowType or Prepare for metadata registration; SQLRowsHooksFromGCVWriter calls those when present after reading the metadata pseudo-row.

type SQLRowsConfig

type SQLRowsConfig struct {
	// ReadResultSetStats, when true, advances past data rows to read the stats
	// pseudo-row into [SQLRowsResult.Stats]. For [WriteRowsAtData] this field is
	// consulted directly (default false). For [WriteRows] the same field applies.
	//
	// Precondition: the driver must produce a stats pseudo result set (for
	// go-sql-spanner, open rows with ReturnResultSetStats: true). When no stats
	// result set follows the data rows, the run fails with
	// [ErrMissingStatsResultSet]. Note that in a multi-statement batch with
	// driver stats disabled, NextResultSet can instead land on the next
	// statement's metadata result set; the resulting scan error is reported only
	// after that pseudo-row has been consumed, so the batch cursor position is
	// not recoverable.
	ReadResultSetStats bool
}

SQLRowsConfig configures a SQL rows streaming run.

func (SQLRowsConfig) WithReadResultSetStats

func (cfg SQLRowsConfig) WithReadResultSetStats(read bool) SQLRowsConfig

WithReadResultSetStats returns a copy of cfg with ReadResultSetStats set.

type SQLRowsHooks

type SQLRowsHooks struct {
	PrepareMetadata func(*sppb.ResultSetMetadata) error
	WriteDataRow    func([]spanner.GenericColumnValue) error
	Finish          func(*SQLRowsResult) error
}

SQLRowsHooks drives RunRows and RunRowsAtData. Nil function fields are skipped.

An empty hooks value (from NewSQLRowsHooks) still advances past data rows while WriteDataRow is nil (no per-row decode), but SQLRowsResult.RowsRead stays zero for drained rows, matching github.com/apstndb/spanvalue/writer.RowIteratorResult RowsRead semantics. Use that to drain rows before reading stats (for example EXPLAIN with SQLRowsConfig.ReadResultSetStats).

PrepareMetadata runs once after metadata is known and before data rows are scanned. WriteDataRow runs per data row when set. The []spanner.GenericColumnValue argument is reused across calls: valid only for the duration of WriteDataRow; copy or format synchronously before returning if the sink retains row data. Finish runs only after all rows and optional stats consumption succeed; it is not called when PrepareMetadata or WriteDataRow returns an error. The returned SQLRowsResult still carries Metadata and RowsRead at the abort point (same partial-result contract as github.com/apstndb/spanvalue/writer.RowIteratorHooks and github.com/apstndb/spanvalue/writer.RunRowIterator).

func NewSQLRowsHooks

func NewSQLRowsHooks() SQLRowsHooks

NewSQLRowsHooks returns an empty hooks value for custom decoration or SQLRowsHooksFromGCVWriter.

func SQLRowsHooksFromGCVWriter

func SQLRowsHooksFromGCVWriter(w GCVStreamWriter) SQLRowsHooks

SQLRowsHooksFromGCVWriter returns hooks that register metadata via GCVStreamWriter PrepareRowType or Prepare when implemented, write each row with GCVStreamWriter.WriteGCVs, and call GCVStreamWriter.Flush in Finish. Finish (and thus Flush) runs only after all rows and optional stats consumption succeed; it is skipped when PrepareMetadata, WriteDataRow, or the stats phase returns an error. A nil writer returns empty hooks.

func (SQLRowsHooks) WithFinish

func (h SQLRowsHooks) WithFinish(fn func(*SQLRowsResult) error) SQLRowsHooks

WithFinish sets Finish and returns h.

func (SQLRowsHooks) WithPrepareMetadata

func (h SQLRowsHooks) WithPrepareMetadata(fn func(*sppb.ResultSetMetadata) error) SQLRowsHooks

WithPrepareMetadata sets PrepareMetadata and returns h.

func (SQLRowsHooks) WithWriteDataRow

func (h SQLRowsHooks) WithWriteDataRow(fn func([]spanner.GenericColumnValue) error) SQLRowsHooks

WithWriteDataRow sets WriteDataRow and returns h. The slice passed to fn is reused on each row; do not retain it after fn returns.

type SQLRowsResult

type SQLRowsResult struct {
	Metadata *sppb.ResultSetMetadata
	Stats    *sppb.ResultSetStats
	RowsRead int
}

SQLRowsResult holds metadata and stats surfaced from driver pseudo result sets, analogous to github.com/apstndb/spanvalue/writer.RowIteratorResult for native iterators. On error paths after metadata is known, Metadata and RowsRead reflect progress at the abort point (same partial-result contract as writer row-iterator helpers). Stats is also populated on the abort path when the stats pseudo-row was already read but the trailing NextResultSet advance failed.

RowsRead counts data rows for which SQLRowsHooks.WriteDataRow returned nil. It stays zero when WriteDataRow is nil (rows may still be drained), matching github.com/apstndb/spanvalue/writer.RowIteratorResult RowsRead semantics.

func RunRows

func RunRows(rows *sql.Rows, hooks SQLRowsHooks, cfg SQLRowsConfig) (*SQLRowsResult, error)

RunRows streams an open *sql.Rows positioned at the metadata pseudo-row using hooks. See WriteRows for driver conventions, ownership, and stats behavior.

func RunRowsAtData

func RunRowsAtData(
	rows *sql.Rows,
	metadata *sppb.ResultSetMetadata,
	hooks SQLRowsHooks,
	cfg SQLRowsConfig,
) (*SQLRowsResult, error)

RunRowsAtData streams rows already positioned on the data result set using hooks. metadata must be non-nil. See WriteRowsAtData for stats and partial-result semantics.

func WriteRows

func WriteRows(rows *sql.Rows, w GCVStreamWriter, cfg SQLRowsConfig) (*SQLRowsResult, error)

WriteRows streams an open *sql.Rows positioned at the metadata pseudo-row into w. The caller must open rows with a driver that returns proto-decoded GCV columns and a leading metadata pseudo result set (see README). The caller retains ownership of rows and must Close it and check sql.Rows.Err when appropriate.

On success WriteRows calls GCVStreamWriter.Flush and returns its error explicitly (do not defer Flush at the call site). Data rows are scanned into []spanner.GenericColumnValue.

When ReadResultSetStats is false (the default), rows remain on the data result set after export so the caller can advance to stats separately.

For custom sinks (for example ASCII table rendering), use RunRows with SQLRowsHooks instead.

func WriteRowsAtData

func WriteRowsAtData(
	rows *sql.Rows,
	metadata *sppb.ResultSetMetadata,
	w GCVStreamWriter,
	cfg SQLRowsConfig,
) (*SQLRowsResult, error)

WriteRowsAtData streams rows already positioned on the data result set into w. metadata must be non-nil (typically from ReadMetadataAndAdvanceToData or an earlier statement in a batch). The writer is prepared from metadata when it implements PrepareRowType or Prepare.

Stats are not consumed unless cfg.ReadResultSetStats is true, so callers can render first and read stats from rows afterward (spannersh execution summary).

For custom sinks, use RunRowsAtData with SQLRowsHooks.