README
¶
spaniter
spaniter adapts Cloud Spanner *spanner.RowIterator streams to Go standard
iterators while preserving Spanner metadata and stats timing.
The module is deliberately lower-level than github.com/apstndb/spanvalue: it
does not format values, choose output formats, or own export policy.
for row, err := range spaniter.RowIteratorSeq(rowIter) {
if err != nil {
return err
}
_ = row
}
See pkg.go.dev for lifecycle
semantics, stats encoding, DrainRowIterator, PullRowIteratorSeq, and
worked examples.
Development
make check # fmt, vet, build, test, race, lint
Agent instructions: AGENTS.md.
Documentation
¶
Overview ¶
Package spaniter adapts Cloud Spanner cloud.google.com/go/spanner.RowIterator streams to Go standard iterators (iter.Seq2).
The package is intentionally lower-level than formatters and writers: it owns only iterator lifecycle concerns such as RowIterator.Stop, result metadata, and post-drain query stats. Formatting, headers, and export policy stay in callers.
RowIterator lifecycle ¶
Cloud Spanner populates cloud.google.com/go/spanner.RowIterator fields at specific points in the stream:
- ResultSetMetadata becomes available after the first successful Next call, including empty result sets where the first Next returns iterator.Done.
- Query plan and query stats become available after Next returns iterator.Done when the query used QueryWithStats.
- DML row count becomes available after iterator.Done for DML statements.
RowIteratorSeq keeps those rules explicit. The returned sequence owns rowIter once iteration starts and always calls Stop before returning. If the sequence is never invoked, the caller must still stop the RowIterator. Each sequence is single-use; invoking it twice yields ErrSequenceReused.
Each yielded pair is either a non-nil row with a nil error, or a nil row with a terminal error. Stop processing on the first non-nil error.
Capturing metadata and stats ¶
Use WithResult to capture metadata, stats, and rows read in a RowIteratorResult. Use WithOnMetadata and WithOnStats for hook-style callbacks instead. RowIteratorResult.StatsCaptured reports whether stats were captured after iterator.Done; a zero Stats value alone does not mean stats were absent.
DrainRowIterator consumes the stream without yielding rows and returns a RowIteratorResult. It still reads the stream internally because the Go client only populates stats after iterator.Done.
Stats encoding ¶
Configure protobuf stats encoding with WithStatsEncoding when draining. StatsEncodingDefault omits row_count_exact for zero row counts. StatsEncodingDMLExact preserves row_count_exact:0 for executed standard DML (not PLAN mode). RowIteratorResult.StatsProto and [ResultSet] are the public protobuf conversion entry points.
Early stop and stats ¶
By default, stopping early (break, return, or pull stop) does not read remaining rows. Enable WithDrainOnEarlyStop when callers need stats after an early stop. Post-stop drain errors cannot be yielded; register WithOnDrainError to observe them. With PullRowIteratorSeq, stop can synchronously read the remaining stream when drain is enabled.
Pull-based consumption ¶
PullRowIteratorSeq adapts RowIteratorSeq for iter.Pull2 consumers. Terminal errors are normalized: pull returns (nil, err, false) instead of (nil, err, true). Check err before ok.
Index ¶
- Variables
- func PullRowIteratorSeq(rowIter *spanner.RowIterator, opts ...Option) (pull func() (*spanner.Row, error, bool), stop func())
- func RowIteratorSeq(rowIter *spanner.RowIterator, opts ...Option) iter.Seq2[*spanner.Row, error]
- func Rows(rows ...*spanner.Row) iter.Seq2[*spanner.Row, error]
- func SliceToRowSeq(rows []*spanner.Row) iter.Seq2[*spanner.Row, error]deprecated
- type Option
- type RowIteratorResult
- type Stats
- type StatsEncoding
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNilRow = errors.New("nil row")
ErrNilRow reports that an adapted source produced a nil row with a nil error.
var ErrNilRowIterator = errors.New("nil row iterator")
ErrNilRowIterator reports that RowIteratorSeq was given a nil iterator.
Because RowIteratorSeq returns an iter.Seq2, the error is yielded when the sequence is consumed rather than returned by the constructor.
var ErrSequenceReused = errors.New("row iterator sequence reused")
ErrSequenceReused reports that a RowIteratorSeq result was invoked more than once. Each sequence is single-use; construct a new one for another pass.
Functions ¶
func PullRowIteratorSeq ¶ added in v0.3.0
func PullRowIteratorSeq(rowIter *spanner.RowIterator, opts ...Option) (pull func() (*spanner.Row, error, bool), stop func())
PullRowIteratorSeq adapts RowIteratorSeq for consumers using iter.Pull2.
The returned pull function normalizes terminal errors from the sequence: when RowIteratorSeq yields (nil, err), pull returns (nil, err, false) instead of (nil, err, true). This matches the usual "check err before ok" pattern and avoids treating iterator failures as EOF.
The returned stop function releases the RowIterator. If stop runs before the first pull, stop calls RowIterator.Stop directly because iter.Pull2 has not started the sequence yet. After the first pull, stop signals the sequence and RowIteratorSeq owns Stop when the sequence goroutine exits. With WithDrainOnEarlyStop, stop can synchronously read the remaining stream when the consumer stopped early.
Example ¶
package main
import (
"cloud.google.com/go/spanner"
"github.com/apstndb/spaniter"
)
func main() {
var rowIter *spanner.RowIterator
pull, stop := spaniter.PullRowIteratorSeq(rowIter)
defer stop()
for {
row, err, ok := pull()
if err != nil {
return
}
if !ok {
break
}
_ = row
}
}
Output:
func RowIteratorSeq ¶
RowIteratorSeq adapts a cloud.google.com/go/spanner.RowIterator to a Go standard iterator.
The returned sequence owns rowIter: once iteration starts it always calls *cloud.google.com/go/spanner.RowIterator.Stop before returning. Metadata and stats are exposed through WithOnMetadata and WithOnStats hooks instead of requiring callers to keep reading fields from the stopped RowIterator. The sequence is single-use and not safe for concurrent consumption; construct a new RowIterator for another pass.
If the returned sequence is never invoked, RowIteratorSeq cannot call Stop. After constructing a sequence, callers must either consume it, pass it to code that will consume or stop it, or retain responsibility for stopping the original RowIterator.
Each yielded pair is either a non-nil row with a nil error, or a nil row with a non-nil terminal error. After yielding a non-nil error, the sequence stops. Consumers should stop processing and return or break on the first non-nil error. On terminal errors, WithResult contains only lifecycle data observed before the error, and WithOnStats is not called.
Consumers using iter.Pull2 should prefer PullRowIteratorSeq, which normalizes terminal errors so pull returns ok=false when err!=nil.
Example ¶
package main
import (
"cloud.google.com/go/spanner"
"github.com/apstndb/spaniter"
)
func main() {
var rowIter *spanner.RowIterator
for row, err := range spaniter.RowIteratorSeq(rowIter) {
if err != nil {
return
}
_ = row
}
}
Output:
Example (WithResult) ¶
package main
import (
"cloud.google.com/go/spanner"
"github.com/apstndb/spaniter"
)
func main() {
var rowIter *spanner.RowIterator
var result spaniter.RowIteratorResult
for row, err := range spaniter.RowIteratorSeq(rowIter, spaniter.WithResult(&result)) {
if err != nil {
return
}
_ = row
}
_ = result.Metadata
_ = result.StatsCaptured()
}
Output:
func Rows ¶
Rows adapts already-built rows to the fallible sequence shape used by RowIteratorSeq. Non-nil rows are yielded with a nil error. A nil row aborts the sequence by yielding ErrNilRow.
Row sources that can fail per row should produce their own iter.Seq2 instead of pre-building a slice for Rows.
Example ¶
package main
import (
"fmt"
"github.com/apstndb/spaniter"
)
func main() {
count := 0
for row, err := range spaniter.Rows() {
if err != nil {
return
}
_ = row
count++
}
fmt.Println(count)
}
Output: 0
func SliceToRowSeq
deprecated
Types ¶
type Option ¶
type Option func(*config)
Option configures RowIteratorSeq and DrainRowIterator.
func WithDrainOnEarlyStop ¶
func WithDrainOnEarlyStop() Option
WithDrainOnEarlyStop configures RowIteratorSeq to consume the remaining rows after the consumer stops early.
Draining is disabled by default to preserve normal iterator early-exit behavior. Use this option when callers need WithOnStats to run after any early stop, including a range-loop break or an adapter that stops pulling because a downstream operation failed. Errors encountered only during this post-stop drain cannot be yielded to the caller and therefore suppress the stats hook; register WithOnDrainError to observe them. It has no effect on DrainRowIterator, which always drains.
When used with PullRowIteratorSeq, stop synchronously reads the remaining stream if the consumer stopped early, so stop can block for the full result set size.
Example ¶
package main
import (
"cloud.google.com/go/spanner"
"github.com/apstndb/spaniter"
)
func main() {
var rowIter *spanner.RowIterator
var stats spaniter.Stats
for row, err := range spaniter.RowIteratorSeq(rowIter,
spaniter.WithDrainOnEarlyStop(),
spaniter.WithOnStats(func(s spaniter.Stats) { stats = s }),
) {
if err != nil {
return
}
_ = row
break
}
_ = stats
}
Output:
func WithOnDrainError ¶ added in v0.3.1
WithOnDrainError registers a hook that runs when WithDrainOnEarlyStop is enabled and the post-stop drain fails before iterator.Done. A nil hook is ignored.
func WithOnMetadata ¶
func WithOnMetadata(f func(*sppb.ResultSetMetadata)) Option
WithOnMetadata registers a hook that runs once when result metadata becomes available.
For a query with rows, the hook runs after the first successful Next call and before that first row is yielded, so metadata captured by the hook is visible inside the first loop body. For an empty result set, the hook runs after Next returns iterator.Done. A nil hook is ignored.
func WithOnStats ¶
WithOnStats registers a hook that runs after the adapted iterator has reached iterator.Done and has been stopped.
If the consumer stops early, stats are available only when WithDrainOnEarlyStop is also configured. A nil hook is ignored.
func WithResult ¶
func WithResult(result *RowIteratorResult) Option
WithResult stores iterator lifecycle data in result as it becomes available.
The pointed value is reset when iteration starts. Metadata is set before the first row is yielded, RowsRead is updated after each consumed row, and Stats is set only after the iterator reaches iterator.Done. On errors, result contains the partial lifecycle data observed before the error. A nil result is ignored.
func WithStatsEncoding ¶ added in v0.3.0
func WithStatsEncoding(enc StatsEncoding) Option
WithStatsEncoding configures how RowIteratorResult.StatsProto encodes row counts for a drained iterator. The default is StatsEncodingDefault.
type RowIteratorResult ¶
type RowIteratorResult struct {
Metadata *sppb.ResultSetMetadata
Stats Stats
RowsRead int64
// contains filtered or unexported fields
}
RowIteratorResult is the metadata and stats available from a cloud.google.com/go/spanner.RowIterator.
RowsRead counts rows consumed from the iterator. Metadata and Stats values are not deep-copied from the underlying RowIterator; treat returned maps and protos as read-only. Stats protobuf encoding is configured by WithStatsEncoding on the drain options.
Populate this type with WithResult, DrainRowIterator, or PullRowIteratorSeq. Use RowIteratorResult.StatsCaptured to tell whether stats were captured after iterator.Done; a zero Stats value alone does not mean stats were absent.
func DrainRowIterator ¶
func DrainRowIterator(rowIter *spanner.RowIterator, opts ...Option) (*RowIteratorResult, error)
DrainRowIterator consumes rowIter to iterator.Done without yielding rows.
The helper owns rowIter and always calls *cloud.google.com/go/spanner.RowIterator.Stop before returning. It is useful when callers need result metadata, query stats, query plan, or DML row count but do not want to expose row values to application code. If iteration fails, the returned result can be non-nil and contain partial metadata and RowsRead observed before the error; stats are only populated after a successful drain to iterator.Done.
Cloud Spanner only populates metadata after the first Next call, and stats after Next returns iterator.Done. DrainRowIterator therefore still consumes the result stream internally; it does not ask Spanner for stats without reading the stream. To avoid reading data rows at the query level, callers must execute a statement that returns no data rows.
Example ¶
package main
import (
"cloud.google.com/go/spanner"
"github.com/apstndb/spaniter"
)
func main() {
var rowIter *spanner.RowIterator
result, err := spaniter.DrainRowIterator(rowIter,
spaniter.WithStatsEncoding(spaniter.StatsEncodingDMLExact),
)
if err != nil {
return
}
stats, err := result.StatsProto()
if err != nil {
return
}
_ = stats
}
Output:
func (RowIteratorResult) ResultSet ¶ added in v0.3.0
ResultSet builds a protobuf ResultSet from materialized rows and iterator lifecycle data captured while draining a RowIterator.
rows may be nil when row values are intentionally omitted. Stats encoding comes from WithStatsEncoding on the drain options. Use only on package-produced RowIteratorResult values; see RowIteratorResult.StatsProto.
func (RowIteratorResult) StatsCaptured ¶ added in v0.3.1
func (r RowIteratorResult) StatsCaptured() bool
StatsCaptured reports whether stats were captured after the iterator reached iterator.Done. When false, Stats is zero and RowIteratorResult.StatsProto omits row counts even under StatsEncodingDMLExact.
func (RowIteratorResult) StatsProto ¶ added in v0.3.0
func (r RowIteratorResult) StatsProto() (*sppb.ResultSetStats, error)
StatsProto returns captured stats as *sppb.ResultSetStats using the encoding configured by WithStatsEncoding when the iterator was drained.
Row counts are encoded only after stats were captured at iterator.Done. Query plan and query stats encode from the captured Stats value whenever present. Partial results from errors omit row counts even when StatsEncodingDMLExact is configured.
Call this only on RowIteratorResult values populated by spaniter. Use RowIteratorResult.StatsCaptured to tell whether stats were captured.
type Stats ¶
Stats holds execution information populated on a cloud.google.com/go/spanner.RowIterator after the iterator reaches iterator.Done.
QueryPlan and QueryStats are set when the query used QueryWithStats. RowCount holds the DML row count after iterator.Done. Values are not deep-copied from the underlying RowIterator; treat returned maps and protos as read-only.
Stats mirrors the public fields exposed by cloud.google.com/go/spanner.RowIterator. Use RowIteratorResult.StatsProto when downstream code needs the protobuf ResultSetStats shape. The Go client has already decoded query stats to a map and exposes row count as a single int64, so Stats cannot distinguish an absent row count from row_count_exact:0. The Go client's PartitionedUpdate APIs return counts directly rather than through RowIterator, so partitioned DML counts are outside this type's normal scope.
type StatsEncoding ¶ added in v0.3.0
type StatsEncoding int
StatsEncoding selects how captured stats are converted to protobuf ResultSetStats when using RowIteratorResult.StatsProto or RowIteratorResult.ResultSet.
Set encoding with WithStatsEncoding when draining a RowIterator.
const ( // StatsEncodingDefault uses default query semantics: omit row_count_exact when // RowCount is zero because absent row count and exact zero are indistinguishable // on [cloud.google.com/go/spanner.RowIterator]. StatsEncodingDefault StatsEncoding = iota // StatsEncodingDMLExact always encodes RowCount as row_count_exact, including // zero. Use only when the caller knows Stats came from executed standard DML // (not PLAN and not read-only queries). StatsEncodingDMLExact )
Source Files