README
¶
scan — results as values
The other result strategy: readers that allocate per row and return values, instead of writing through pointers the caller supplied.
Nothing here takes the query guard, and adding one would be a regression. A Reader holds no caller memory, so the guard would refuse a concurrent execution that is safe, contradicting the one property this package exists to offer. TestDynamic_SafeAcrossGoroutines runs one shared Query from sixty-four goroutines under -race. Map2 is the only combinator and stays that way: Go generics cannot express a variadic type list.
Full documentation
The page for this package is docs/scan/README.md, with runnable snippets — every one of them copied from a test whose expected output go test checks. The documentation index is ../docs/README.md and the repository README is ../README.md.
Documentation
¶
Overview ¶
Package scan is a result strategy that returns values instead of writing through pointers.
It is a peer of the binding API in expr, not a replacement for it, and the two differ in exactly one way that matters. A Binding captures a pointer to a caller variable when the query is built, so the query aliases caller memory and two goroutines executing it interleave each other's rows. A Reader captures no memory at all: it allocates fresh destinations for every row and hands back a value, so one Reader is safe to share across any number of concurrent executions.
The cost is that a Reader has to be composed rather than declared inline. The composition is pairwise, which is the whole trick: Map2 combines two Readers into one, so a Reader of any width is Map2 applied repeatedly, and no variadic machinery or reflection is needed to reach ten columns.
It sits above exec and imports no engine. Which columns a Reader selects is decided by the expressions handed to Field, so a Reader over an engine's constructs needs nothing added here.
Index ¶
- Variables
- func All[T any, D render.Dialect](ctx context.Context, db exec.DBTX, s *stmt.Select[D], r Reader[T]) ([]T, error)
- func Collect[T, R any](rows []T, f func(T) R) []R
- func CollectPtr[T, R any](rows []T, f func(T) *R) []R
- func EachMap(ctx context.Context, db exec.DBTX, q stmt.Query, fn func(Row) error) error
- func GroupBy[T any, K comparable, R any](rows []T, key func(T) K, combine func(first T, group []T) R) []R
- func GroupByPtr[T any, K comparable, R any](rows []T, key func(T) *K, combine func(first T, group []T) R) []R
- func One[T any, D render.Dialect](ctx context.Context, db exec.DBTX, s *stmt.Select[D], r Reader[T]) (T, error)
- func Slices(ctx context.Context, db exec.DBTX, q stmt.Query) ([]string, [][]any, error)
- type Reader
- func Field[T any](e expr.Expr[T]) Reader[T]
- func JSON[T any](e render.SQLer) Reader[T]
- func Map[A, B any](a Reader[A], f func(A) B) Reader[B]
- func Map2[A, B, C any](a Reader[A], b Reader[B], f func(A, B) C) Reader[C]
- func New[T any](cols []render.SQLer, alloc func() (dests []any, value func() T)) Reader[T]
- type Row
Constants ¶
This section is empty.
Variables ¶
var ErrDuplicateColumn = errors.New("sqlb/scan: result has two columns with the same name, which a map cannot hold; use Slices")
ErrDuplicateColumn is returned by Maps and EachMap when a result has two columns of the same name.
A map cannot hold both, so the alternative is to keep one and discard the other silently, which is the failure this package exists to avoid. It is an ordinary query that does it, "SELECT a.id, b.id FROM a JOIN b", so the error names the column and points at Slices, which is positional and loses nothing.
var ErrNoColumns = errors.New("sqlb/scan: reader selects no columns")
ErrNoColumns is returned when a Reader selects nothing. A statement with an empty select list is always a mistake, and catching it here names the Reader rather than letting the database report a syntax error.
var ErrNoRows = sql.ErrNoRows
ErrNoRows is returned by One when the statement matched nothing. It aliases sql.ErrNoRows so an existing errors.Is check keeps working and a caller is never asked to learn a second spelling of the same condition.
Functions ¶
func All ¶
func All[T any, D render.Dialect](ctx context.Context, db exec.DBTX, s *stmt.Select[D], r Reader[T]) ([]T, error)
All runs s with r's columns and returns every row as a value.
Unlike the binding API there is no per-row callback and no warning about copying, because each row is already its own value. Appending the result of one iteration cannot alias the next.
r's columns are appended to s's own select list rather than replacing it; see build for what that means where the width of a select list carries meaning of its own.
func Collect ¶
func Collect[T, R any](rows []T, f func(T) R) []R
Collect maps each row of a group through f, which is the body most combine functions would otherwise write by hand.
func(first flat, g []flat) actorWithFilms {
return actorWithFilms{first.Actor, scan.Collect(g, func(r flat) film { return r.Film })}
}
func CollectPtr ¶
func CollectPtr[T, R any](rows []T, f func(T) *R) []R
CollectPtr is Collect for a nullable child, dropping rows whose value is nil and dereferencing the rest.
func EachMap ¶
EachMap reads every row as a map and calls fn once per row.
Unlike the binding API's Each, the map handed to fn is fresh for every row and is never reused, so a callback may keep it. That follows from this package's rule rather than being a special case: a Reader holds no caller memory, and neither does this.
Returning an error from fn stops iteration and returns that error.
func GroupBy ¶
func GroupBy[T any, K comparable, R any]( rows []T, key func(T) K, combine func(first T, group []T) R, ) []R
GroupBy folds the flat rows a join returns into a nested shape.
A join returns the parent repeated once per child, and what a caller wants is one parent holding its children. That reshaping is the job every ORM does with reflection over primary-key tags; here it is a function, so the key and the shape are both decided by the caller and both checked by the compiler.
rows, err := scan.All(ctx, db, sel, flatReader)
actors := scan.GroupBy(rows,
func(r flat) int64 { return r.Actor.ID },
func(first flat, group []flat) actorWithFilms {
films := make([]film, len(group))
for i, r := range group {
films[i] = r.Film
}
return actorWithFilms{first.Actor, films}
})
combine receives the first row of the group, which carries the parent's own columns, and every row in it. Taking both rather than only the slice is what keeps the parent available without the caller indexing into group[0] and having to reason about whether it can be empty; it cannot.
The result is in first-appearance order, not map order, so a query with an ORDER BY keeps it. That is worth stating because a grouping built on a map alone would return a different order on every run and the difference would only show up in a test somebody wrote later.
Nesting deeper is this function applied again inside combine. Two levels is the common case and there is no limit.
func GroupByPtr ¶
func GroupByPtr[T any, K comparable, R any]( rows []T, key func(T) *K, combine func(first T, group []T) R, ) []R
GroupByPtr is GroupBy for a LEFT JOIN, where a parent with no children still produces a row and every child column in it is NULL.
Rows whose key is nil are dropped rather than grouped under a zero key, so a parent with no children yields an empty slice instead of a slice holding one zero-valued child. That is the difference between "Bob has no films" and "Bob has one film with no name", and getting it wrong is the classic bug of hand-written join mapping.
films := scan.GroupByPtr(rowsForThisActor,
func(r flat) *int64 { return r.Film.ID },
func(f flat, group []flat) filmWithCategories { ... })
Use GroupBy when the join is inner and the key cannot be NULL; the compiler will tell you which you have, because the column's Go type is a pointer only when it is nullable.
func One ¶
func One[T any, D render.Dialect](ctx context.Context, db exec.DBTX, s *stmt.Select[D], r Reader[T]) (T, error)
One runs s and returns the single row it matched.
r's columns are appended to s's own select list rather than replacing it; see build for what that means where the width of a select list carries meaning of its own.
It returns ErrNoRows when nothing matched. It does not reject a second row: the binding API's ErrMultipleRows exists because its caller is resolving an identity through a shared statement, whereas a Reader is normally applied to a query that already carries its own LIMIT. Use All and check the length when the uniqueness itself is the thing being asserted.
func Slices ¶
Slices reads every row positionally, returning the column names once and one slice per row.
This is the form to use when a result may have two columns of the same name, which a join across tables sharing a column does routinely, and the form to use for anything writing rows out in order, such as a CSV export, where the names are a header written once rather than a key per cell.
Types ¶
type Reader ¶
type Reader[T any] struct { // contains filtered or unexported fields }
Reader knows which columns to select and how to turn one scanned row into a value of type T.
The invariant that makes it safe to share: alloc is called once per row and returns freshly allocated destinations together with a closure that reads them. Nothing is retained between rows and nothing is shared between executions, so the zero-value Reader is the only unusable one and every constructed Reader is safe for concurrent use.
func Field ¶
Field makes a Reader for a single typed expression.
The destination is a fresh T per row, which is what lets the value be returned rather than aliased. T comes from the expression, so scanning a string column into an int is a compile error here exactly as it is in Bind.
func JSON ¶
JSON reads a column holding JSON and unmarshals it into T.
It is the other half of the database-side answer to nesting. GroupBy folds a join's repeated rows in Go; an aggregate such as json_agg or SQLite's json_group_array builds the array in the database instead, returning one row per parent with its children already collected. This is what turns that column into Go values.
reader := scan.Map2(
scan.Field(authorName),
scan.JSON[[]string](pg.JSONAgg(bookTitle)),
func(name string, titles []string) author { return author{name, titles} },
)
Which of the two to use is a real decision rather than a preference.
Grouping in Go sends the parent's columns once per child, so a wide parent with many children is mostly duplicated bytes on the wire; it keeps every value SQL-typed, and it works on every engine.
Aggregating in the database sends each parent once, which is dramatically less for a wide parent, and it is the only way to get a correct LIMIT on the parents: a join's LIMIT counts rows, not parents, so it truncates a parent's children. The cost is that the values arrive JSON-typed, so a timestamp becomes a string and a large integer can lose precision, and the aggregate is engine-specific. JSON returns a Reader that unmarshals a JSON column into T.
The column is taken as a SQLer rather than a typed expression on purpose. JSON arrives as a different Go type per engine: Postgres hands back json.RawMessage for jsonb, SQLite hands back a string because json_group_array returns TEXT. Insisting on one of those would make the helper unusable on the other engine, and the type parameter that matters is T, the shape you get back, which is still checked.
Unmarshalling happens per row into a fresh T, so a JSON reader composes with Map2 and stays safe to share exactly like every other Reader.
An SQL NULL becomes the zero T rather than an error: a LEFT JOIN with no match produces NULL, which means "no children", and failing there would make the common case the error case.
func Map ¶
Map turns a Reader of A into a Reader of B. It is the one-column case of Map2, useful for wrapping a scanned value in a domain type.
func Map2 ¶
Map2 combines two Readers into one, selecting a's columns then b's.
This is the only combinator the package needs. A three-column Reader is Map2(Map2(a, b, pair), c, ...), and because composition is pairwise the arity is unbounded without a generated family of Map3, Map4 and so on. Go generics cannot express a variadic type list, and this is the standard answer to that, not a workaround for it.
func New ¶
New builds a Reader from the columns it selects and a function that allocates one row's destinations.
It exists because a Reader cannot otherwise be made outside this package, and that asymmetry had a cost. stmt.Binding is an interface precisely so a package outside this module can implement it, which is what let contrib/structbind exist; the result layer's other half had no such door, so a reflective mapping could produce bindings and could not produce a Reader. Reading a query into a slice of structs, the single most common thing a caller wants, was therefore a hand-written loop over exec.Each with a copy per row.
The contract is the whole of the safety argument, so it is stated rather than assumed. alloc is called once per row and must return destinations it has just allocated, together with a closure reading those same destinations. It must not hand back a pointer into memory the caller keeps: nothing in this package takes the query guard, because a Reader is supposed to hold no caller memory, and a Reader that broke that rule would interleave rows across goroutines with no error and no guard to catch it.
The columns are what All and One append to the statement's select list, in scan order, so they must correspond one to one with the destinations.
type Row ¶
Row is one row of a dynamically shaped result, keyed by column name.
The values are whatever the driver produced, not converted: an integer column arrives as int64, a text column as string, a blob as []byte, a NULL as nil. That is deliberate. Normalising them would have to guess, and a guess here is invisible, so the driver's own answer is what you get.
Source Files