data

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 11, 2026 License: BSD-3-Clause Imports: 8 Imported by: 0

README

go-widgets/data

The headless data spine of the go-widgets UI ecosystem: a typed record model with validation, a bindable collection with sort / filter / group / pagination / aggregation, and a pluggable proxy so the very same query pipeline runs in-process or against a remote service — natively and in a browser/wasm build alike.

Pure Go, CGO=0, BSD-3-Clause. The core package imports only the standard library and go-widgets/mvvm; nothing here depends on a GUI.

Layers

Piece What it is
Value / Kind a comparable typed scalar (string / int / float / bool)
Record / Schema / Field / Rule a typed row and its validation
Query → Apply → View the pure query engine: filter, sort, group, page, aggregate
Proxy the backend seam: List / Query / Mutate
MemoryProxy the in-process reference backend
grpcproxy.Server / grpcproxy.Client the same contract over gRPC, carried by grpc-transports/websocket
Store[R] a typed, bindable collection wired through mvvm.ObservableList

Why a proxy seam

Store talks to a Proxy; it never knows whether the data is a MemoryProxy in the same process or a grpcproxy.Client reaching a server across a websocket. Because the query engine (Apply) is a pure function of (records, Query), the client and the server run the same code on the same rows — so a MemoryProxy and a grpcproxy.Client return byte-identical Views.

That equality is asserted directly: grpcproxy's conformance test runs a battery of sort→filter→group→page→aggregate queries through both proxies and through the engine, canonicalises every resulting View, and requires all three to be identical byte for byte. The WebSocket transport compiles to js/wasm, so the grpcproxy.Client is exactly what a go-widgets wasm app uses to speak this same service from the browser — no second data path.

Example

schema := data.Schema{Fields: []data.Field{
    {Name: "id", Kind: data.KindInt},
    {Name: "name", Kind: data.KindString, Rules: []data.Rule{data.Required("name required")}},
    {Name: "salary", Kind: data.KindFloat},
}}
mem, _ := data.NewMemoryProxy(schema, "id",
    data.Record{"id": data.Int(1), "name": data.String("ann"), "salary": data.Float(30)},
)

type person struct{ ID int64; Name string; Salary float64 }
store := data.NewStore(mem, data.Codec[person]{
    Encode: func(p person) data.Record {
        return data.Record{"id": data.Int(p.ID), "name": data.String(p.Name), "salary": data.Float(p.Salary)}
    },
    Decode: func(r data.Record) person {
        return person{ID: r["id"].Int, Name: r["name"].Str, Salary: r["salary"].Float}
    },
})
store.SetQuery(data.Query{Sorts: []data.Sort{{Field: "salary", Desc: true}}, Limit: 20})
store.Load(ctx)                 // runs the query, fills the ObservableList
_ = store.Items()               // bind this to a view — it re-renders on change

Swap mem for a grpcproxy.Client and nothing else changes.

Status

Core data and grpcproxy are at 100% statement coverage (the generated datapb/ protobuf code excepted); CI builds all six 64-bit Go targets plus js/wasm, macOS and Windows, and runs the race detector. Org-conformance landing/logo/docs are a follow-up.

Documentation

Overview

Package data is the headless data spine of the go-widgets ecosystem: a typed record model with validation, a collection with sort/filter/group/pagination/ aggregation, and a pluggable proxy so the very same query pipeline runs in-process (MemoryProxy) or against a remote service (the grpcproxy subpackage) — natively and in a browser/wasm build alike. It imports only the standard library and go-widgets/mvvm, so nothing here depends on a GUI.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrKeyExists = errors.New("data: key already exists")
	ErrNotFound  = errors.New("data: record not found")
)

ErrKeyExists / ErrNotFound report the two identity failures a mutation can hit: inserting a key that already exists, or updating/deleting one that does not.

Functions

func Canonical 

func Canonical(v View) []byte

Canonical renders a View as a deterministic byte string: the same View value always yields the same bytes, and — crucially — a View computed by MemoryProxy and the identical View reconstructed from a grpcproxy round-trip encode to the SAME bytes. That is the equality the proxy-conformance test asserts.

Determinism is achieved by never depending on Go map iteration order: record fields and aggregate labels are always emitted in sorted order, and rows and groups keep their query order. The format is unambiguous (length-tagged enough for the fields we carry), but it is meant for equality, not parsing.

Types

type Agg 

type Agg struct {
	Field string
	Func  AggFunc
}

Agg requests one aggregate over a column. For AggCount the Field is ignored.

type AggFunc 

type AggFunc uint8

AggFunc is a column aggregation. 

const (
	// AggCount counts rows (its Field is ignored).
	AggCount AggFunc = iota
	// AggSum / AggAvg / AggMin / AggMax reduce a numeric column.
	AggSum
	AggAvg
	AggMin
	AggMax
)

type Codec 

type Codec[R any] struct {
	Encode func(R) Record
	Decode func(Record) R
}

Codec converts between a caller's typed row R and the schema-typed Record the proxy stores. Supplying the two functions keeps Store reflection-free and lets the app choose exactly how its struct maps onto fields.

type Field 

type Field struct {
	Name  string
	Kind  Kind
	Rules []Rule
}

Field declares one column of a Schema: its name, its scalar Kind, and any validation Rules run against a row's value for it.

type Filter 

type Filter struct {
	Field string
	Op    FilterOp
	Value Value
}

Filter is one predicate: keep the rows for which Field's cell relates to Value under Op. A filter on a field a row lacks never matches.

type FilterOp 

type FilterOp uint8

FilterOp is a comparison a Filter applies between a record's cell and a reference Value. 

const (
	// OpEq keeps rows whose cell equals the reference value.
	OpEq FilterOp = iota
	// OpNe keeps rows whose cell differs from the reference value.
	OpNe
	// OpLt / OpLe / OpGt / OpGe keep rows ordered below / at-or-below / above /
	// at-or-above the reference value (Value.compare ordering).
	OpLt
	OpLe
	OpGt
	OpGe
	// OpContains keeps rows whose string cell contains the reference substring.
	// It only matches string cells (a non-string cell never contains).
	OpContains
)

type Group 

type Group struct {
	Key        Value
	Rows       []Record
	Aggregates map[string]Value
}

Group is one bucket of a grouped View: the shared key value, the group's rows (in the query's sort order), and its per-group aggregates.

type Kind 

type Kind uint8

Kind is the scalar type of a Value. The set is deliberately small — the four types a data grid needs — so a Value stays comparable and round-trips through the wire codec without loss. 

const (
	// KindString is a UTF-8 text value.
	KindString Kind = iota
	// KindInt is a signed 64-bit integer value.
	KindInt
	// KindFloat is a 64-bit IEEE-754 float value.
	KindFloat
	// KindBool is a boolean value.
	KindBool
)

type MemoryProxy 

type MemoryProxy struct {
	// contains filtered or unexported fields
}

MemoryProxy is an in-process Proxy over a slice of validated records. It is the reference backend: its Query runs the shared Apply engine directly, and a grpcproxy Server wraps one to serve the same results across the wire.

It is safe for concurrent use.

func NewMemoryProxy 

func NewMemoryProxy(schema Schema, keyField string, seed ...Record) (*MemoryProxy, error)

NewMemoryProxy builds a MemoryProxy for the given schema, using keyField as the record identity (it must be a declared field). Each seed record is validated and must have a unique key. It returns an error on an unknown key field, an invalid seed, or a duplicate key.

func (*MemoryProxy) List 

func (m *MemoryProxy) List(_ context.Context) ([]Record, error)

List returns a deep copy of every record.

func (*MemoryProxy) Mutate 

func (m *MemoryProxy) Mutate(_ context.Context, mut Mutation) error

Mutate applies one write under the write lock.

func (*MemoryProxy) Query 

func (m *MemoryProxy) Query(_ context.Context, q Query) (View, error)

Query applies q via the shared engine over a snapshot of the records.

type Mutation 

type Mutation struct {
	Kind   MutationKind
	Record Record
	Key    Value
}

Mutation is a single write. Insert and Update carry the full Record; Delete carries only the Key (the value of the backend's key field to remove).

type MutationKind 

type MutationKind uint8

MutationKind is the operation a Mutation performs. 

const (
	// MutInsert adds Record (which must be new under the key field).
	MutInsert MutationKind = iota
	// MutUpdate replaces the record whose key field equals Record's key field.
	MutUpdate
	// MutDelete removes the record whose key field equals Key.
	MutDelete
)

type Proxy 

type Proxy interface {
	// List returns every record, unfiltered and unordered (a fresh copy each
	// call, so the caller can retain it safely).
	List(ctx context.Context) ([]Record, error)
	// Query applies q to the backend's records and returns the resulting View.
	Query(ctx context.Context, q Query) (View, error)
	// Mutate inserts, updates or deletes one record and reports any validation
	// or not-found error.
	Mutate(ctx context.Context, m Mutation) error
}

Proxy is the pluggable data backend a Store talks to. The same three operations are served in-process by MemoryProxy and remotely by grpcproxy.Client, so a Store — and every sort/filter/group/page/aggregate it drives — is oblivious to whether the data is local or across a websocket.

type Query 

type Query struct {
	// Filters are ANDed: a row must satisfy all of them.
	Filters []Filter
	// Sorts order the surviving rows (primary key first).
	Sorts []Sort
	// GroupBy, when non-empty, groups the ordered rows by that field's value.
	GroupBy string
	// Offset skips this many rows (ungrouped) or groups (grouped) before the page.
	Offset int
	// Limit caps the page to this many rows (ungrouped) or groups (grouped); 0
	// means no limit.
	Limit int
	// Aggs are computed over the whole filtered set (View.Aggregates) and, when
	// grouped, over each group (Group.Aggregates).
	Aggs []Agg
}

Query is a full read specification the engine applies to a set of records: keep the rows matching every Filter, order them by Sorts, optionally group by a field, take a page (Offset/Limit), and compute Aggs. Zero-value fields are inert — an empty Query returns every row unchanged with no grouping or paging.

type Record 

type Record map[string]Value

Record is one row: a set of named typed cells. It is a plain map so callers build and read rows with ordinary Go, while the Schema gives it type and the canonical encoder gives it a stable serialization (fields are always emitted in sorted-name order, so map iteration order never leaks into a comparison).

type Rule 

type Rule func(Value) error

Rule validates one cell value, returning a non-nil error (whose text is the message shown to the user) when the value fails. It mirrors the string-rule shape of go-widgets/toolkit's validation, lifted to a typed Value so a schema can validate numbers and booleans, not only text.

func NumMax 

func NumMax(hi float64, msg string) Rule

NumMax rejects a numeric value above hi.

func NumMin 

func NumMin(lo float64, msg string) Rule

NumMin rejects a numeric value below lo (int and float compared as float64).

func Required 

func Required(msg string) Rule

Required rejects a zero value for the field's kind: an empty string, a zero number, or false. Use it to demand a present, non-default cell.

func StrMaxLen 

func StrMaxLen(n int, msg string) Rule

StrMaxLen rejects a string value longer than n runes.

func StrMinLen 

func StrMinLen(n int, msg string) Rule

StrMinLen rejects a string value shorter than n runes.

type Schema 

type Schema struct {
	Fields []Field
}

Schema is an ordered list of Fields — the typed shape a Record must satisfy.

func (Schema) Field 

func (s Schema) Field(name string) (Field, bool)

Field looks a field up by name.

func (Schema) Validate 

func (s Schema) Validate(r Record) error

Validate checks a Record against the schema: every declared field must be present with the declared Kind and must pass its Rules, and the record must carry no field the schema does not declare. It returns the first violation, so the message is the one to surface. A nil error means the row is well-formed.

type Sort 

type Sort struct {
	Field string
	Desc  bool
}

Sort is one ordering key: order by Field ascending, or descending when Desc. A Query's Sorts apply in order, the first being the primary key.

type Store 

type Store[R any] struct {
	// contains filtered or unexported fields
}

Store is the typed, bindable collection at the top of the spine. It holds a Query, runs it against a Proxy (local or remote — Store neither knows nor cares), and mirrors the resulting page of rows into an mvvm.ObservableList[R] so a view re-renders itself when the data changes. Sort/filter/group/page/ aggregate all live in the Query; Load re-materialises the list from the proxy.

func NewStore 

func NewStore[R any](proxy Proxy, codec Codec[R]) *Store[R]

NewStore builds a Store over proxy using codec to (de)serialise rows. Its Query starts empty (every row, no grouping/paging); set one with SetQuery.

func (*Store[R]) Add 

func (s *Store[R]) Add(ctx context.Context, row R) error

Add inserts a typed row, then reloads so Items reflects the new data through the current query.

func (*Store[R]) Delete 

func (s *Store[R]) Delete(ctx context.Context, key Value) error

Delete removes the row whose key field equals key, then reloads.

func (*Store[R]) Items 

func (s *Store[R]) Items() *mvvm.ObservableList[R]

Items is the observable list a view binds to; it holds the decoded rows of the last Load (the flattened group rows when the query groups).

func (*Store[R]) Load 

func (s *Store[R]) Load(ctx context.Context) (View, error)

Load runs the current query against the proxy, resets Items to the decoded page rows (group rows in order when grouped), and returns the full View so a caller can also read Total, Groups and Aggregates. On a proxy error Items is left unchanged.

func (*Store[R]) Query 

func (s *Store[R]) Query() Query

Query returns the current query.

func (*Store[R]) SetQuery 

func (s *Store[R]) SetQuery(q Query)

SetQuery replaces the query used by the next Load.

func (*Store[R]) Update 

func (s *Store[R]) Update(ctx context.Context, row R) error

Update replaces the row sharing this row's key, then reloads.

type Value 

type Value struct {
	Kind  Kind
	Str   string
	Int   int64
	Float float64
	Bool  bool
}

Value is one typed scalar cell. It is a tagged union kept comparable (no slices or maps) so it can be a map key, sorted, and compared by ==. Only the field selected by Kind is meaningful; the others hold their zero value.

func Bool 

func Bool(b bool) Value

Bool makes a KindBool Value.

func Float 

func Float(f float64) Value

Float makes a KindFloat Value.

func Int 

func Int(i int64) Value

Int makes a KindInt Value.

func String 

func String(s string) Value

String makes a KindString Value.

type View 

type View struct {
	Rows       []Record
	Groups     []Group
	Total      int
	Aggregates map[string]Value
}

View is the result of applying a Query. When the query is ungrouped, Rows is the requested page and Groups is nil; when it is grouped, Groups is the page of groups and Rows is nil. Total is the number of rows that matched the filter (before paging), and Aggregates holds the query's aggregates over that whole filtered set.

func Apply 

func Apply(records []Record, q Query) View

Apply runs a Query over records and returns the resulting View. It is a pure function of its inputs — no clock, no map-order dependence, no mutation of records — which is exactly why a MemoryProxy and a remote grpcproxy can share it and produce byte-identical Views: the client and the server call the same Apply on the same rows with the same Query.

The pipeline is: filter, then compute the grand aggregates over the filtered set, then sort, then either group (and paginate the groups) or paginate the rows. Sorting before grouping keeps each group's rows in the query's order and makes the group order itself deterministic.

Source Files

View all Source files

Directories

Path Synopsis
Package grpcproxy carries the data spine over gRPC: a Server that serves any data.Proxy, and a Client (itself a data.Proxy) that reaches it.
 Package grpcproxy carries the data spine over gRPC: a Server that serves any data.Proxy, and a Client (itself a data.Proxy) that reaches it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.