queries

README

queries

Package queries contains shared types for the reusable query helper packages under github.com/FrogoAI/fdb-client/pkg/queries. It is intentionally small: it defines the event Source abstraction, a map-backed Source implementation, aggregation field flags, storage bin names, and a deprecated aggregate client interface kept for compatibility.

The helper subpackages (timeseries, window, previous, and lsh) define their own small consumer-side client interfaces. A *client.Client satisfies those interfaces, and tests can provide mocks with only the operations used by the helper under test.

Strengths

• Source keeps helper packages reusable with maps, structs, protobuf wrappers, JSON wrappers, or other caller-owned event models.
• Field centralizes aggregation flags, bin names, dependency expansion, and the low-level read/write operations needed by aggregation helpers.
• Helper subpackages depend on small operation-specific client interfaces instead of the legacy broad aggregate interface, which keeps tests and standalone helper usage simple.
• Shared constants and helpers keep SDK behavior aligned across query packages without importing private server modules.

Weaknesses / Tradeoffs

• queries is shared plumbing, not a complete validation layer. Request rules such as TTL, range, and window-size constraints live in the subpackages that own those behaviors.
• MapSource returns zero values for missing keys or type mismatches. That is convenient for permissive ingestion paths, but callers should validate required event fields before relying on aggregates.
• EventTime falls back to time.Now() when event_time is missing or zero, so deterministic bucketing requires callers to provide an explicit event timestamp.
• Field is a package-wide bitmask. Individual helper packages may implement only the fields that make sense for their storage model.
• queries.Client remains for older code, but new code should prefer the smaller interfaces exposed by each helper package.

Source data

Helpers read event values through queries.Source.

src := queries.NewMapSource(map[string]any{
    "event_id":         "evt-123",
    "standard.user_id": "user-42",
    "amount":           42.5,
    "event_time":       time.Now(),
})

userID := src.String("standard.user_id")
amount := src.Float("amount")
_ = userID
_ = amount

NewMapSource accepts map[string]any and returns default zero values when a key is missing or has the wrong type. MapSource.Date accepts time.Time, int64 Unix seconds, and int Unix seconds.

queries.EventTime(src) reads the event_time key via src.Date. If the key is missing or zero, it falls back to time.Now().

Implement Source directly when events already live in a struct, protobuf, JSON wrapper, or other data model:

type Source interface {
    String(key string) string
    Int(key string) int64
    Float(key string) float64
    Bool(key string) bool
    Has(key string) bool
    Date(key string) time.Time
    Value(key string) any
}

Aggregation fields

Field is a bitmask used by timeseries and window helpers to select which aggregations to compute.

fields := queries.FieldCount | queries.FieldAvg | queries.FieldSTD
fields = fields | queries.FieldMin | queries.FieldMax

Supported flags include FieldCount, FieldSum, FieldMin, FieldMax, FieldAvg, FieldSTD, FieldDistinct, FieldPercentile, FieldGMean, FieldCMean, and FieldZScore.

Derived fields expand to the storage fields they need. For example, FieldAvg.Resolve() includes count and sum, while FieldSTD.Resolve() includes count, sum, and sum of squares.

The field helpers are useful when building lower-level operations:

bins := (queries.FieldCount | queries.FieldAvg).RequiredReadBins()
ops := queries.FieldSTD.RequiredWriteOps(42.5, nil)
_ = bins
_ = ops

Using helper packages

Pass a *client.Client directly to a helper. This example records the current event in a 24-hour timeseries and returns the updated aggregation.

func recordAmount(ctx context.Context) error {
    c, err := client.New("127.0.0.1:3000")
    if err != nil {
        return err
    }
    defer c.Close()

    src := queries.NewMapSource(map[string]any{
        "standard.user_id": "user-42",
        "amount":           42.5,
        "event_time":       time.Now(),
    })

    result, err := timeseries.Execute(ctx, c, timeseries.Request{
        Name:           "user_amounts_24h",
        Namespace:      "scoring",
        GroupBy:        []string{"standard.user_id"},
        Range:          24 * time.Hour,
        Fields:         queries.FieldCount | queries.FieldAvg,
        Value:          "amount",
        TTL:            48 * time.Hour,
        IncludeCurrent: true,
    }, src)
    if err != nil {
        return err
    }

    _ = result.Average()
    return nil
}

queries.Client still exists for older code that names the aggregate interface, but new helper code should rely on the smaller interfaces exposed by each subpackage.

Documentation

Index

• Constants
• func EventTime(src Source) time.Time
• type Client
• type Field
  • func (f Field) RequiredBins() []string
  • func (f Field) RequiredReadBins() []string
  • func (f Field) RequiredWriteOps(value float64, distinctBy []byte) []client.Operation
  • func (f Field) Resolve() Field
• type MapSource
  • func NewMapSource(data map[string]any) *MapSource
  • func (s *MapSource) Bool(key string) bool
  • func (s *MapSource) Date(key string) time.Time
  • func (s *MapSource) Float(key string) float64
  • func (s *MapSource) Has(key string) bool
  • func (s *MapSource) Int(key string) int64
  • func (s *MapSource) String(key string) string
  • func (s *MapSource) Value(key string) any
• type Source

Constants

const (
	BinCount     = "count"
	BinValue     = "value"
	BinSumQ      = "sumQ"
	BinMin       = "min"
	BinMax       = "max"
	BinHLL       = "hll"
	BinTDigest   = "tdigest"
	BinSumLog    = "sumLog"
	BinSumCos    = "sumCos"
	BinSumSin    = "sumSin"
	BinTimestamp = "timestamp"
)

Bin name constants used by the query engine.

const EventTimeKey = "event_time"

EventTimeKey is the source key for caller-provided event timestamps.

Functions

func EventTime

func EventTime(src Source) time.Time

EventTime extracts the event timestamp from the source via src.Date(). If the source does not contain "event_time" or the value is zero time, returns time.Now(). This matches the original Estimator pattern where event time comes from the caller, not the database layer.

Types

type Client

type Client interface {
	Get(ctx context.Context, ns, set, key string, binNames ...string) (*client.Record, error)
	Operate(
		ctx context.Context,
		ns, set, key string,
		ops []client.Operation,
		opts ...client.WriteOption,
	) (*client.Record, error)
	BatchGet(ctx context.Context, keys []client.BatchKey, opts ...client.BatchOption) ([]*client.Record, error)
	BatchOperate(
		ctx context.Context,
		records []client.BatchOperate,
		opts ...client.BatchOption,
	) ([]*client.Record, error)
	Delete(ctx context.Context, ns, set, key string, opts ...client.WriteOption) (bool, error)
	LSHDedup(ctx context.Context, ns, group, input string, opts ...client.WriteOption) (string, error)
	LSHVector(ctx context.Context, ns, group string, vector []float64, opts ...client.WriteOption) (string, error)
}

Client is the legacy aggregate client contract for query helpers. *client.Client satisfies this interface.

Deprecated: helper packages define smaller consumer-side interfaces for the exact operations they use. Callers can pass *client.Client directly, and test mocks only need to implement the methods required by the helper under test.

type Field

type Field uint32

Field is a bitmask selecting which aggregation fields to compute. Multiple fields can be combined with bitwise OR.

const (
	FieldCount      Field = 1 << iota // event count (AddOp count bin)
	FieldSum                          // sum of values (AddOp value bin)
	FieldMin                          // minimum (PutOp min bin)
	FieldMax                          // maximum (PutOp max bin)
	FieldAvg                          // average (requires Count + Sum)
	FieldSTD                          // std deviation (requires Count + Sum + SumQ)
	FieldDistinct                     // distinct count (HLL)
	FieldPercentile                   // quantile (TDigest)
	FieldGMean                        // geometric mean (requires Count + SumLog)
	FieldCMean                        // circular mean (requires SumCos + SumSin)
	FieldZScore                       // z-score (requires Avg + STD + current value)
)

Aggregation field selectors. Combine with | for multiple fields.

func (Field) RequiredBins

func (f Field) RequiredBins() []string

RequiredBins returns the storage bin names needed to persist the requested fields. This is the union of all bins required by the resolved field set.

func (Field) RequiredReadBins

func (f Field) RequiredReadBins() []string

RequiredReadBins returns the bin names to request from the server when reading results for the requested fields.

func (Field) RequiredWriteOps

func (f Field) RequiredWriteOps(value float64, distinctBy []byte) []client.Operation

RequiredWriteOps returns the client operations needed to persist one event for the requested fields. value is the numeric event value; distinctBy is the raw bytes for HLL distinct counting (may be nil if FieldDistinct is not requested).

HLL uses FrogoDB default (log2m=14, ~16 KB, 0.81% error). TDigest uses FrogoDB default (compression=100, ~3-4 KB).

func (Field) Resolve

func (f Field) Resolve() Field

Resolve expands derived fields into their dependencies. For example, FieldAvg requires FieldCount and FieldSum.

type MapSource

type MapSource struct {
	// contains filtered or unexported fields
}

MapSource is the default Source backed by map[string]any.

func NewMapSource

func NewMapSource(data map[string]any) *MapSource

NewMapSource creates a Source from a map. Nil maps are treated as empty.

func (*MapSource) Bool

func (s *MapSource) Bool(key string) bool

Bool returns the bool value for key, or false if missing or not a bool.

func (*MapSource) Date

func (s *MapSource) Date(key string) time.Time

Date returns the time.Time value for key, or zero time if missing/wrong type. Handles: time.Time directly, int64 (Unix seconds), int (Unix seconds).

func (*MapSource) Float

func (s *MapSource) Float(key string) float64

Float returns the float64 value for key, or 0 if missing or not numeric.

func (*MapSource) Has

func (s *MapSource) Has(key string) bool

Has returns true if the key exists in the source data.

func (*MapSource) Int

func (s *MapSource) Int(key string) int64

Int returns the int64 value for key, or 0 if missing or not numeric.

func (*MapSource) String

func (s *MapSource) String(key string) string

String returns the string value for key, or "" if missing or not a string.

func (*MapSource) Value

func (s *MapSource) Value(key string) any

Value returns the raw value for key without type coercion, or nil if missing.

type Source

type Source interface {
	String(key string) string
	Int(key string) int64
	Float(key string) float64
	Bool(key string) bool
	Has(key string) bool
	Date(key string) time.Time
	Value(key string) any
}

Source provides read access to event data. Users implement this with their own data structure (flat map, protobuf, JSON, etc.).