functions

package
v0.0.0-...-f5b6858 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 3, 2019 License: MIT Imports: 40 Imported by: 9

Documentation

Overview

Package functions is a collection of built-in functions that are callable in the flux query processor. While flux may be extended at runtime by writing function expressions, there are some limitations for which a built-in function is necessary, such as the need for custom data structures, stateful procedures, complex looping and branching, and connection to external services. Another reason for implementing a built-in function is to provide a function that is broadly applicable for many users (e.g., sum() or max()).

The functions package is rarely accessed as a direct API. Rather, the query processing engine accepts named registrations for various interfaces implemented within the functions package and executes them generically using an API that is common to all functions. The registration process is executed by running the init() function in each function file, and is then finalized by importing package builtin, which itself imports the functions package and runs a final setup routine that finalizes installation of the builtin functions to the query processor.

Because of this design, a built-in function implementation consists of a bundle of different interface implementations that are required at various phases of query execution. These phases are query, plan and execute. The query phase is for identifying initializing the parameter sets for each function, and initializing the internal representation of a query. The plan phase is for creating a final execution plan, and the execution phase is for physically accessing the data and computing a final result.

The query phase takes each function call in a query and performs a match against the registry to see if there are type definitions for a built-in operation. If matched, it will instantiate the correct query.OperationSpec type for that function, given the runtime parameters. If a builtin OperationSpec is not found, then it will check for functions defined at runtime, and otherwise return an error. The following registrations are typically executed in the function's init() for the query phase to execute properly:

query.RegisterFunction(name string, c query.CreateOperationSpec, sig semantic.FunctionSignature)
query.RegisterOpSpec(k query.OperationKind, c query.NewOperationSpec)

In the plan phase, an operation spec must be converted to a plan.ProcedureSpec. A query plan must know what operations to carry out, including the function names and parameters. In the trivial case, the OperationSpec and ProcedureSpec have identical fields and the operation spec may be encapsulated as part of the procedure spec. The base interface for a plan.ProcedureSpec requires a Kind() function, as well as a Copy() function which should perform a deep copy of the object. Refer to the following interfaces for more information about designing a procedure spec:

plan.ProcedureSpec
plan.PushDownProcedureSpec
plan.BoundedProcedureSpec
plan.YieldProcedureSpec
plan.AggregateProcedureSpec
plan.ParentAwareProcedureSpec

Once you have determined the interface(s) that must be implemented for your function, you register them with

plan.RegisterProcedureSpec(k ProcedureKind, c CreateProcedureSpec, qks ...query.OperationKind)

The registration in this phase creates two lookups. First, it creates a named lookup in a similar fashion as for OperationSpecs in the query phase. Second, it creates a mapping from OperationSpec types to ProcedureSpec types so that the collection of OperationSpecs for the query can be quickly converted to corresponding Procedure specs. One feature to note is that the registration takes a list of query.OperationSpec values. This is because several user-facing query functions may map to the same internal procedure.

The primary function of the plan phase is to re-order, re-write and possibly combine the operations described in the incoming query in order to improve the performance of the query execution. The planner has two primary operations for doing this: Pushdowns and ReWrites.

A push down operation is a planning technique for pushing the logic from one operation into another so that only a single composite function needs to be called instead of two simpler function call. A pushdown is implemented by implementing the plan.PushDownProcedureSpec interface, which requires functions that define the rules and methods for executing a pushdown operation.

A Rewrite rule is used to modify one or more ProcedureSpecs in cases where redundant or complementary operations can be combined to get a simpler result. Similar to a pushdown operation, the rewrite is triggered whenever certain rules apply. Rewrite rules are implemented differently and require a separate registration:

plan.RegisterRewriteRule(r RewriteRule)

Which in turn requires an implementation of plan.RewriteRule.

Finally, the execute phase is tasked with executing the specific data processing algorithm for the function. A function implementation registers an implementation of the execute.Transformation interface that implements functions that control how the execution engine will take an input table, apply the function, and produce an output table. A transformation implementation is registered via:

execute.RegisterTransformation(k plan.ProcedureKind, c execute.CreateTransformation)

The registration will record a mapping of the procedure's kind to the given transformation type.

In addition to implementing the transformation type, a number of helper types and functions are provided that facilitate the transformation process:

execute.Administration
execute.Dataset
execute.TableBuilderCache
execute.TableBuilder
execute.NewAggregateTransformationAndDataset
execute.NewRowSelectorTransformationAndDataset
query.Table
query.GroupKey
query.ColMeta
query.ColReader

The most important part of a function implementation is for the interface method execute.Transformation.Process(id execute.DatasetID, tbl query.Table). While the full details of how to implement this function are out of the scope of this document, a typical implementation will do the following: 1. Validate the incoming table schema if needed 2. Construct the column and group key schema for the output table via the table builder. 3. Process the incoming table via query.Table.Do, and use the input data to determine the output rows for the table builder. 4. Add rows to the output table.

Finally, there is a special class of functions do not receive an input table from another function's output. In other words, these transformations do not have a parent process that supplies it with table data. These transformation functions are referred to as sources, and naturally implement a connection to a data source (e.g. influxdb, prometheus, csvFile, etc.). They are registered using:

execute.RegisterSource(k plan.ProcedureKind, c execute.CreateSource)

The substantial part of a source implementation is its Run method, which should connect to the data source, collect its data into query.Table structures, and apply any transformations associated with the source.

Package functions contains the implementations for the builtin functions.

Index

Constants

View Source
const (
	ToHTTPKind           = "toHTTP"
	DefaultToHTTPTimeout = 1 * time.Second
)
View Source
const CountKind = "count"
View Source
const CovarianceKind = "covariance"
View Source
const CumulativeSumKind = "cumulativeSum"
View Source
const DedupKind = "dedup"
View Source
const DedupSortCol = "_time"
View Source
const DefaultUpperBoundColumnLabel = "le"
View Source
const DerivativeKind = "derivative"
View Source
const DifferenceKind = "difference"
View Source
const DistinctKind = "distinct"
View Source
const DropKind = "drop"
View Source
const DuplicateKind = "duplicate"
View Source
const ExactPercentileAggKind = "exact-percentile-aggregate"
View Source
const ExactPercentileSelectKind = "exact-percentile-selector"
View Source
const FilterKind = "filter"
View Source
const FirstKind = "first"
View Source
const FromCSVKind = "fromCSV"
View Source
const FromKind = "from"
View Source
const GroupKind = "group"
View Source
const HistogramKind = "histogram"
View Source
const HistogramQuantileKind = "histogramQuantile"
View Source
const IntegralKind = "integral"
View Source
const JoinKind = "join"
View Source
const KeepKind = "keep"
View Source
const KeysKind = "keys"
View Source
const LastKind = "last"
View Source
const LimitKind = "limit"
View Source
const MapKind = "map"
View Source
const MaxKind = "max"
View Source
const MeanKind = "mean"
View Source
const MergeJoinKind = "merge-join"
View Source
const MinKind = "min"
View Source
const PercentileKind = "percentile"
View Source
const PivotKind = "pivot"
View Source
const PredictLinearKind = "predictLinear"
View Source
const RangeKind = "range"
View Source
const RenameKind = "rename"
View Source
const SampleKind = "sample"
View Source
const SchemaMutationKind = "SchemaMutation"

The base kind for SchemaMutations

View Source
const SetKind = "set"
View Source
const ShiftKind = "shift"
View Source
const SkewKind = "skew"
View Source
const SortKind = "sort"
View Source
const SpreadKind = "spread"

SpreadKind is the registration name for Flux, query, plan, and execution.

View Source
const StateTrackingKind = "stateTracking"
View Source
const StddevKind = "stddev"
View Source
const SumKind = "sum"
View Source
const (
	// ToKafkaKind is the Kind for the ToKafka Flux function
	ToKafkaKind = "toKafka"
)
View Source
const UnionKind = "union"
View Source
const UniqueKind = "unique"
View Source
const WindowKind = "window"
View Source
const YieldKind = "yield"

Variables

View Source
var DefaultKafkaWriterFactory = func(conf kafka.WriterConfig) KafkaWriter {
	return kafka.NewWriter(conf)
}

DefaultKafkaWriterFactory is a terrible name for a way to make a kafkaWriter that is injectable for testing

View Source
var DefaultToHTTPUserAgent = "fluxd/dev"

DefaultToHTTPUserAgent is the default user agent used by ToHttp

View Source
var Registrars = []MutationRegistrar{
	{
		Kind: RenameKind,
		Args: map[string]semantic.Type{
			"columns": semantic.Object,
			"fn":      semantic.Function,
		},
		Create: createRenameOpSpec,
		New:    newRenameOp,
	},
	{
		Kind: DropKind,
		Args: map[string]semantic.Type{
			"columns": semantic.NewArrayType(semantic.String),
			"fn":      semantic.Function,
		},
		Create: createDropOpSpec,
		New:    newDropOp,
	},
	{
		Kind: KeepKind,
		Args: map[string]semantic.Type{
			"columns": semantic.NewArrayType(semantic.String),
			"fn":      semantic.Function,
		},
		Create: createKeepOpSpec,
		New:    newKeepOp,
	},
	{
		Kind: DuplicateKind,
		Args: map[string]semantic.Type{
			"column": semantic.String,
			"as":     semantic.String,
		},
		Create: createDuplicateOpSpec,
		New:    newDuplicateOp,
	},
}

A list of all MutationRegistrars to register. To register a new mutation, add an entry to this list.

View Source
var SchemaMutationOps = []query.OperationKind{}

A list of all operations which should map to the SchemaMutationProcedure Added to dynamically upon calls to `Register()`

View Source
var ToHTTPSignature = query.DefaultFunctionSignature()
View Source
var ToKafkaSignature = query.DefaultFunctionSignature()

Functions

func InjectFromDependencies

func InjectFromDependencies(depsMap execute.Dependencies, deps storage.Dependencies) error

func NewCumulativeSumTransformation

func NewCumulativeSumTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *CumulativeSumProcedureSpec) *cumulativeSumTransformation

func NewDedupTransformation

func NewDedupTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *DedupProcedureSpec) *dedupTransformation

func NewDerivativeTransformation

func NewDerivativeTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *DerivativeProcedureSpec) *derivativeTransformation

func NewDifferenceTransformation

func NewDifferenceTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *DifferenceProcedureSpec) *differenceTransformation

func NewDistinctTransformation

func NewDistinctTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *DistinctProcedureSpec) *distinctTransformation

func NewFilterTransformation

func NewFilterTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *FilterProcedureSpec) (*filterTransformation, error)

func NewFixedWindowTransformation

func NewFixedWindowTransformation(
	d execute.Dataset,
	cache execute.TableBuilderCache,
	bounds execute.Bounds,
	w execute.Window,
	timeCol,
	startColLabel,
	stopColLabel string,
	createEmpty bool,
) execute.Transformation

func NewGroupTransformation

func NewGroupTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *GroupProcedureSpec) *groupTransformation

func NewHistogramTransformation

func NewHistogramTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *HistogramProcedureSpec) *histogramTransformation

func NewIntegralTransformation

func NewIntegralTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *IntegralProcedureSpec) *integralTransformation

func NewKeysTransformation

func NewKeysTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *KeysProcedureSpec) *keysTransformation

func NewLimitTransformation

func NewLimitTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *LimitProcedureSpec) *limitTransformation

func NewMapTransformation

func NewMapTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *MapProcedureSpec) (*mapTransformation, error)

func NewMergeJoinTransformation

func NewMergeJoinTransformation(d execute.Dataset, cache *MergeJoinCache, spec *MergeJoinProcedureSpec, parents []execute.DatasetID, tableNames map[execute.DatasetID]string) *mergeJoinTransformation

func NewPivotTransformation

func NewPivotTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *PivotProcedureSpec) *pivotTransformation

func NewRangeTransformation

func NewRangeTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *RangeProcedureSpec, absolute execute.Bounds) (*rangeTransformation, error)

func NewSchemaMutationTransformation

func NewSchemaMutationTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec plan.ProcedureSpec) (*schemaMutationTransformation, error)

func NewShiftTransformation

func NewShiftTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *ShiftProcedureSpec) *shiftTransformation

func NewSortTransformation

func NewSortTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *SortProcedureSpec) *sortTransformation

func NewStateTrackingTransformation

func NewStateTrackingTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *StateTrackingProcedureSpec) (*stateTrackingTransformation, error)

func NewUniqueTransformation

func NewUniqueTransformation(d execute.Dataset, cache execute.TableBuilderCache, spec *UniqueProcedureSpec) *uniqueTransformation

func SystemTime

func SystemTime() values.Value

SystemTime return a function value that when called will give the current system time

Types

type AggregateGroupRewriteRule

type AggregateGroupRewriteRule struct {
}

func (AggregateGroupRewriteRule) Rewrite

func (AggregateGroupRewriteRule) Root

type BuilderContext

type BuilderContext struct {
	TableColumns []query.ColMeta
	TableKey     query.GroupKey
	ColIdxMap    []int
}

func NewBuilderContext

func NewBuilderContext(tbl query.Table) *BuilderContext

func (*BuilderContext) ColMap

func (b *BuilderContext) ColMap() []int

func (*BuilderContext) Cols

func (b *BuilderContext) Cols() []query.ColMeta

func (*BuilderContext) Key

func (b *BuilderContext) Key() query.GroupKey

type CSVSource

type CSVSource struct {
	// contains filtered or unexported fields
}

func (*CSVSource) AddTransformation

func (c *CSVSource) AddTransformation(t execute.Transformation)

func (*CSVSource) Run

func (c *CSVSource) Run(ctx context.Context)

type CountAgg

type CountAgg struct {
	// contains filtered or unexported fields
}

func (*CountAgg) DoBool

func (a *CountAgg) DoBool(vs []bool)

func (*CountAgg) DoFloat

func (a *CountAgg) DoFloat(vs []float64)

func (*CountAgg) DoInt

func (a *CountAgg) DoInt(vs []int64)

func (*CountAgg) DoString

func (a *CountAgg) DoString(vs []string)

func (*CountAgg) DoUInt

func (a *CountAgg) DoUInt(vs []uint64)

func (*CountAgg) NewBoolAgg

func (a *CountAgg) NewBoolAgg() execute.DoBoolAgg

func (*CountAgg) NewFloatAgg

func (a *CountAgg) NewFloatAgg() execute.DoFloatAgg

func (*CountAgg) NewIntAgg

func (a *CountAgg) NewIntAgg() execute.DoIntAgg

func (*CountAgg) NewStringAgg

func (a *CountAgg) NewStringAgg() execute.DoStringAgg

func (*CountAgg) NewUIntAgg

func (a *CountAgg) NewUIntAgg() execute.DoUIntAgg

func (*CountAgg) Type

func (a *CountAgg) Type() query.DataType

func (*CountAgg) ValueInt

func (a *CountAgg) ValueInt() int64

type CountOpSpec

type CountOpSpec struct {
	execute.AggregateConfig
}

func (*CountOpSpec) Kind

func (s *CountOpSpec) Kind() query.OperationKind

type CountProcedureSpec

type CountProcedureSpec struct {
	execute.AggregateConfig
}

func (*CountProcedureSpec) AggregateMethod

func (s *CountProcedureSpec) AggregateMethod() string

func (*CountProcedureSpec) Copy

func (*CountProcedureSpec) Kind

func (*CountProcedureSpec) PushDown

func (s *CountProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*CountProcedureSpec) PushDownRules

func (s *CountProcedureSpec) PushDownRules() []plan.PushDownRule

func (*CountProcedureSpec) ReAggregateSpec

func (s *CountProcedureSpec) ReAggregateSpec() plan.ProcedureSpec

type CovarianceOpSpec

type CovarianceOpSpec struct {
	PearsonCorrelation bool   `json:"pearsonr"`
	ValueDst           string `json:"valueDst"`
	execute.AggregateConfig
}

func (*CovarianceOpSpec) Kind

type CovarianceProcedureSpec

type CovarianceProcedureSpec struct {
	PearsonCorrelation bool
	ValueLabel         string
	execute.AggregateConfig
}

func (*CovarianceProcedureSpec) Copy

func (*CovarianceProcedureSpec) Kind

type CovarianceTransformation

type CovarianceTransformation struct {
	// contains filtered or unexported fields
}

func (*CovarianceTransformation) DoFloat

func (t *CovarianceTransformation) DoFloat(xs, ys []float64)

func (*CovarianceTransformation) Finish

func (t *CovarianceTransformation) Finish(id execute.DatasetID, err error)

func (*CovarianceTransformation) Process

func (*CovarianceTransformation) RetractTable

func (t *CovarianceTransformation) RetractTable(id execute.DatasetID, key query.GroupKey) error

func (*CovarianceTransformation) UpdateProcessingTime

func (t *CovarianceTransformation) UpdateProcessingTime(id execute.DatasetID, pt execute.Time) error

func (*CovarianceTransformation) UpdateWatermark

func (t *CovarianceTransformation) UpdateWatermark(id execute.DatasetID, mark execute.Time) error

type CumulativeSumOpSpec

type CumulativeSumOpSpec struct {
	Columns []string `json:"columns"`
}

func (*CumulativeSumOpSpec) Kind

type CumulativeSumProcedureSpec

type CumulativeSumProcedureSpec struct {
	Columns []string
}

func (*CumulativeSumProcedureSpec) Copy

func (*CumulativeSumProcedureSpec) Kind

type DedupOpSpec

type DedupOpSpec struct {
}

func (*DedupOpSpec) Kind

func (s *DedupOpSpec) Kind() query.OperationKind

type DedupProcedureSpec

type DedupProcedureSpec struct {
	Column string
}

func (*DedupProcedureSpec) Copy

func (*DedupProcedureSpec) Kind

type DerivativeOpSpec

type DerivativeOpSpec struct {
	Unit        query.Duration `json:"unit"`
	NonNegative bool           `json:"nonNegative"`
	Columns     []string       `json:"columns"`
	TimeSrc     string         `json:"timeSrc"`
}

func (*DerivativeOpSpec) Kind

type DerivativeProcedureSpec

type DerivativeProcedureSpec struct {
	Unit        query.Duration `json:"unit"`
	NonNegative bool           `json:"non_negative"`
	Columns     []string       `json:"columns"`
	TimeCol     string         `json:"time_col"`
}

func (*DerivativeProcedureSpec) Copy

func (*DerivativeProcedureSpec) Kind

type DifferenceOpSpec

type DifferenceOpSpec struct {
	NonNegative bool     `json:"nonNegative"`
	Columns     []string `json:"columns"`
}

func (*DifferenceOpSpec) Kind

type DifferenceProcedureSpec

type DifferenceProcedureSpec struct {
	NonNegative bool     `json:"non_negative"`
	Columns     []string `json:"columns"`
}

func (*DifferenceProcedureSpec) Copy

func (*DifferenceProcedureSpec) Kind

type DistinctOpSpec

type DistinctOpSpec struct {
	Column string `json:"column"`
}

func (*DistinctOpSpec) Kind

type DistinctPointLimitRewriteRule

type DistinctPointLimitRewriteRule struct {
}

func (DistinctPointLimitRewriteRule) Rewrite

func (DistinctPointLimitRewriteRule) Root

type DistinctProcedureSpec

type DistinctProcedureSpec struct {
	Column string
}

func (*DistinctProcedureSpec) Copy

func (*DistinctProcedureSpec) Kind

type DropKeepMutator

type DropKeepMutator struct {
	KeepCols      map[string]bool
	DropCols      map[string]bool
	Predicate     compiler.Func
	FlipPredicate bool
	ParamName     string
	Scope         map[string]values.Value
}

func NewDropKeepMutator

func NewDropKeepMutator(qs query.OperationSpec) (*DropKeepMutator, error)

func (*DropKeepMutator) Mutate

func (m *DropKeepMutator) Mutate(ctx *BuilderContext) error

type DropOpSpec

type DropOpSpec struct {
	Cols      []string                     `json:"columns"`
	Predicate *semantic.FunctionExpression `json:"fn"`
}

func (*DropOpSpec) Copy

func (s *DropOpSpec) Copy() SchemaMutation

func (*DropOpSpec) Kind

func (s *DropOpSpec) Kind() query.OperationKind

func (*DropOpSpec) Mutator

func (s *DropOpSpec) Mutator() (SchemaMutator, error)

type DuplicateMutator

type DuplicateMutator struct {
	Col string
	As  string
}

func NewDuplicateMutator

func NewDuplicateMutator(qs query.OperationSpec) (*DuplicateMutator, error)

func (*DuplicateMutator) Mutate

func (m *DuplicateMutator) Mutate(ctx *BuilderContext) error

type DuplicateOpSpec

type DuplicateOpSpec struct {
	Col string `json:"columns"`
	As  string `json:"as"`
}

func (*DuplicateOpSpec) Copy

func (s *DuplicateOpSpec) Copy() SchemaMutation

func (*DuplicateOpSpec) Kind

func (*DuplicateOpSpec) Mutator

func (s *DuplicateOpSpec) Mutator() (SchemaMutator, error)

type ExactPercentileAgg

type ExactPercentileAgg struct {
	Quantile float64
	// contains filtered or unexported fields
}

func (*ExactPercentileAgg) Copy

func (*ExactPercentileAgg) DoFloat

func (a *ExactPercentileAgg) DoFloat(vs []float64)

func (*ExactPercentileAgg) NewBoolAgg

func (a *ExactPercentileAgg) NewBoolAgg() execute.DoBoolAgg

func (*ExactPercentileAgg) NewFloatAgg

func (a *ExactPercentileAgg) NewFloatAgg() execute.DoFloatAgg

func (*ExactPercentileAgg) NewIntAgg

func (a *ExactPercentileAgg) NewIntAgg() execute.DoIntAgg

func (*ExactPercentileAgg) NewStringAgg

func (a *ExactPercentileAgg) NewStringAgg() execute.DoStringAgg

func (*ExactPercentileAgg) NewUIntAgg

func (a *ExactPercentileAgg) NewUIntAgg() execute.DoUIntAgg

func (*ExactPercentileAgg) Type

func (a *ExactPercentileAgg) Type() query.DataType

func (*ExactPercentileAgg) ValueFloat

func (a *ExactPercentileAgg) ValueFloat() float64

type ExactPercentileAggProcedureSpec

type ExactPercentileAggProcedureSpec struct {
	Percentile float64 `json:"percentile"`
	execute.AggregateConfig
}

func (*ExactPercentileAggProcedureSpec) Copy

func (*ExactPercentileAggProcedureSpec) Kind

type ExactPercentileSelectProcedureSpec

type ExactPercentileSelectProcedureSpec struct {
	Percentile float64 `json:"percentile"`
	execute.SelectorConfig
}

func (*ExactPercentileSelectProcedureSpec) Copy

func (*ExactPercentileSelectProcedureSpec) Kind

type ExactPercentileSelectorTransformation

type ExactPercentileSelectorTransformation struct {
	// contains filtered or unexported fields
}

func (*ExactPercentileSelectorTransformation) Finish

func (*ExactPercentileSelectorTransformation) Process

func (*ExactPercentileSelectorTransformation) RetractTable

func (*ExactPercentileSelectorTransformation) UpdateProcessingTime

func (t *ExactPercentileSelectorTransformation) UpdateProcessingTime(id execute.DatasetID, pt execute.Time) error

func (*ExactPercentileSelectorTransformation) UpdateWatermark

type FilterOpSpec

type FilterOpSpec struct {
	Fn *semantic.FunctionExpression `json:"fn"`
}

func (*FilterOpSpec) Kind

func (s *FilterOpSpec) Kind() query.OperationKind

type FilterProcedureSpec

type FilterProcedureSpec struct {
	Fn *semantic.FunctionExpression
}

func (*FilterProcedureSpec) Copy

func (*FilterProcedureSpec) Kind

func (*FilterProcedureSpec) PushDown

func (s *FilterProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*FilterProcedureSpec) PushDownRules

func (s *FilterProcedureSpec) PushDownRules() []plan.PushDownRule

type FirstOpSpec

type FirstOpSpec struct {
	execute.SelectorConfig
}

func (*FirstOpSpec) Kind

func (s *FirstOpSpec) Kind() query.OperationKind

type FirstProcedureSpec

type FirstProcedureSpec struct {
	execute.SelectorConfig
}

func (*FirstProcedureSpec) Copy

func (*FirstProcedureSpec) Kind

func (*FirstProcedureSpec) PushDown

func (s *FirstProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*FirstProcedureSpec) PushDownRules

func (s *FirstProcedureSpec) PushDownRules() []plan.PushDownRule

type FirstSelector

type FirstSelector struct {
	// contains filtered or unexported fields
}

func (*FirstSelector) DoBool

func (s *FirstSelector) DoBool(vs []bool) []int

func (*FirstSelector) DoFloat

func (s *FirstSelector) DoFloat(vs []float64) []int

func (*FirstSelector) DoInt

func (s *FirstSelector) DoInt(vs []int64) []int

func (*FirstSelector) DoString

func (s *FirstSelector) DoString(vs []string) []int

func (*FirstSelector) DoUInt

func (s *FirstSelector) DoUInt(vs []uint64) []int

func (*FirstSelector) NewBoolSelector

func (s *FirstSelector) NewBoolSelector() execute.DoBoolIndexSelector

func (*FirstSelector) NewFloatSelector

func (s *FirstSelector) NewFloatSelector() execute.DoFloatIndexSelector

func (*FirstSelector) NewIntSelector

func (s *FirstSelector) NewIntSelector() execute.DoIntIndexSelector

func (*FirstSelector) NewStringSelector

func (s *FirstSelector) NewStringSelector() execute.DoStringIndexSelector

func (*FirstSelector) NewUIntSelector

func (s *FirstSelector) NewUIntSelector() execute.DoUIntIndexSelector

type FromCSVOpSpec

type FromCSVOpSpec struct {
	CSV  string `json:"csv"`
	File string `json:"file"`
}

func (*FromCSVOpSpec) Kind

func (s *FromCSVOpSpec) Kind() query.OperationKind

type FromCSVProcedureSpec

type FromCSVProcedureSpec struct {
	CSV  string
	File string
}

func (*FromCSVProcedureSpec) Copy

func (*FromCSVProcedureSpec) Kind

type FromOpSpec

type FromOpSpec struct {
	Bucket   string      `json:"bucket,omitempty"`
	BucketID platform.ID `json:"bucketID,omitempty"`
}

func (*FromOpSpec) BucketsAccessed

func (s *FromOpSpec) BucketsAccessed() (readBuckets, writeBuckets []platform.BucketFilter)

func (*FromOpSpec) Kind

func (s *FromOpSpec) Kind() query.OperationKind

type FromProcedureSpec

type FromProcedureSpec struct {
	Bucket   string
	BucketID platform.ID

	BoundsSet bool
	Bounds    query.Bounds

	FilterSet bool
	Filter    *semantic.FunctionExpression

	DescendingSet bool
	Descending    bool

	LimitSet     bool
	PointsLimit  int64
	SeriesLimit  int64
	SeriesOffset int64

	WindowSet bool
	Window    plan.WindowSpec

	GroupingSet bool
	OrderByTime bool
	GroupMode   GroupMode
	GroupKeys   []string

	AggregateSet    bool
	AggregateMethod string
}

func (*FromProcedureSpec) Copy

func (*FromProcedureSpec) Kind

func (*FromProcedureSpec) TimeBounds

func (s *FromProcedureSpec) TimeBounds() query.Bounds

type GroupMode

type GroupMode int
const (
	// GroupModeDefault will use the default grouping of GroupModeAll.
	GroupModeDefault GroupMode = 0

	// GroupModeNone merges all series into a single group.
	GroupModeNone GroupMode = 1 << iota
	// GroupModeAll produces a separate table for each series.
	GroupModeAll
	// GroupModeBy produces a table for each unique value of the specified GroupKeys.
	GroupModeBy
	// GroupModeExcept produces a table for the unique values of all keys, except those specified by GroupKeys.
	GroupModeExcept
)

type GroupOpSpec

type GroupOpSpec struct {
	By     []string `json:"by"`
	Except []string `json:"except"`
	All    bool     `json:"all"`
	None   bool     `json:"none"`
}

func (*GroupOpSpec) Kind

func (s *GroupOpSpec) Kind() query.OperationKind

type GroupProcedureSpec

type GroupProcedureSpec struct {
	GroupMode GroupMode
	GroupKeys []string
}

func (*GroupProcedureSpec) Copy

func (*GroupProcedureSpec) Kind

func (*GroupProcedureSpec) PushDown

func (s *GroupProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*GroupProcedureSpec) PushDownRules

func (s *GroupProcedureSpec) PushDownRules() []plan.PushDownRule

type HistogramOpSpec

type HistogramOpSpec struct {
	Column           string    `json:"column"`
	UpperBoundColumn string    `json:"upperBoundColumn"`
	CountColumn      string    `json:"countColumn"`
	Buckets          []float64 `json:"buckets"`
	Normalize        bool      `json:"normalize"`
}

func (*HistogramOpSpec) Kind

type HistogramProcedureSpec

type HistogramProcedureSpec struct {
	HistogramOpSpec
}

func (*HistogramProcedureSpec) Copy

func (*HistogramProcedureSpec) Kind

type HistogramQuantileOpSpec

type HistogramQuantileOpSpec struct {
	Quantile         float64 `json:"quantile"`
	CountColumn      string  `json:"countColumn"`
	UpperBoundColumn string  `json:"upperBoundColumn"`
	ValueColumn      string  `json:"valueColumn"`
	MinValue         float64 `json:"minValue"`
}

func (*HistogramQuantileOpSpec) Kind

type HistogramQuantileProcedureSpec

type HistogramQuantileProcedureSpec struct {
	Quantile         float64 `json:"quantile"`
	CountColumn      string  `json:"countColumn"`
	UpperBoundColumn string  `json:"upperBoundColumn"`
	ValueColumn      string  `json:"valueColumn"`
	MinValue         float64 `json:"minValue"`
}

func (*HistogramQuantileProcedureSpec) Copy

func (*HistogramQuantileProcedureSpec) Kind

type IntegralOpSpec

type IntegralOpSpec struct {
	Unit query.Duration `json:"unit"`
	execute.AggregateConfig
}

func (*IntegralOpSpec) Kind

type IntegralProcedureSpec

type IntegralProcedureSpec struct {
	Unit query.Duration `json:"unit"`
	execute.AggregateConfig
}

func (*IntegralProcedureSpec) Copy

func (*IntegralProcedureSpec) Kind

type JoinOpSpec

type JoinOpSpec struct {
	// On is a list of tags on which to join.
	On []string `json:"on"`
	// TableNames are the names to give to each parent when populating the parameter for the function.
	// The first parent is referenced by the first name and so forth.
	// TODO(nathanielc): Change this to a map of parent operation IDs to names.
	// Then make it possible for the transformation to map operation IDs to parent IDs.
	TableNames map[query.OperationID]string `json:"tableNames"`
	// Method is a the type of join to perform
	Method string `json:"method"`
	// contains filtered or unexported fields
}

func (*JoinOpSpec) IDer

func (t *JoinOpSpec) IDer(ider query.IDer)

func (*JoinOpSpec) Kind

func (s *JoinOpSpec) Kind() query.OperationKind

type KafkaWriter

type KafkaWriter interface {
	io.Closer
	WriteMessages(context.Context, ...kafka.Message) error
}

KafkaWriter is an interface for what we need fromDefaultKafkaWriterFactory

type KeepOpSpec

type KeepOpSpec struct {
	Cols      []string                     `json:"columns"`
	Predicate *semantic.FunctionExpression `json:"fn"`
}

func (*KeepOpSpec) Copy

func (s *KeepOpSpec) Copy() SchemaMutation

func (*KeepOpSpec) Kind

func (s *KeepOpSpec) Kind() query.OperationKind

func (*KeepOpSpec) Mutator

func (s *KeepOpSpec) Mutator() (SchemaMutator, error)

type KeysOpSpec

type KeysOpSpec struct {
	Except []string `json:"except"`
}

func (*KeysOpSpec) Kind

func (s *KeysOpSpec) Kind() query.OperationKind

type KeysPointLimitRewriteRule

type KeysPointLimitRewriteRule struct {
}

func (KeysPointLimitRewriteRule) Rewrite

func (KeysPointLimitRewriteRule) Root

type KeysProcedureSpec

type KeysProcedureSpec struct {
	Except []string
}

func (*KeysProcedureSpec) Copy

func (*KeysProcedureSpec) Kind

type LastOpSpec

type LastOpSpec struct {
	execute.SelectorConfig
}

func (*LastOpSpec) Kind

func (s *LastOpSpec) Kind() query.OperationKind

type LastProcedureSpec

type LastProcedureSpec struct {
	execute.SelectorConfig
}

func (*LastProcedureSpec) Copy

func (*LastProcedureSpec) Kind

func (*LastProcedureSpec) PushDown

func (s *LastProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*LastProcedureSpec) PushDownRules

func (s *LastProcedureSpec) PushDownRules() []plan.PushDownRule

type LastSelector

type LastSelector struct {
	// contains filtered or unexported fields
}

func (*LastSelector) DoBool

func (s *LastSelector) DoBool(vs []bool, cr query.ColReader)

func (*LastSelector) DoFloat

func (s *LastSelector) DoFloat(vs []float64, cr query.ColReader)

func (*LastSelector) DoInt

func (s *LastSelector) DoInt(vs []int64, cr query.ColReader)

func (*LastSelector) DoString

func (s *LastSelector) DoString(vs []string, cr query.ColReader)

func (*LastSelector) DoUInt

func (s *LastSelector) DoUInt(vs []uint64, cr query.ColReader)

func (*LastSelector) NewBoolSelector

func (s *LastSelector) NewBoolSelector() execute.DoBoolRowSelector

func (*LastSelector) NewFloatSelector

func (s *LastSelector) NewFloatSelector() execute.DoFloatRowSelector

func (*LastSelector) NewIntSelector

func (s *LastSelector) NewIntSelector() execute.DoIntRowSelector

func (*LastSelector) NewStringSelector

func (s *LastSelector) NewStringSelector() execute.DoStringRowSelector

func (*LastSelector) NewUIntSelector

func (s *LastSelector) NewUIntSelector() execute.DoUIntRowSelector

func (*LastSelector) Rows

func (s *LastSelector) Rows() []execute.Row

type LimitOpSpec

type LimitOpSpec struct {
	N      int64 `json:"n"`
	Offset int64 `json:"offset"`
}

LimitOpSpec limits the number of rows returned per table. Currently offset is not supported.

func (*LimitOpSpec) Kind

func (s *LimitOpSpec) Kind() query.OperationKind

type LimitProcedureSpec

type LimitProcedureSpec struct {
	N      int64 `json:"n"`
	Offset int64 `json:"offset"`
}

func (*LimitProcedureSpec) Copy

func (*LimitProcedureSpec) Kind

func (*LimitProcedureSpec) PushDown

func (s *LimitProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*LimitProcedureSpec) PushDownRules

func (s *LimitProcedureSpec) PushDownRules() []plan.PushDownRule

type MapOpSpec

type MapOpSpec struct {
	Fn       *semantic.FunctionExpression `json:"fn"`
	MergeKey bool                         `json:"mergeKey"`
}

func (*MapOpSpec) Kind

func (s *MapOpSpec) Kind() query.OperationKind

type MapProcedureSpec

type MapProcedureSpec struct {
	Fn       *semantic.FunctionExpression
	MergeKey bool
}

func (*MapProcedureSpec) Copy

func (*MapProcedureSpec) Kind

type MaxFloatSelector

type MaxFloatSelector struct {
	MaxSelector
	// contains filtered or unexported fields
}

func (*MaxFloatSelector) DoFloat

func (s *MaxFloatSelector) DoFloat(vs []float64, cr query.ColReader)

type MaxIntSelector

type MaxIntSelector struct {
	MaxSelector
	// contains filtered or unexported fields
}

func (*MaxIntSelector) DoInt

func (s *MaxIntSelector) DoInt(vs []int64, cr query.ColReader)

type MaxOpSpec

type MaxOpSpec struct {
	execute.SelectorConfig
}

func (*MaxOpSpec) Kind

func (s *MaxOpSpec) Kind() query.OperationKind

type MaxProcedureSpec

type MaxProcedureSpec struct {
	execute.SelectorConfig
}

func (*MaxProcedureSpec) Copy

func (*MaxProcedureSpec) Kind

type MaxSelector

type MaxSelector struct {
	// contains filtered or unexported fields
}

func (*MaxSelector) NewBoolSelector

func (s *MaxSelector) NewBoolSelector() execute.DoBoolRowSelector

func (*MaxSelector) NewFloatSelector

func (s *MaxSelector) NewFloatSelector() execute.DoFloatRowSelector

func (*MaxSelector) NewIntSelector

func (s *MaxSelector) NewIntSelector() execute.DoIntRowSelector

func (*MaxSelector) NewStringSelector

func (s *MaxSelector) NewStringSelector() execute.DoStringRowSelector

func (*MaxSelector) NewUIntSelector

func (s *MaxSelector) NewUIntSelector() execute.DoUIntRowSelector

func (*MaxSelector) Rows

func (s *MaxSelector) Rows() []execute.Row

type MaxUIntSelector

type MaxUIntSelector struct {
	MaxSelector
	// contains filtered or unexported fields
}

func (*MaxUIntSelector) DoUInt

func (s *MaxUIntSelector) DoUInt(vs []uint64, cr query.ColReader)

type MeanAgg

type MeanAgg struct {
	// contains filtered or unexported fields
}

func (*MeanAgg) DoFloat

func (a *MeanAgg) DoFloat(vs []float64)

func (*MeanAgg) DoInt

func (a *MeanAgg) DoInt(vs []int64)

func (*MeanAgg) DoUInt

func (a *MeanAgg) DoUInt(vs []uint64)

func (*MeanAgg) NewBoolAgg

func (a *MeanAgg) NewBoolAgg() execute.DoBoolAgg

func (*MeanAgg) NewFloatAgg

func (a *MeanAgg) NewFloatAgg() execute.DoFloatAgg

func (*MeanAgg) NewIntAgg

func (a *MeanAgg) NewIntAgg() execute.DoIntAgg

func (*MeanAgg) NewStringAgg

func (a *MeanAgg) NewStringAgg() execute.DoStringAgg

func (*MeanAgg) NewUIntAgg

func (a *MeanAgg) NewUIntAgg() execute.DoUIntAgg

func (*MeanAgg) Type

func (a *MeanAgg) Type() query.DataType

func (*MeanAgg) ValueFloat

func (a *MeanAgg) ValueFloat() float64

type MeanOpSpec

type MeanOpSpec struct {
	execute.AggregateConfig
}

func (*MeanOpSpec) Kind

func (s *MeanOpSpec) Kind() query.OperationKind

type MeanProcedureSpec

type MeanProcedureSpec struct {
	execute.AggregateConfig
}

func (*MeanProcedureSpec) Copy

func (*MeanProcedureSpec) Kind

type MergeJoinCache

type MergeJoinCache struct {
	// contains filtered or unexported fields
}

MergeJoinCache implements execute.DataCache This is where the all the tables to be joined are stored.

buffers: Buffers to hold the tables for each incoming stream.

postJoinKeys: The post-join group keys for all joined tables.

These group keys are constructed and stored as soon
as a table is consumed by the join operator, but prior
to actually joining the data.

reverseLookup: Each output group key that is stored is mapped to its

corresponding pre-join group keys. These pre-join group
keys are then used to retrieve their correspoinding
tables from the buffers.

tables: All output tables are materialized and stored in this

map before being sent to downstream operators.

func NewMergeJoinCache

func NewMergeJoinCache(alloc *execute.Allocator, datasetIDs []execute.DatasetID, tableNames map[execute.DatasetID]string, key []string) *MergeJoinCache

NewMergeJoinCache constructs a new instance of a MergeJoinCache

func (*MergeJoinCache) DiscardTable

func (c *MergeJoinCache) DiscardTable(key query.GroupKey)

DiscardTable removes a table from the output buffer

func (*MergeJoinCache) ExpireTable

func (c *MergeJoinCache) ExpireTable(key query.GroupKey)

ExpireTable removes the a key from the set of postJoinKeys. ExpireTable will be called after the table associated with key has already been materialized. As a result, it cannot not be materialized again. Each buffer is cleared of any stale data that arises as a result of this process.

func (*MergeJoinCache) ForEach

func (c *MergeJoinCache) ForEach(f func(query.GroupKey))

ForEach iterates over each table in the output stream

func (*MergeJoinCache) ForEachWithContext

func (c *MergeJoinCache) ForEachWithContext(f func(query.GroupKey, execute.Trigger, execute.TableContext))

ForEachWithContext iterates over each table in the output stream

func (*MergeJoinCache) SetTriggerSpec

func (c *MergeJoinCache) SetTriggerSpec(spec query.TriggerSpec)

SetTriggerSpec sets the trigger rule for this cache

func (*MergeJoinCache) Table

func (c *MergeJoinCache) Table(key query.GroupKey) (query.Table, error)

Table joins the two tables associated with a single output group key and returns the resulting table

type MergeJoinProcedureSpec

type MergeJoinProcedureSpec struct {
	On         []string                    `json:"keys"`
	TableNames map[plan.ProcedureID]string `json:"table_names"`
}

func (*MergeJoinProcedureSpec) Copy

func (*MergeJoinProcedureSpec) Kind

func (*MergeJoinProcedureSpec) ParentChanged

func (s *MergeJoinProcedureSpec) ParentChanged(old, new plan.ProcedureID)

type MinFloatSelector

type MinFloatSelector struct {
	MinSelector
	// contains filtered or unexported fields
}

func (*MinFloatSelector) DoFloat

func (s *MinFloatSelector) DoFloat(vs []float64, cr query.ColReader)

type MinIntSelector

type MinIntSelector struct {
	MinSelector
	// contains filtered or unexported fields
}

func (*MinIntSelector) DoInt

func (s *MinIntSelector) DoInt(vs []int64, cr query.ColReader)

type MinOpSpec

type MinOpSpec struct {
	execute.SelectorConfig
}

func (*MinOpSpec) Kind

func (s *MinOpSpec) Kind() query.OperationKind

type MinProcedureSpec

type MinProcedureSpec struct {
	execute.SelectorConfig
}

func (*MinProcedureSpec) Copy

func (*MinProcedureSpec) Kind

type MinSelector

type MinSelector struct {
	// contains filtered or unexported fields
}

func (*MinSelector) NewBoolSelector

func (s *MinSelector) NewBoolSelector() execute.DoBoolRowSelector

func (*MinSelector) NewFloatSelector

func (s *MinSelector) NewFloatSelector() execute.DoFloatRowSelector

func (*MinSelector) NewIntSelector

func (s *MinSelector) NewIntSelector() execute.DoIntRowSelector

func (*MinSelector) NewStringSelector

func (s *MinSelector) NewStringSelector() execute.DoStringRowSelector

func (*MinSelector) NewUIntSelector

func (s *MinSelector) NewUIntSelector() execute.DoUIntRowSelector

func (*MinSelector) Rows

func (s *MinSelector) Rows() []execute.Row

type MinUIntSelector

type MinUIntSelector struct {
	MinSelector
	// contains filtered or unexported fields
}

func (*MinUIntSelector) DoUInt

func (s *MinUIntSelector) DoUInt(vs []uint64, cr query.ColReader)

type MutationRegistrar

type MutationRegistrar struct {
	Kind   query.OperationKind
	Args   map[string]semantic.Type
	Create query.CreateOperationSpec
	New    query.NewOperationSpec
}

A MutationRegistrar contains information needed to register a type of Operation Spec that will be converted into a SchemaMutator and embedded in a SchemaMutationProcedureSpec. Operations with a corresponding MutationRegistrar should not have their own ProcedureSpec.

func (MutationRegistrar) Register

func (m MutationRegistrar) Register()

type PercentileAgg

type PercentileAgg struct {
	Quantile,
	Compression float64
	// contains filtered or unexported fields
}

func (*PercentileAgg) Copy

func (a *PercentileAgg) Copy() *PercentileAgg

func (*PercentileAgg) DoFloat

func (a *PercentileAgg) DoFloat(vs []float64)

func (*PercentileAgg) NewBoolAgg

func (a *PercentileAgg) NewBoolAgg() execute.DoBoolAgg

func (*PercentileAgg) NewFloatAgg

func (a *PercentileAgg) NewFloatAgg() execute.DoFloatAgg

func (*PercentileAgg) NewIntAgg

func (a *PercentileAgg) NewIntAgg() execute.DoIntAgg

func (*PercentileAgg) NewStringAgg

func (a *PercentileAgg) NewStringAgg() execute.DoStringAgg

func (*PercentileAgg) NewUIntAgg

func (a *PercentileAgg) NewUIntAgg() execute.DoUIntAgg

func (*PercentileAgg) Type

func (a *PercentileAgg) Type() query.DataType

func (*PercentileAgg) ValueFloat

func (a *PercentileAgg) ValueFloat() float64

type PercentileOpSpec

type PercentileOpSpec struct {
	Percentile  float64 `json:"percentile"`
	Compression float64 `json:"compression"`
	Method      string  `json:"method"`
	// percentile is either an aggregate, or a selector based on the options
	execute.AggregateConfig
	execute.SelectorConfig
}

func (*PercentileOpSpec) Kind

type PivotOpSpec

type PivotOpSpec struct {
	RowKey   []string `json:"rowKey"`
	ColKey   []string `json:"colKey"`
	ValueCol string   `json:"valueCol"`
}

func (*PivotOpSpec) Kind

func (s *PivotOpSpec) Kind() query.OperationKind

type PivotProcedureSpec

type PivotProcedureSpec struct {
	RowKey   []string
	ColKey   []string
	ValueCol string
}

func (*PivotProcedureSpec) Copy

func (*PivotProcedureSpec) Kind

type PredictLinearOpSpec

type PredictLinearOpSpec struct {
	ValueDst    string  `json:"value_dst"`
	WantedValue float64 `json:"wanted_value"`
	execute.AggregateConfig
}

func (*PredictLinearOpSpec) Kind

type PredictLinearProcedureSpec

type PredictLinearProcedureSpec struct {
	ValueLabel  string
	WantedValue float64
	execute.AggregateConfig
}

func (*PredictLinearProcedureSpec) Copy

func (*PredictLinearProcedureSpec) Kind

type PredictLinearTransformation

type PredictLinearTransformation struct {
	// contains filtered or unexported fields
}

func (*PredictLinearTransformation) DoFloat

func (t *PredictLinearTransformation) DoFloat(ys []float64, xs []values.Time)

func (*PredictLinearTransformation) DoInt

func (t *PredictLinearTransformation) DoInt(ys []int64, xs []values.Time)

func (*PredictLinearTransformation) Finish

func (*PredictLinearTransformation) Process

func (*PredictLinearTransformation) RetractTable

func (*PredictLinearTransformation) UpdateProcessingTime

func (t *PredictLinearTransformation) UpdateProcessingTime(id execute.DatasetID, pt execute.Time) error

func (*PredictLinearTransformation) UpdateWatermark

func (t *PredictLinearTransformation) UpdateWatermark(id execute.DatasetID, mark execute.Time) error

type RangeOpSpec

type RangeOpSpec struct {
	Start    query.Time `json:"start"`
	Stop     query.Time `json:"stop"`
	TimeCol  string     `json:"timeCol"`
	StartCol string     `json:"startCol"`
	StopCol  string     `json:"stopCol"`
}

func (*RangeOpSpec) Kind

func (s *RangeOpSpec) Kind() query.OperationKind

type RangeProcedureSpec

type RangeProcedureSpec struct {
	Bounds   query.Bounds
	TimeCol  string
	StartCol string
	StopCol  string
}

func (*RangeProcedureSpec) Copy

func (*RangeProcedureSpec) Kind

func (*RangeProcedureSpec) PushDown

func (s *RangeProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*RangeProcedureSpec) PushDownRules

func (s *RangeProcedureSpec) PushDownRules() []plan.PushDownRule

func (*RangeProcedureSpec) TimeBounds

func (s *RangeProcedureSpec) TimeBounds() query.Bounds

type RenameMutator

type RenameMutator struct {
	Cols      map[string]string
	Fn        compiler.Func
	Scope     map[string]values.Value
	ParamName string
}

func NewRenameMutator

func NewRenameMutator(qs query.OperationSpec) (*RenameMutator, error)

func (*RenameMutator) Mutate

func (m *RenameMutator) Mutate(ctx *BuilderContext) error

type RenameOpSpec

type RenameOpSpec struct {
	Cols map[string]string            `json:"columns"`
	Fn   *semantic.FunctionExpression `json:"fn"`
}

func (*RenameOpSpec) Copy

func (s *RenameOpSpec) Copy() SchemaMutation

func (*RenameOpSpec) Kind

func (s *RenameOpSpec) Kind() query.OperationKind

func (*RenameOpSpec) Mutator

func (s *RenameOpSpec) Mutator() (SchemaMutator, error)

type SampleOpSpec

type SampleOpSpec struct {
	N   int64 `json:"n"`
	Pos int64 `json:"pos"`
	execute.SelectorConfig
}

func (*SampleOpSpec) Kind

func (s *SampleOpSpec) Kind() query.OperationKind

type SampleProcedureSpec

type SampleProcedureSpec struct {
	N   int64
	Pos int64
	execute.SelectorConfig
}

func (*SampleProcedureSpec) Copy

func (*SampleProcedureSpec) Kind

type SampleSelector

type SampleSelector struct {
	N   int
	Pos int
	// contains filtered or unexported fields
}

func (*SampleSelector) DoBool

func (s *SampleSelector) DoBool(vs []bool) []int

func (*SampleSelector) DoFloat

func (s *SampleSelector) DoFloat(vs []float64) []int

func (*SampleSelector) DoInt

func (s *SampleSelector) DoInt(vs []int64) []int

func (*SampleSelector) DoString

func (s *SampleSelector) DoString(vs []string) []int

func (*SampleSelector) DoUInt

func (s *SampleSelector) DoUInt(vs []uint64) []int

func (*SampleSelector) NewBoolSelector

func (s *SampleSelector) NewBoolSelector() execute.DoBoolIndexSelector

func (*SampleSelector) NewFloatSelector

func (s *SampleSelector) NewFloatSelector() execute.DoFloatIndexSelector

func (*SampleSelector) NewIntSelector

func (s *SampleSelector) NewIntSelector() execute.DoIntIndexSelector

func (*SampleSelector) NewStringSelector

func (s *SampleSelector) NewStringSelector() execute.DoStringIndexSelector

func (*SampleSelector) NewUIntSelector

func (s *SampleSelector) NewUIntSelector() execute.DoUIntIndexSelector

type SchemaMutation

type SchemaMutation interface {
	Mutator() (SchemaMutator, error)
	Copy() SchemaMutation
}

type SchemaMutationProcedureSpec

type SchemaMutationProcedureSpec struct {
	Mutations []SchemaMutation
}

func (*SchemaMutationProcedureSpec) Copy

func (*SchemaMutationProcedureSpec) Kind

type SchemaMutator

type SchemaMutator interface {
	Mutate(ctx *BuilderContext) error
}

type SetOpSpec

type SetOpSpec struct {
	Key   string `json:"key"`
	Value string `json:"value"`
}

func (*SetOpSpec) Kind

func (s *SetOpSpec) Kind() query.OperationKind

type SetProcedureSpec

type SetProcedureSpec struct {
	Key, Value string
}

func (*SetProcedureSpec) Copy

func (*SetProcedureSpec) Kind

type ShiftOpSpec

type ShiftOpSpec struct {
	Shift   query.Duration `json:"shift"`
	Columns []string       `json:"columns"`
}

func (*ShiftOpSpec) Kind

func (s *ShiftOpSpec) Kind() query.OperationKind

type ShiftProcedureSpec

type ShiftProcedureSpec struct {
	Shift   query.Duration
	Columns []string
}

func (*ShiftProcedureSpec) Copy

func (*ShiftProcedureSpec) Kind

type SkewAgg

type SkewAgg struct {
	// contains filtered or unexported fields
}

func (*SkewAgg) DoFloat

func (a *SkewAgg) DoFloat(vs []float64)

func (*SkewAgg) DoInt

func (a *SkewAgg) DoInt(vs []int64)

func (*SkewAgg) DoUInt

func (a *SkewAgg) DoUInt(vs []uint64)

func (*SkewAgg) NewBoolAgg

func (a *SkewAgg) NewBoolAgg() execute.DoBoolAgg

func (*SkewAgg) NewFloatAgg

func (a *SkewAgg) NewFloatAgg() execute.DoFloatAgg

func (*SkewAgg) NewIntAgg

func (a *SkewAgg) NewIntAgg() execute.DoIntAgg

func (*SkewAgg) NewStringAgg

func (a *SkewAgg) NewStringAgg() execute.DoStringAgg

func (*SkewAgg) NewUIntAgg

func (a *SkewAgg) NewUIntAgg() execute.DoUIntAgg

func (*SkewAgg) Type

func (a *SkewAgg) Type() query.DataType

func (*SkewAgg) ValueFloat

func (a *SkewAgg) ValueFloat() float64

type SkewOpSpec

type SkewOpSpec struct {
	execute.AggregateConfig
}

func (*SkewOpSpec) Kind

func (s *SkewOpSpec) Kind() query.OperationKind

type SkewProcedureSpec

type SkewProcedureSpec struct {
	execute.AggregateConfig
}

func (*SkewProcedureSpec) Copy

func (*SkewProcedureSpec) Kind

type SortOpSpec

type SortOpSpec struct {
	Cols []string `json:"cols"`
	Desc bool     `json:"desc"`
}

func (*SortOpSpec) Kind

func (s *SortOpSpec) Kind() query.OperationKind

type SortProcedureSpec

type SortProcedureSpec struct {
	Cols []string
	Desc bool
}

func (*SortProcedureSpec) Copy

func (*SortProcedureSpec) Kind

type SpreadAgg

type SpreadAgg struct {
	// contains filtered or unexported fields
}

SpreadAgg finds the difference between the max and min values a table

func (*SpreadAgg) NewBoolAgg

func (a *SpreadAgg) NewBoolAgg() execute.DoBoolAgg

func (*SpreadAgg) NewFloatAgg

func (a *SpreadAgg) NewFloatAgg() execute.DoFloatAgg

func (*SpreadAgg) NewIntAgg

func (a *SpreadAgg) NewIntAgg() execute.DoIntAgg

func (*SpreadAgg) NewStringAgg

func (a *SpreadAgg) NewStringAgg() execute.DoStringAgg

func (*SpreadAgg) NewUIntAgg

func (a *SpreadAgg) NewUIntAgg() execute.DoUIntAgg

type SpreadFloatAgg

type SpreadFloatAgg struct {
	SpreadAgg
	// contains filtered or unexported fields
}

func (*SpreadFloatAgg) DoFloat

func (a *SpreadFloatAgg) DoFloat(vs []float64)

Do searches for the min and max value of the array and caches them in the aggregate

func (*SpreadFloatAgg) Type

func (a *SpreadFloatAgg) Type() query.DataType

func (*SpreadFloatAgg) ValueFloat

func (a *SpreadFloatAgg) ValueFloat() float64

Value returns the difference between max and min

type SpreadIntAgg

type SpreadIntAgg struct {
	SpreadAgg
	// contains filtered or unexported fields
}

func (*SpreadIntAgg) DoInt

func (a *SpreadIntAgg) DoInt(vs []int64)

DoInt searches for the min and max value of the array and caches them in the aggregate

func (*SpreadIntAgg) Type

func (a *SpreadIntAgg) Type() query.DataType

func (*SpreadIntAgg) ValueInt

func (a *SpreadIntAgg) ValueInt() int64

Value returns the difference between max and min

type SpreadOpSpec

type SpreadOpSpec struct {
	execute.AggregateConfig
}

SpreadOpSpec defines the required arguments for Flux. Currently, spread takes no arguments.

func (*SpreadOpSpec) Kind

func (s *SpreadOpSpec) Kind() query.OperationKind

Kind is used to lookup createSpreadOpSpec producing SpreadOpSpec

type SpreadProcedureSpec

type SpreadProcedureSpec struct {
	execute.AggregateConfig
}

SpreadProcedureSpec is created when mapping from SpreadOpSpec.Kind to a CreateProcedureSpec.

func (*SpreadProcedureSpec) Copy

func (*SpreadProcedureSpec) Kind

Kind is used to lookup CreateTransformation producing SpreadAgg

type SpreadUIntAgg

type SpreadUIntAgg struct {
	SpreadAgg
	// contains filtered or unexported fields
}

func (*SpreadUIntAgg) DoUInt

func (a *SpreadUIntAgg) DoUInt(vs []uint64)

Do searches for the min and max value of the array and caches them in the aggregate

func (*SpreadUIntAgg) Type

func (a *SpreadUIntAgg) Type() query.DataType

func (*SpreadUIntAgg) ValueUInt

func (a *SpreadUIntAgg) ValueUInt() uint64

Value returns the difference between max and min

type StateTrackingOpSpec

type StateTrackingOpSpec struct {
	Fn            *semantic.FunctionExpression `json:"fn"`
	CountLabel    string                       `json:"countLabel"`
	DurationLabel string                       `json:"durationLabel"`
	DurationUnit  query.Duration               `json:"durationUnit"`
	TimeCol       string                       `json:"timeCol"`
}

func (*StateTrackingOpSpec) Kind

type StateTrackingProcedureSpec

type StateTrackingProcedureSpec struct {
	Fn *semantic.FunctionExpression
	CountLabel,
	DurationLabel string
	DurationUnit query.Duration
	TimeCol      string
}

func (*StateTrackingProcedureSpec) Copy

func (*StateTrackingProcedureSpec) Kind

type StddevAgg

type StddevAgg struct {
	// contains filtered or unexported fields
}

func (*StddevAgg) DoFloat

func (a *StddevAgg) DoFloat(vs []float64)

func (*StddevAgg) DoInt

func (a *StddevAgg) DoInt(vs []int64)

func (*StddevAgg) DoUInt

func (a *StddevAgg) DoUInt(vs []uint64)

func (*StddevAgg) NewBoolAgg

func (a *StddevAgg) NewBoolAgg() execute.DoBoolAgg

func (*StddevAgg) NewFloatAgg

func (a *StddevAgg) NewFloatAgg() execute.DoFloatAgg

func (*StddevAgg) NewIntAgg

func (a *StddevAgg) NewIntAgg() execute.DoIntAgg

func (*StddevAgg) NewStringAgg

func (a *StddevAgg) NewStringAgg() execute.DoStringAgg

func (*StddevAgg) NewUIntAgg

func (a *StddevAgg) NewUIntAgg() execute.DoUIntAgg

func (*StddevAgg) Type

func (a *StddevAgg) Type() query.DataType

func (*StddevAgg) ValueFloat

func (a *StddevAgg) ValueFloat() float64

type StddevOpSpec

type StddevOpSpec struct {
	execute.AggregateConfig
}

func (*StddevOpSpec) Kind

func (s *StddevOpSpec) Kind() query.OperationKind

type StddevProcedureSpec

type StddevProcedureSpec struct {
	execute.AggregateConfig
}

func (*StddevProcedureSpec) Copy

func (*StddevProcedureSpec) Kind

type SumAgg

type SumAgg struct{}

func (*SumAgg) NewBoolAgg

func (a *SumAgg) NewBoolAgg() execute.DoBoolAgg

func (*SumAgg) NewFloatAgg

func (a *SumAgg) NewFloatAgg() execute.DoFloatAgg

func (*SumAgg) NewIntAgg

func (a *SumAgg) NewIntAgg() execute.DoIntAgg

func (*SumAgg) NewStringAgg

func (a *SumAgg) NewStringAgg() execute.DoStringAgg

func (*SumAgg) NewUIntAgg

func (a *SumAgg) NewUIntAgg() execute.DoUIntAgg

type SumFloatAgg

type SumFloatAgg struct {
	// contains filtered or unexported fields
}

func (*SumFloatAgg) DoFloat

func (a *SumFloatAgg) DoFloat(vs []float64)

func (*SumFloatAgg) Type

func (a *SumFloatAgg) Type() query.DataType

func (*SumFloatAgg) ValueFloat

func (a *SumFloatAgg) ValueFloat() float64

type SumIntAgg

type SumIntAgg struct {
	// contains filtered or unexported fields
}

func (*SumIntAgg) DoInt

func (a *SumIntAgg) DoInt(vs []int64)

func (*SumIntAgg) Type

func (a *SumIntAgg) Type() query.DataType

func (*SumIntAgg) ValueInt

func (a *SumIntAgg) ValueInt() int64

type SumOpSpec

type SumOpSpec struct {
	execute.AggregateConfig
}

func (*SumOpSpec) Kind

func (s *SumOpSpec) Kind() query.OperationKind

type SumProcedureSpec

type SumProcedureSpec struct {
	execute.AggregateConfig
}

func (*SumProcedureSpec) AggregateMethod

func (s *SumProcedureSpec) AggregateMethod() string

func (*SumProcedureSpec) Copy

func (*SumProcedureSpec) Kind

func (*SumProcedureSpec) PushDown

func (s *SumProcedureSpec) PushDown(root *plan.Procedure, dup func() *plan.Procedure)

func (*SumProcedureSpec) PushDownRules

func (s *SumProcedureSpec) PushDownRules() []plan.PushDownRule

func (*SumProcedureSpec) ReAggregateSpec

func (s *SumProcedureSpec) ReAggregateSpec() plan.ProcedureSpec

type SumUIntAgg

type SumUIntAgg struct {
	// contains filtered or unexported fields
}

func (*SumUIntAgg) DoUInt

func (a *SumUIntAgg) DoUInt(vs []uint64)

func (*SumUIntAgg) Type

func (a *SumUIntAgg) Type() query.DataType

func (*SumUIntAgg) ValueUInt

func (a *SumUIntAgg) ValueUInt() uint64

type TDigestPercentileProcedureSpec

type TDigestPercentileProcedureSpec struct {
	Percentile  float64 `json:"percentile"`
	Compression float64 `json:"compression"`
	execute.AggregateConfig
}

func (*TDigestPercentileProcedureSpec) Copy

func (*TDigestPercentileProcedureSpec) Kind

type ToHTTPOpSpec

type ToHTTPOpSpec struct {
	URL          string            `json:"url"`
	Method       string            `json:"method"` // default behavior should be POST
	Name         string            `json:"name"`
	NameColumn   string            `json:"nameColumn"` // either name or name_column must be set, if none is set try to use the "_measurement" column.
	Headers      map[string]string `json:"headers"`    // TODO: implement Headers after bug with keys and arrays and objects is fixed (new parser implemented, with string literals as keys)
	URLParams    map[string]string `json:"urlParams"`  // TODO: implement URLParams after bug with keys and arrays and objects is fixed (new parser implemented, with string literals as keys)
	Timeout      time.Duration     `json:"timeout"`    // default to something reasonable if zero
	NoKeepAlive  bool              `json:"noKeepAlive"`
	TimeColumn   string            `json:"timeColumn"`
	TagColumns   []string          `json:"tagColumns"`
	ValueColumns []string          `json:"valueColumns"`
}

func (ToHTTPOpSpec) Kind

func (*ToHTTPOpSpec) ReadArgs

func (o *ToHTTPOpSpec) ReadArgs(args query.Arguments) error

ReadArgs loads a query.Arguments into ToHTTPOpSpec. It sets several default values. If the http method isn't set, it defaults to POST, it also uppercases the http method. If the time_column isn't set, it defaults to execute.TimeColLabel. If the value_column isn't set it defaults to a []string{execute.DefaultValueColLabel}.

func (*ToHTTPOpSpec) UnmarshalJSON

func (o *ToHTTPOpSpec) UnmarshalJSON(b []byte) (err error)

UnmarshalJSON unmarshals and validates toHTTPOpSpec into JSON.

type ToHTTPProcedureSpec

type ToHTTPProcedureSpec struct {
	Spec *ToHTTPOpSpec
}

func (*ToHTTPProcedureSpec) Copy

func (*ToHTTPProcedureSpec) Kind

type ToHTTPTransformation

type ToHTTPTransformation struct {
	// contains filtered or unexported fields
}

func (*ToHTTPTransformation) Finish

func (t *ToHTTPTransformation) Finish(id execute.DatasetID, err error)

func (*ToHTTPTransformation) Process

func (t *ToHTTPTransformation) Process(id execute.DatasetID, tbl query.Table) error

func (*ToHTTPTransformation) RetractTable

func (t *ToHTTPTransformation) RetractTable(id execute.DatasetID, key query.GroupKey) error

func (*ToHTTPTransformation) UpdateProcessingTime

func (t *ToHTTPTransformation) UpdateProcessingTime(id execute.DatasetID, pt execute.Time) error

func (*ToHTTPTransformation) UpdateWatermark

func (t *ToHTTPTransformation) UpdateWatermark(id execute.DatasetID, pt execute.Time) error

type ToKafkaOpSpec

type ToKafkaOpSpec struct {
	Brokers      []string `json:"brokers"`
	Topic        string   `json:"topic"`
	Balancer     string   `json:"balancer"`
	Name         string   `json:"name"`
	NameColumn   string   `json:"nameColumn"` // either name or name_column must be set, if none is set try to use the "_measurement" column.
	TimeColumn   string   `json:"timeColumn"`
	TagColumns   []string `json:"tagColumns"`
	ValueColumns []string `json:"valueColumns"`
	MsgBufSize   int      `json:"msgBufferSize"` // the maximim number of messages to buffer before sending to kafka, the library we use defaults to 100
}

func (ToKafkaOpSpec) Kind

func (*ToKafkaOpSpec) ReadArgs

func (o *ToKafkaOpSpec) ReadArgs(args query.Arguments) error

ReadArgs loads a query.Arguments into ToKafkaOpSpec. It sets several default values. If the time_column isn't set, it defaults to execute.TimeColLabel. If the value_column isn't set it defaults to a []string{execute.DefaultValueColLabel}.

type ToKafkaProcedureSpec

type ToKafkaProcedureSpec struct {
	Spec *ToKafkaOpSpec
	// contains filtered or unexported fields
}

func (*ToKafkaProcedureSpec) Copy

func (*ToKafkaProcedureSpec) Kind

type ToKafkaTransformation

type ToKafkaTransformation struct {
	// contains filtered or unexported fields
}

func (*ToKafkaTransformation) Finish

func (t *ToKafkaTransformation) Finish(id execute.DatasetID, err error)

func (*ToKafkaTransformation) Process

func (t *ToKafkaTransformation) Process(id execute.DatasetID, tbl query.Table) (err error)

func (*ToKafkaTransformation) RetractTable

func (t *ToKafkaTransformation) RetractTable(id execute.DatasetID, key query.GroupKey) error

func (*ToKafkaTransformation) UpdateProcessingTime

func (t *ToKafkaTransformation) UpdateProcessingTime(id execute.DatasetID, pt execute.Time) error

func (*ToKafkaTransformation) UpdateWatermark

func (t *ToKafkaTransformation) UpdateWatermark(id execute.DatasetID, pt execute.Time) error

type UnionOpSpec

type UnionOpSpec struct {
}

func (*UnionOpSpec) Kind

func (s *UnionOpSpec) Kind() flux.OperationKind

type UnionProcedureSpec

type UnionProcedureSpec struct {
}

func (*UnionProcedureSpec) Copy

func (*UnionProcedureSpec) Kind

type UniqueOpSpec

type UniqueOpSpec struct {
	Column string `json:"column"`
}

func (*UniqueOpSpec) Kind

func (s *UniqueOpSpec) Kind() query.OperationKind

type UniqueProcedureSpec

type UniqueProcedureSpec struct {
	Column string
}

func (*UniqueProcedureSpec) Copy

func (*UniqueProcedureSpec) Kind

type WindowOpSpec

type WindowOpSpec struct {
	Every         query.Duration    `json:"every"`
	Period        query.Duration    `json:"period"`
	Start         query.Time        `json:"start"`
	Round         query.Duration    `json:"round"`
	Triggering    query.TriggerSpec `json:"triggering"`
	TimeCol       string            `json:"time_col"`
	StopColLabel  string            `json:"stop_col_label"`
	StartColLabel string            `json:"start_col_label"`
	CreateEmpty   bool              `json:"createEmpty"`
}

func (*WindowOpSpec) Kind

func (s *WindowOpSpec) Kind() query.OperationKind

type WindowProcedureSpec

type WindowProcedureSpec struct {
	Window     plan.WindowSpec
	Triggering query.TriggerSpec
	TimeCol,
	StartColLabel,
	StopColLabel string
	CreateEmpty bool
}

func (*WindowProcedureSpec) Copy

func (*WindowProcedureSpec) Kind

func (*WindowProcedureSpec) TriggerSpec

func (s *WindowProcedureSpec) TriggerSpec() query.TriggerSpec

type YieldOpSpec

type YieldOpSpec struct {
	Name string `json:"name"`
}

func (*YieldOpSpec) Kind

func (s *YieldOpSpec) Kind() query.OperationKind

type YieldProcedureSpec

type YieldProcedureSpec struct {
	Name string `json:"name"`
}

func (*YieldProcedureSpec) Copy

func (*YieldProcedureSpec) Kind

func (*YieldProcedureSpec) YieldName

func (s *YieldProcedureSpec) YieldName() string

Directories

Path Synopsis
Package storage implements reading from a storage engine into a table as a data source.
Package storage implements reading from a storage engine into a table as a data source.
pb

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL