Documentation ¶
Index ¶
- Constants
- Variables
- func GenStrategyDecremental(typ Type, seed int) interface{}
- func GenStrategyIncremental(typ Type, seed int) interface{}
- func GenStrategyRandom(typ Type, seed int) interface{}
- func GenStrategyRandomDecremental(typ Type, seed int) interface{}
- func GenStrategyRandomIncremental(typ Type, seed int) interface{}
- func ToBoolean(input interface{}) (output bool, ok bool)
- func ToFloat64(input interface{}) (output float64, ok bool)
- func ToInt64(input interface{}) (output int64, ok bool)
- func ToString(input interface{}) (output string, ok bool)
- type Bow
- func AppendBows(bows ...Bow) (Bow, error)
- func NewBow(series ...Series) (Bow, error)
- func NewBowEmpty() Bow
- func NewBowFromColBasedInterfaces(colNames []string, colTypes []Type, colBasedData [][]interface{}) (Bow, error)
- func NewBowFromParquet(path string, verbose bool) (Bow, error)
- func NewBowFromRowBasedInterfaces(colNames []string, colTypes []Type, rowBasedData [][]interface{}) (Bow, error)
- func NewBowWithMetadata(metadata Metadata, series ...Series) (Bow, error)
- func NewGenBow(numRows int, options ...GenSeriesOptions) (Bow, error)
- type Buffer
- func (b *Buffer) GetValue(i int) interface{}
- func (b Buffer) IsNull(rowIndex int) bool
- func (b Buffer) IsSorted() bool
- func (b Buffer) IsValid(rowIndex int) bool
- func (b Buffer) Len() int
- func (b Buffer) Less(i, j int) bool
- func (b *Buffer) SetOrDrop(i int, value interface{})
- func (b *Buffer) SetOrDropStrict(i int, value interface{})
- func (b Buffer) Swap(i, j int)
- type CommonRows
- type GenSeriesOptions
- type GenStrategy
- type JSONBow
- type JSONSchema
- type Metadata
- type RowCmp
- type Series
- type Type
Examples ¶
Constants ¶
const ( // Unknown is placed first to be the default when allocating Type or []Type. Unknown = Type(iota) // Float64 and following types are native arrow type supported by bow. Float64 Int64 Boolean String // InputDependent is used in aggregations when the output type is dependent on the input type. InputDependent // IteratorDependent is used in aggregations when the output type is dependent on the iterator type. IteratorDependent )
Variables ¶
var ErrColTimeUnitNotFound = errors.New("column time unit not found in parquet metadata")
Functions ¶
func GenStrategyDecremental ¶ added in v0.16.0
GenStrategyDecremental generates a number of type `typ` equal to the opposite of the converted `seed` value.
func GenStrategyIncremental ¶ added in v0.16.0
GenStrategyIncremental generates a number of type `typ` equal to the converted `seed` value.
func GenStrategyRandom ¶ added in v0.16.0
GenStrategyRandom generates a random number of type `typ`.
func GenStrategyRandomDecremental ¶ added in v0.16.0
GenStrategyRandomDecremental generates a random number of type `typ` by using the `seed` value.
func GenStrategyRandomIncremental ¶ added in v0.16.0
GenStrategyRandomIncremental generates a random number of type `typ` by using the `seed` value.
func ToBoolean ¶ added in v0.16.0
ToBoolean attempts to convert `input` to bool. Return also a false boolean if the conversion failed. In case of numeric type, returns true if the value is non-zero.
func ToFloat64 ¶
ToFloat64 attempts to convert `input` to float64. Return also a false boolean if the conversion failed.
Types ¶
type Bow ¶
type Bow interface { String() string Schema() *arrow.Schema ArrowRecord() *arrow.Record ColumnName(colIndex int) string NumRows() int NumCols() int ColumnType(colIndex int) Type ColumnIndex(colName string) (int, error) NewBufferFromCol(colIndex int) Buffer NewSeriesFromCol(colIndex int) Series Metadata() Metadata WithMetadata(metadata Metadata) Bow SetMetadata(key, value string) Bow GetRow(rowIndex int) map[string]interface{} GetRowsChan() <-chan map[string]interface{} GetValue(colIndex, rowIndex int) interface{} GetPrevValue(colIndex, rowIndex int) (value interface{}, resRowIndex int) GetNextValue(colIndex, rowIndex int) (value interface{}, resRowIndex int) GetPrevValues(colIndex1, colIndex2, rowIndex int) (value1, value2 interface{}, resRowIndex int) GetNextValues(colIndex1, colIndex2, rowIndex int) (value1, value2 interface{}, resRowIndex int) GetPrevRowIndex(colIndex, rowIndex int) int GetNextRowIndex(colIndex, rowIndex int) int GetInt64(colIndex, rowIndex int) (value int64, valid bool) GetPrevInt64(colIndex, rowIndex int) (value int64, resRowIndex int) GetNextInt64(colIndex, rowIndex int) (value int64, resRowIndex int) GetFloat64(colIndex, rowIndex int) (value float64, valid bool) GetPrevFloat64(colIndex, rowIndex int) (value float64, resRowIndex int) GetNextFloat64(colIndex, rowIndex int) (value float64, resRowIndex int) GetPrevFloat64s(colIndex1, colIndex2, rowIndex int) (value1, value2 float64, resRowIndex int) GetNextFloat64s(colIndex1, colIndex2, rowIndex int) (value1, value2 float64, resRowIndex int) Distinct(colIndex int) Bow Find(columnIndex int, value interface{}) int FindNext(columnIndex, rowIndex int, value interface{}) int Contains(columnIndex int, value interface{}) bool Filter(fns ...RowCmp) Bow MakeFilterValues(colIndex int, values ...interface{}) RowCmp AddCols(newCols ...Series) (Bow, error) RenameCol(colIndex int, newName string) (Bow, error) Apply(colIndex int, returnType Type, fn func(interface{}) interface{}) (Bow, error) Convert(colIndex int, t Type) (Bow, error) InnerJoin(other Bow) Bow OuterJoin(other Bow) Bow Diff(colIndices ...int) (Bow, error) NewSlice(i, j int) Bow Select(colIndices ...int) (Bow, error) NewEmptySlice() Bow DropNils(colIndices ...int) (Bow, error) SortByCol(colIndex int) (Bow, error) FillPrevious(colIndices ...int) (Bow, error) FillNext(colIndices ...int) (Bow, error) FillMean(colIndices ...int) (Bow, error) FillLinear(refColIndex, toFillColIndex int) (Bow, error) Equal(other Bow) bool IsColEmpty(colIndex int) bool IsColSorted(colIndex int) bool MarshalJSON() (buf []byte, err error) UnmarshalJSON(data []byte) error NewValuesFromJSON(jsonB JSONBow) error WriteParquet(path string, verbose bool) error GetParquetMetaColTimeUnit(colIndex int) (time.Duration, error) }
Bow is wrapping the Apache Arrow arrow.Record interface, which is a collection of equal-length arrow.Array matching a particular arrow.Schema. Its purpose is to add convenience methods to easily manipulate dataframes.
func AppendBows ¶
AppendBows attempts to append bows with equal schemas. Different schemas will lead to undefined behavior. Resulting metadata is copied from the first bow.
func NewBow ¶
NewBow returns a new Bow from one or more Series.
Example ¶
b, err := NewBow( NewSeries("col1", Int64, []int64{1, 2, 3, 4}, nil), NewSeries("col2", Float64, []float64{1.1, 2.2, 3.3, 4}, []bool{true, false, true, true}), NewSeries("col3", Boolean, []bool{true, false, true, false}, []bool{true, false, true, true}), ) if err != nil { panic(err) } fmt.Println(b)
Output: col1:int64 col2:float64 col3:bool 1 1.1 true 2 <nil> <nil> 3 3.3 true 4 4 false
func NewBowFromColBasedInterfaces ¶ added in v0.7.3
func NewBowFromColBasedInterfaces(colNames []string, colTypes []Type, colBasedData [][]interface{}) (Bow, error)
NewBowFromColBasedInterfaces returns a new Bow:
- colNames contains the Series names
- colTypes contains the Series data types, optional (if nil, the types will be automatically seeked)
- colBasedData contains the data itself as a two-dimensional slice, with the first dimension being the columns (colNames and colBasedData need to be of the same size)
Example ¶
colNames := []string{"time", "value", "valueFromJSON"} colTypes := make([]Type, len(colNames)) colTypes[0] = Int64 colBasedData := [][]interface{}{ {1, 1.2, json.Number("3")}, {1, json.Number("1.2"), 3}, {json.Number("1.1"), 2, 1.3}, } b, err := NewBowFromColBasedInterfaces(colNames, colTypes, colBasedData) if err != nil { panic(err) } fmt.Println(b)
Output: time:int64 value:int64 valueFromJSON:float64 1 1 1.1 1 <nil> 2 3 3 1.3
func NewBowFromParquet ¶ added in v0.12.0
NewBowFromParquet loads a parquet object from the file path, returning a new Bow. Only value columns are used to create the new Bow. Argument verbose is used to print information about the file loaded.
func NewBowFromRowBasedInterfaces ¶
func NewBowFromRowBasedInterfaces(colNames []string, colTypes []Type, rowBasedData [][]interface{}) (Bow, error)
NewBowFromRowBasedInterfaces returns a new Bow:
- colNames contains the Series names
- colTypes contains the Series data types, required
- rowBasedData contains the data itself as a two-dimensional slice, with the first dimension being the rows (colNames and rowBasedData need to be of the same size)
Example ¶
colNames := []string{"time", "value", "valueFromJSON"} colTypes := []Type{Int64, Int64, Float64} rowBasedData := [][]interface{}{ {1, 1, json.Number("1.1")}, {1.2, json.Number("1.2"), 2}, {json.Number("3"), 3, 1.3}, } b, err := NewBowFromRowBasedInterfaces(colNames, colTypes, rowBasedData) if err != nil { panic(err) } fmt.Println(b)
Output: time:int64 value:int64 valueFromJSON:float64 1 1 1.1 1 <nil> 2 3 3 1.3
func NewBowWithMetadata ¶ added in v0.12.0
NewBowWithMetadata returns a new Bow from Metadata and Series.
type Buffer ¶
type Buffer struct { Data interface{} DataType Type // contains filtered or unexported fields }
Buffer is a mutable data structure with the purpose of easily building data Series with: - Data: slice of data. - DataType: type of the data. - nullBitmapBytes: slice of bytes representing valid or null values.
func NewBufferFromInterfaces ¶
NewBufferFromInterfaces returns a new typed Buffer with the data represented as a slice of interface{}, with eventual nil values.
func (Buffer) IsSorted ¶ added in v0.17.0
IsSorted returns true if the values of the Buffer are sorted in ascending order.
func (Buffer) IsValid ¶ added in v0.16.0
IsValid return true if the value at row `rowIndex` is valid.
func (Buffer) Len ¶ added in v0.16.0
Len returns the size of the underlying slice of data in the Buffer.
func (*Buffer) SetOrDrop ¶
SetOrDrop sets the Buffer data at index `i` by attempting to convert `value` to its DataType. Sets the value to nil if the conversion failed or if `value` is nil.
func (*Buffer) SetOrDropStrict ¶ added in v0.16.0
SetOrDropStrict sets the Buffer data at index `i` by attempting a type assertion of `value` to its DataType. Sets the value to nil if the assertion failed or if `value` is nil.
type CommonRows ¶ added in v0.16.0
type CommonRows struct {
// contains filtered or unexported fields
}
type GenSeriesOptions ¶ added in v0.16.0
type GenSeriesOptions struct { NumRows int Name string Type Type GenStrategy GenStrategy MissingData bool }
GenSeriesOptions are options to generate random Series: - NumRows: number of rows of the resulting Series - Name: name of the Series - Type: data type of the Series - GenStrategy: strategy of data generation - MissingData: sets whether the Series includes random nil values
type GenStrategy ¶ added in v0.16.0
GenStrategy defines how random values are generated.
type JSONBow ¶ added in v0.9.0
type JSONBow struct { Schema JSONSchema `json:"schema"` RowBasedData []map[string]interface{} `json:"data"` }
JSONBow is a structure representing a Bow for JSON marshaling purpose.
func NewJSONBow ¶ added in v0.9.0
NewJSONBow returns a new JSONBow structure from a Bow.
type JSONSchema ¶ added in v0.9.0
type JSONSchema struct {
Fields []jsonField `json:"fields"`
}
type Metadata ¶ added in v0.12.0
Metadata is wrapping arrow.Metadata.
func NewMetadata ¶ added in v0.12.0
NewMetadata returns a new Metadata.
type RowCmp ¶ added in v0.17.0
RowCmp implementation is required for Filter passing full dataset multidimensional comparators implementations, cross column for instance index argument is the current row to compare
type Series ¶
Series is wrapping the Apache Arrow arrow.Array interface, with the addition of a name. It represents an immutable sequence of values using the Arrow in-memory format.
func NewGenSeries ¶ added in v0.16.0
func NewGenSeries(o GenSeriesOptions) Series
NewGenSeries returns a new randomly generated Series.
func NewSeries ¶
NewSeries returns a new Series from: - name: string - typ: Bow data Type - dataArray: slice of the data - validityArray:
- if nil, the data will be non-nil
- can be of type []bool or []byte to represent nil values
func NewSeriesFromBuffer ¶ added in v0.16.0
NewSeriesFromBuffer returns a new Series from a name and a Buffer.
func NewSeriesFromInterfaces ¶
NewSeriesFromInterfaces returns a new Series from: - name: string - typ: Bow Type - data: represented by a slice of interface{}, with eventually nil values
type Type ¶
type Type int
func (Type) IsSupported ¶ added in v0.8.0
IsSupported ensures that the Type t is currently supported by Bow and matches a convertible concrete type.