records

Documentation

Index

• Constants
• func ExtractLocationIDs(locs []uuid.UUID) []byte
• func ToSnakeCase(str string) string
• type Build
  • func NewBuild[T any](mem memory.Allocator) *Build[T]
  • func (b *Build[T]) Append(values ...T) error
  • func (b *Build[T]) NewRecord() *Record
  • func (b *Build[T]) Release()
  • func (b Build[T]) Schema(name string) (s *schemapb.Schema)
• type Reader
  • func NewReader[T any](records ...arrow.Record) *Reader[T]
  • func (r *Reader[T]) NumRows() int64
  • func (r *Reader[T]) Value(i int) T
• type Record

Constants

const (
	TagName = "frostdb"
)

Functions

func ExtractLocationIDs

func ExtractLocationIDs(locs []uuid.UUID) []byte

func ToSnakeCase

func ToSnakeCase(str string) string

Types

type Build

type Build[T any] struct {
	// contains filtered or unexported fields
}

Build is a generic arrow.Record builder that ingests structs of type T. The generated record can be passed to (*Table).InsertRecord.

Struct tag `frostdb` is used to pass options for the schema for T and use (*Build[T]).Schema to obtain schema v1alpha1.

This api is opinionated.

• Nested Columns are not supported

Tags

Use `frostdb` to define tags that customizes field values. You can express everything needed to construct schema v1alpha1.

Tags are defined as a comma separated list. The first item is the column name. Column name is optional, when omitted it is derived from the field name (snake_cased)

Supported Tags

    delta_binary_packed | Delta binary packed encoding.
                 brotli | Brotli compression.
                    asc | Sorts in ascending order.Use asc(n) where n is an integer for sorting order.
                   gzip | GZIP compression.
                 snappy | Snappy compression.
delta_length_byte_array | Delta Length Byte Array encoding.
       delta_byte_array | Delta Byte Array encoding.
                   desc | Sorts in descending order.Use desc(n) where n is an integer for sorting order
                lz4_raw | LZ4_RAW compression.
               pre_hash | Prehash the column before storing it.
             null_first | When used wit asc nulls are smallest and with des nulls are largest.
                   zstd | ZSTD compression.
               rle_dict | Dictionary run-length encoding.
                  plain | Plain encoding.

Example tagged Sample struct

type Sample struct {
	ExampleType string      `frostdb:"example_type,rle_dict,asc(0)"`
	Labels      []Label     `frostdb:"labels,rle_dict,null,dyn,asc(1),null_first"`
	Stacktrace  []uuid.UUID `frostdb:"stacktrace,rle_dict,asc(3),null_first"`
	Timestamp   int64       `frostdb:"timestamp,asc(2)"`
	Value       int64       `frostdb:"value"`
}

Dynamic columns

Field of type map<string, T> is a dynamic column by default.

type Example struct {
	// Use supported tags to customize the column value
	Labels map[string]string `frostdb:"labels"`
}

Repeated columns

Fields of type []int64, []float64, []bool, and []string are supported. These are represented as arrow.LIST.

Generated schema for the repeated columns applies all supported tags. By default repeated fields are nullable. You can safely pass nil slices for repeated columns.

func NewBuild

func NewBuild[T any](mem memory.Allocator) *Build[T]

func (*Build[T]) Append

func (b *Build[T]) Append(values ...T) error

func (*Build[T]) NewRecord

func (b *Build[T]) NewRecord() *Record

func (*Build[T]) Release

func (b *Build[T]) Release()

func (Build[T]) Schema

func (b Build[T]) Schema(name string) (s *schemapb.Schema)

type Reader

type Reader[T any] struct {
	// contains filtered or unexported fields
}

func NewReader

func NewReader[T any](records ...arrow.Record) *Reader[T]

func (*Reader[T]) NumRows

func (r *Reader[T]) NumRows() int64

func (*Reader[T]) Value

func (r *Reader[T]) Value(i int) T

type Record

type Record struct {
	arrow.Record
	SortingColumns []arrowutils.SortingColumn
}