Back to godoc.org
github.com/grailbio/bigslice/frame

package frame

v0.0.0 (8e22b3d)
Latest Go to latest
Published: May 28, 2020 | License: Apache-2.0 | Module: github.com/grailbio/bigslice

Overview

Package frame implements a typed, columnar data structure that represents data vectors throughout Bigslice.

The package contains the definition of Frame as well as a set of index-based operators that amortize runtime safety overhead.

Index

Variables

var Empty = Frame{/* contains filtered or unexported fields */}

Empty is the empty frame.

func CanCompare

func CanCompare(typ reflect.Type) bool

CanCompare returns whether values of the provided type are comparable.

func CanHash

func CanHash(typ reflect.Type) bool

CanHash returns whether values of the provided type can be hashed.

func Compatible

func Compatible(f, g Frame) bool

Compatible reports whether frames f and g are assignment compatible: that is, they have the same number of columns and the same column types.

func Copy

func Copy(dst, src Frame) (n int)

Copy copies the contents of src until either dst has been filled or src exhausted. It returns the number of elements copied.

func RegisterOps

func RegisterOps(make interface{})

RegisterOps registers an ops implementation. The provided argument make should be a function of the form

func(slice []t) Ops

returning operations for a t-typed slice. RegisterOps panics if the argument does not have the required shape or if operations have already been registered for type t.

type Decoder

type Decoder interface {
	Session
	// Decode decodes a value from the underlying stream using
	// Gob.
	Decode(v interface{}) error
}

type Encoder

type Encoder interface {
	Session
	// Encode encodes the provided value into the
	// encoder's stream using Gob.
	Encode(v interface{}) error
}

An Encoder manages transmission of data over a connection or file.

type Frame

type Frame struct {
	// contains filtered or unexported fields
}

A Frame is a collection of 0 or more typed, equal-length columns that form a logical table. Each column is represented by a Go slice. Frames can be sliced efficiently, and the package provides computed operators that can perform efficient index-based operations on the Frame.

func AppendFrame

func AppendFrame(dst, src Frame) Frame

AppendFrame appends src to dst, growing dst if needed.

func Make

func Make(types slicetype.Type, len, cap int) Frame

Make returns a new frame with the provided type, length, and capacity.

func Slices

func Slices(cols ...interface{}) Frame

Slices returns a new Frame constructed from a set of Go slices, each representing a column. The slices must have the same length, or Slices panics.

func Values

func Values(cols []reflect.Value) Frame

Values returns a new Frame constructed from a set of reflect.Values, each representing a column. The slices must have the same length, or Values panics.

func (Frame) Cap

func (f Frame) Cap() int

Cap returns the Frame's capacity.

func (Frame) Decode

func (f Frame) Decode(col int, d Decoder) error

Decode decodes a column col as encoded by Frame.Encode. may only Decode be invoked for columns where HasCodec is true.

func (Frame) Encode

func (f Frame) Encode(col int, e Encoder) error

Encode encodes column col of this frame. The given scratch buffer may be used by the encode function to avoid extra allocation. Encode may only be invoked for columns where HasCodec is true.

func (Frame) Ensure

func (f Frame) Ensure(n int) Frame

Ensure Slice(0, n), growing the frame as needed.

func (Frame) Grow

func (f Frame) Grow(n int) Frame

Grow returns a Frame with at least n extra capacity. The returned frame will have length f.Len()+n.

func (Frame) HasCodec

func (f Frame) HasCodec(col int) bool

HasCodec returns whether column col has a type-specific codec.

func (Frame) Hash

func (f Frame) Hash(i int) uint32

Hash returns a 32-bit hash of the prefix columns of frame f with a seed of 0.

func (Frame) HashWithSeed

func (f Frame) HashWithSeed(i int, seed uint32) uint32

HashWithSeed returns a 32-bit seeded hash of the prefix columns of frame f.

func (Frame) Index

func (f Frame) Index(col, i int) reflect.Value

Index returns the i'th row of col'th column as a reflect.Value.

func (Frame) Interface

func (f Frame) Interface(i int) interface{}

Interface returns the i'th column as an empty interface.

func (Frame) Interfaces

func (f Frame) Interfaces() []interface{}

Interfaces returns the frame's columns as empty interfaces.

func (Frame) IsZero

func (f Frame) IsZero() bool

IsZero tells whether this frame is zero-valued.

func (Frame) Len

func (f Frame) Len() int

Len returns the Frame's length.

func (Frame) Less

func (f Frame) Less(i, j int) bool

Less reports whether the row with index i should sort before the element with index j. Less operates on the frame's prefix columns, and is available only if the operation is defined for those column types. See RegisterOps for more details.

TODO(marius): this method presents an unnecessary indirection; provide a way to get at a sort.Interface directly.

func (Frame) NumOut

func (f Frame) NumOut() int

NumOut implements slicetype.Type

func (Frame) Out

func (f Frame) Out(i int) reflect.Type

Out implements slicetype.Type.

func (Frame) Prefix

func (f Frame) Prefix() int

Prefix implements slicetype.Type.

func (Frame) Prefixed

func (f Frame) Prefixed(prefix int) Frame

Prefixed returns f with the given prefix.

func (Frame) Slice

func (f Frame) Slice(i, j int) Frame

Slice returns the frame f[i:j]. It panics if indices are out of bounds.

func (Frame) SliceHeader

func (f Frame) SliceHeader(i int) reflect.SliceHeader

SliceHeader returns the slice header for column i. As with other uses of SliceHeader, the user must ensure that a reference to the frame is maintained so that the underlying slice is not garbage collected while (unsafely) using the slice header.

func (Frame) String

func (f Frame) String() string

String returns a descriptive string of the frame.

func (Frame) Swap

func (f Frame) Swap(i, j int)

Swap swaps rows i and j in frame f.

func (Frame) TabString

func (f Frame) TabString() string

TabString returns a string representing the frame in tabular format.

func (Frame) UnsafeIndexAddr

func (f Frame) UnsafeIndexAddr(col, i int) uintptr

UnsafeIndexAddr returns the address of the i'th row of the col'th column. This can be used by advanced clients that import the unsafe package. Clients are responsible for managing reference lifetimes so that the underlying objects will not be garbage collected while an address returned from this method may still be used.

func (Frame) Value

func (f Frame) Value(i int) reflect.Value

Value returns the ith column as a reflect.Value.

func (Frame) Values

func (f Frame) Values() []reflect.Value

Values returns the frame's columns as reflect.Values.

func (Frame) WriteTab

func (f Frame) WriteTab(w io.Writer)

WriteTab writes the frame in tabular format to the provided io.Writer.

func (Frame) Zero

func (f Frame) Zero()

Zero zeros the memory all columnns.

type Key

type Key uint64

Key represents a key that can be used to store session specific state.

func FreshKey

func FreshKey() Key

FreshKey returns a unique key that can be used to store state in sessions.

type Ops

type Ops struct {
	// Less compares two indices of a slice.
	Less func(i, j int) bool
	// HashWithSeed computes a 32-bit hash, given a seed, of an index
	// of a slice.
	HashWithSeed func(i int, seed uint32) uint32

	// Encode encodes a slice of the underlying vector. Encode is
	// optional, and overrides the default encoding (gob) used when
	// serializing and deserializing frames. Encode and decode must
	// always be specified together.
	Encode func(enc Encoder, i, j int) error

	// Decode decodes a slice encoded by Encode(i, j).
	Decode func(dec Decoder, i, j int) error
	// contains filtered or unexported fields
}

Ops represents a set of operations on a single frame instance. Ops are instantiated from implementations registered with RegisterOps and are managed by the frame instance.

Note that not all types may support all operations.

type Session

type Session interface {
	// State retrieves the session state for the provided key
	// into the pointer state. If state is not a pointer, then
	// State will panic. State returns true the first time the key
	// is encountered in the session. This is typically used by
	// user code to initialize the state.
	//
	// If the state value is a pointer, then the first call to State
	// sets the state pointer (a pointer to a pointer) to a newly
	// allocated value. Otherwise it is set to a zero value of the
	// value type.
	State(key Key, state interface{}) bool
}

Session is a key-value store that is used by frame codecs to store session-specific state.

Documentation was rendered with GOOS=linux and GOARCH=amd64.

Jump to identifier

Keyboard shortcuts

? : This menu
f or F : Jump to identifier