Back to godoc.org
github.com/TileDB-Inc/TileDB-Go

Package tiledb

v0.9.0
Latest Go to latest
Published: May 14, 2020 | License: MIT | Module: github.com/TileDB-Inc/TileDB-Go

Overview

Package tiledb is a idomatic go binding to tiledb's c_api. Go structs are used for object style access to tiledb types, such as Config and ArraySchema. Tiledb c objects that are alloc'ed are set to be freeded on garbage collection using `runtime.SetFinalizer`.

For more information on TileDB see official docs, https://docs.tiledb.io/en/stable .

Semantic Versioning is followed for this package and for compatibility with upcoming vgo. See compatibility section of Readme for a mapping of TileDB-Go package to tiledb core library versions, https://github.com/TileDB-Inc/TileDB-Go/blob/master/README.md#compatibility .

Installation

See readme for installation requirements and instructions: https://github.com/TileDB-Inc/TileDB-Go/blob/master/README.md#installation .

Quickstart

See quickstart_dense_test.go and quickstart_sparse_test.go for examples. Also checkout the official tiledb quickstart docs https://docs.tiledb.io/en/latest/quickstart.html

Index

Examples

Package Files

Variables

var TILEDB_COORDS = C.GoString(C.TILEDB_COORDS)

TILEDB_COORDS A special name indicating the coordinates attribute.

var TILEDB_VAR_NUM = uint(C.TILEDB_VAR_NUM)

TILEDB_VAR_NUM indicates variable sized attributes for cell values

func DeserializeArrayMetadata

func DeserializeArrayMetadata(a *Array, buffer *Buffer, serializationType SerializationType) error

DeserializeArrayMetadata deserializes array metadata

func DeserializeArrayNonEmptyDomainAllDimensions

func DeserializeArrayNonEmptyDomainAllDimensions(a *Array, buffer *Buffer, serializationType SerializationType) error

DeserializeArrayNonEmptyDomainAllDimensions deserializes an array nonempty domain

func DeserializeQuery

func DeserializeQuery(query *Query, buffer *Buffer, serializationType SerializationType, clientSide bool) error

DeserializeQuery deserializes a buffer into an existing query

func DeserializeQueryEstResultSizes

func DeserializeQueryEstResultSizes(q *Query, buffer *Buffer, serializationType SerializationType, clientSide bool) error

DeserializeQueryEstResultSizes deserializes query estimated result sizes

func GetTimeFromTimestamp

func GetTimeFromTimestamp(datatype Datatype, timestamp int64) time.Time

GetTimeFromTimestamp returns a time.Time object for a time related TileDB datatype Datetimes in TileDB are deltas from unix epoch with a resolution of the specified time

func GroupCreate

func GroupCreate(context *Context, group string) error

GroupCreate creates a new tiledb group. A Group is a logical grouping of Objects on the storage system (a directory).

Example

Code:

context, err := NewContext(nil)
if err != nil {

	return
}

err = GroupCreate(context, "my_group")
if err != nil {

	return
}

func StatsDisable

func StatsDisable() error

StatsDisable disable internal statistics gathering.

func StatsDump

func StatsDump(path string) error

StatsDump prints internal stats to file path given

func StatsDumpSTDOUT

func StatsDumpSTDOUT() error

StatsDumpSTDOUT prints internal stats to stdout

func StatsEnable

func StatsEnable() error

StatsEnable enable internal statistics gathering

Example

Example usage of tiledb statistics

Code:

err := StatsEnable()
if err != nil {

}

err = StatsDumpSTDOUT()
if err != nil {

}

func StatsReset

func StatsReset() error

StatsReset reset all internal statistics counters to 0.

func Version

func Version() (major int, minor int, rev int)

Version returns the TileDB shared library version these bindings are linked against at runtime

Example

Code:

major, minor, rev := Version()
fmt.Printf("TileDB shared library version is %d.%d.%d", major, minor, rev)

type Array

type Array struct {
	// contains filtered or unexported fields
}

Array struct representing a TileDB array object.

An Array object represents array data in TileDB at some persisted location, e.g. on disk, in an S3 bucket, etc. Once an array has been opened for reading or writing, interact with the data through Query objects.

func NewArray

func NewArray(ctx *Context, uri string) (*Array, error)

NewArray alloc a new array

Example

Code:

config, err := NewConfig()
if err != nil {

	return
}

context, err := NewContext(config)
if err != nil {

	return
}

dimension, err := NewDimension(context, "test", []int32{1, 10}, 5)
if err != nil {

	return
}

domain, err := NewDomain(context)
if err != nil {

	return
}

err = domain.AddDimensions(dimension)
if err != nil {

	return
}

arraySchema, err := NewArraySchema(context, TILEDB_DENSE)
if err != nil {

	return
}

attribute, err := NewAttribute(context, "a1", TILEDB_INT32)

if err != nil {

	return
}

err = arraySchema.AddAttributes(attribute)
if err != nil {

	return
}

array, err := NewArray(context, "my_array")
if err != nil {

	return
}

err = array.Create(arraySchema)
if err != nil {

	return
}

func (*Array) Close

func (a *Array) Close() error

Close a tiledb array, this is called on garbage collection automatically

func (*Array) Consolidate

func (a *Array) Consolidate(config *Config) error

Consolidate Consolidates the fragments of an array into a single fragment. You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).

func (*Array) ConsolidateMetadata

func (a *Array) ConsolidateMetadata(config *Config) error

ConsolidateMetadata consolidates the array metadata into a single array metadata file. You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).

func (*Array) ConsolidateMetadataWithKey

func (a *Array) ConsolidateMetadataWithKey(encryptionType EncryptionType, key string, config *Config) error

ConsolidateMetadataWithKey consolidates the array metadata of an encrypted array into a single file. You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).

func (*Array) ConsolidateWithKey

func (a *Array) ConsolidateWithKey(encryptionType EncryptionType, key string, config *Config) error

ConsolidateWithKey Consolidates the fragments of an encrypted array into a single fragment. You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).

func (*Array) Create

func (a *Array) Create(arraySchema *ArraySchema) error

Create a new TileDB array given an input schema.

func (*Array) CreateWithKey

func (a *Array) CreateWithKey(arraySchema *ArraySchema, encryptionType EncryptionType, key string) error

CreateWithKey a new TileDB array given an input schema.

func (*Array) DeleteMetadata

func (a *Array) DeleteMetadata(key string) error

DeleteMetadata deletes a metadata key-value item from an open array. The array must be opened in WRITE mode, otherwise the function will error out.

func (*Array) Free

func (a *Array) Free()

Free tiledb_array_t that was allocated on heap in c

func (*Array) GetMetadata

func (a *Array) GetMetadata(key string) (Datatype, uint, interface{}, error)

GetMetadata gets a metadata key-value item from an open array. The array must be opened in READ mode, otherwise the function will error out.

func (*Array) GetMetadataFromIndex

func (a *Array) GetMetadataFromIndex(index uint64) (*ArrayMetadata, error)

GetMetadataFromIndex gets a metadata item from an open array using an index. The array must be opened in READ mode, otherwise the function will error out.

func (*Array) GetMetadataFromIndexWithValueLimit

func (a *Array) GetMetadataFromIndexWithValueLimit(index uint64, limit *uint) (*ArrayMetadata, error)

GetMetadataFromIndexWithValueLimit gets a metadata item from an open array using an index. The array must be opened in READ mode, otherwise the function will error out. limit parameter limits the number of values returned if string or array This is helpful for pushdown of limitting metadata. If nil value is returned in full

func (*Array) GetMetadataMap

func (a *Array) GetMetadataMap() (map[string]*ArrayMetadata, error)

GetMetadataMap returns a map[string]*ArrayMetadata where key is the key of each metadata added and value is an ArrayMetadata struct. The map contains all array metadata previously added

func (*Array) GetMetadataMapWithValueLimit

func (a *Array) GetMetadataMapWithValueLimit(limit *uint) (map[string]*ArrayMetadata, error)

GetMetadataMapWithValueLimit returns a map[string]*ArrayMetadata where key is the key of each metadata added and value is an ArrayMetadata struct. The map contains all array metadata previously added limit parameter limits the number of values returned if string or array This is helpful for pushdown of limitting metadata. If nil value is returned in full

func (*Array) GetMetadataNum

func (a *Array) GetMetadataNum() (uint64, error)

GetMetadataNum gets then number of metadata items in an open array. The array must be opened in READ mode, otherwise the function will error out.

func (*Array) MaxBufferElements

func (a *Array) MaxBufferElements(subarray interface{}) (map[string][2]uint64, error)

MaxBufferElements compute an upper bound on the buffer elements needed to read a subarray. Returns A map of attribute name (including TILEDB_COORDS) to the maximum number of elements that can be read in the given subarray. For each attribute, a pair of numbers are returned. The first, for variable-length attributes, is the maximum number of offsets for that attribute in the given subarray. For fixed-length attributes and coordinates, the first is always 0. The second is the maximum number of elements for that attribute in the given subarray.

func (*Array) MaxBufferSize

func (a *Array) MaxBufferSize(attributeName string, subarray interface{}) (uint64, error)

MaxBufferSize computes the upper bound on the buffer size (in bytes) required for a read query for a given fixed attribute and subarray

func (*Array) MaxBufferSizeVar

func (a *Array) MaxBufferSizeVar(attributeName string, subarray interface{}) (uint64, uint64, error)

MaxBufferSizeVar computes the upper bound on the buffer size (in bytes) required for a read query for a given variable sized attribute and subarray

func (*Array) NonEmptyDomain

func (a *Array) NonEmptyDomain() ([]NonEmptyDomain, bool, error)

NonEmptyDomain retrieves the non-empty domain from an array This returns the bounding coordinates for each dimension

func (*Array) NonEmptyDomainFromIndex

func (a *Array) NonEmptyDomainFromIndex(dimIdx uint) (*NonEmptyDomain, bool, error)

NonEmptyDomainFromIndex retrieves the non-empty domain from an array for a given fixed-sized dimension index. Returns the bounding coordinates for the dimension

func (*Array) NonEmptyDomainFromName

func (a *Array) NonEmptyDomainFromName(dimName string) (*NonEmptyDomain, bool, error)

NonEmptyDomainFromName retrieves the non-empty domain from an array for a given fixed-sized dimension name Returns the bounding coordinates for the dimension

func (*Array) NonEmptyDomainVarFromIndex

func (a *Array) NonEmptyDomainVarFromIndex(dimIdx uint) (*NonEmptyDomain, bool, error)

NonEmptyDomainVarFromIndex retrieves the non-empty domain from an array for a given var-sized dimension index. Supports only TILEDB_STRING_ASCII type Returns the bounding coordinates for the dimension

func (*Array) NonEmptyDomainVarFromName

func (a *Array) NonEmptyDomainVarFromName(dimName string) (*NonEmptyDomain, bool, error)

NonEmptyDomainVarFromName retrieves the non-empty domain from an array for a given var-sized dimension name. Supports only TILEDB_STRING_ASCII type Returns the bounding coordinates for the dimension

func (*Array) Open

func (a *Array) Open(queryType QueryType) error

Open the array. The array is opened using a query type as input. This is to indicate that queries created for this Array object will inherit the query type. In other words, Array objects are opened to receive only one type of queries. They can always be closed and be re-opened with another query type. Also there may be many different Array objects created and opened with different query types. For instance, one may create and open an array object array_read for reads and another one array_write for writes, and interleave creation and submission of queries for both these array objects.

func (*Array) OpenAt

func (a *Array) OpenAt(queryType QueryType, timestamp uint64) error

OpenAt Similar to tiledb_array_open, but this function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This function is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.

func (*Array) OpenAtWithKey

func (a *Array) OpenAtWithKey(queryType QueryType, encryptionType EncryptionType, key string, timestamp uint64) error

OpenAtWithKey Similar to tiledb_array_open_with_key, but this function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This function is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.

func (*Array) OpenWithKey

func (a *Array) OpenWithKey(queryType QueryType, encryptionType EncryptionType, key string) error

OpenWithKey Opens an encrypted array using the given encryption key. This function has the same semantics as tiledb_array_open() but is used for encrypted arrays.

An encrypted array must be opened with this function before queries can be issued to it.

func (*Array) PutMetadata

func (a *Array) PutMetadata(key string, value interface{}) error

PutMetadata puts a metadata key-value item to an open array. The array must be opened in WRITE mode, otherwise the function will error out.

func (*Array) QueryType

func (a *Array) QueryType() (QueryType, error)

QueryType return the current query type of an open array

func (*Array) Reopen

func (a *Array) Reopen() error

Reopen the array (the array must be already open). This is useful when the array got updated after it got opened and the Array object got created. To sync-up with the updates, the user must either close the array and open with open(), or just use reopen() without closing. This function will be generally faster than the former alternative.

func (*Array) Schema

func (a *Array) Schema() (*ArraySchema, error)

Schema returns the ArraySchema for the array

func (*Array) URI

func (a *Array) URI() (string, error)

URI returns the array's uri

type ArrayMetadata

type ArrayMetadata struct {
	Key      string
	KeyLen   uint32
	Datatype Datatype
	ValueNum uint
	Value    interface{}
}

ArrayMetadata defines metadata for the array

func (ArrayMetadata) MarshalJSON

func (a ArrayMetadata) MarshalJSON() ([]byte, error)

MarshalJSON implements the Marshaler interface for ArrayMetadata

type ArraySchema

type ArraySchema struct {
	// contains filtered or unexported fields
}

ArraySchema Schema describing an array.

The schema is an independent description of an array. A schema can be used to create multiple array’s, and stores information about its domain, cell types, and compression details. An array schema is composed of:

A Domain
A set of Attributes
Memory layout definitions: tile and cell
Compression details for Array level factors like offsets and coordinates

func DeserializeArraySchema

func DeserializeArraySchema(buffer *Buffer, serializationType SerializationType, clientSide bool) (*ArraySchema, error)

DeserializeArraySchema deserializes a new array schema from the given buffer

func LoadArraySchema

func LoadArraySchema(context *Context, path string) (*ArraySchema, error)

LoadArraySchema reads a directory for a ArraySchema

func LoadArraySchemaWithKey

func LoadArraySchemaWithKey(context *Context, path string, encryptionType EncryptionType, key string) (*ArraySchema, error)

LoadArraySchemaWithKey retrieves the schema of an encrypted array from the disk, creating an array schema struct.

func NewArraySchema

func NewArraySchema(ctx *Context, arrayType ArrayType) (*ArraySchema, error)

NewArraySchema alloc a new ArraySchema

Example

Code:

config, err := NewConfig()
if err != nil {

	return
}

context, err := NewContext(config)
if err != nil {

	return
}

dimension, err := NewDimension(context, "test", []int32{1, 10}, 5)
if err != nil {

	return
}

domain, err := NewDomain(context)
if err != nil {

	return
}

err = domain.AddDimensions(dimension)
if err != nil {

	return
}

arraySchema, err := NewArraySchema(context, TILEDB_DENSE)
if err != nil {

	return
}

attribute, err := NewAttribute(context, "a1", TILEDB_INT32)

if err != nil {

	return
}

err = arraySchema.AddAttributes(attribute)
if err != nil {

	return
}

func (*ArraySchema) AddAttributes

func (a *ArraySchema) AddAttributes(attributes ...*Attribute) error

AddAttributes add one or more attributes to the array

func (*ArraySchema) AttributeFromIndex

func (a *ArraySchema) AttributeFromIndex(index uint) (*Attribute, error)

AttributeFromIndex get a copy of an Attribute in the schema by name.

func (*ArraySchema) AttributeFromName

func (a *ArraySchema) AttributeFromName(attrName string) (*Attribute, error)

AttributeFromName Get a copy of an Attribute in the schema by index. Attributes are ordered the same way they were defined when constructing the array schema.

func (*ArraySchema) AttributeNum

func (a *ArraySchema) AttributeNum() (uint, error)

AttributeNum returns the number of attributes

func (*ArraySchema) Attributes

func (a *ArraySchema) Attributes() ([]*Attribute, error)

Attributes gets all attributes in the array.

func (*ArraySchema) Capacity

func (a *ArraySchema) Capacity() (uint64, error)

Capacity returns the tile capacity.

func (*ArraySchema) CellOrder

func (a *ArraySchema) CellOrder() (Layout, error)

CellOrder return the cell order

func (*ArraySchema) Check

func (a *ArraySchema) Check() error

Check validates the schema

func (*ArraySchema) CoordsFilterList

func (a *ArraySchema) CoordsFilterList() (*FilterList, error)

CoordsFilterList Returns a copy of the filter list of the coordinates.

func (*ArraySchema) Domain

func (a *ArraySchema) Domain() (*Domain, error)

Domain returns the array's domain

func (*ArraySchema) Dump

func (a *ArraySchema) Dump(path string) error

Dump Dumps the array schema in ASCII format in the selected output.

func (*ArraySchema) DumpSTDOUT

func (a *ArraySchema) DumpSTDOUT() error

DumpSTDOUT Dumps the array schema in ASCII format to stdout

func (*ArraySchema) Free

func (a *ArraySchema) Free()

Free tiledb_array_schema_t that was allocated on heap in c

func (*ArraySchema) MarshalJSON

func (a *ArraySchema) MarshalJSON() ([]byte, error)

MarshalJSON marshal arraySchema struct to json using tiledb

func (*ArraySchema) OffsetsFilterList

func (a *ArraySchema) OffsetsFilterList() (*FilterList, error)

OffsetsFilterList returns a copy of the FilterList of the offsets for variable-length attributes.

func (*ArraySchema) SetCapacity

func (a *ArraySchema) SetCapacity(capacity uint64) error

SetCapacity sets the tile capacity.

func (*ArraySchema) SetCellOrder

func (a *ArraySchema) SetCellOrder(cellOrder Layout) error

SetCellOrder set the cell order

func (*ArraySchema) SetCoordsFilterList

func (a *ArraySchema) SetCoordsFilterList(filterList *FilterList) error

SetCoordsFilterList sets the filter list used for coordinates

func (*ArraySchema) SetDomain

func (a *ArraySchema) SetDomain(domain *Domain) error

SetDomain sets the array domain

func (*ArraySchema) SetOffsetsFilterList

func (a *ArraySchema) SetOffsetsFilterList(filterList *FilterList) error

SetOffsetsFilterList sets the filter list for the offsets of variable-length attributes

func (*ArraySchema) SetTileOrder

func (a *ArraySchema) SetTileOrder(tileOrder Layout) error

SetTileOrder set the tile order

func (*ArraySchema) TileOrder

func (a *ArraySchema) TileOrder() (Layout, error)

TileOrder return the tile order

func (*ArraySchema) Type

func (a *ArraySchema) Type() (ArrayType, error)

Type fetch the tiledb array type

func (*ArraySchema) UnmarshalJSON

func (a *ArraySchema) UnmarshalJSON(b []byte) error

UnmarshalJSON marshal arraySchema struct to json using tiledb

type ArrayType

type ArrayType int8

ArrayType enum for tiledb arrays

const (
	// TILEDB_DENSE dense array
	TILEDB_DENSE ArrayType = C.TILEDB_DENSE
	// TILEDB_SPARSE dense array
	TILEDB_SPARSE ArrayType = C.TILEDB_SPARSE
)

type Attribute

type Attribute struct {
	// contains filtered or unexported fields
}

Attribute describes an attribute of an Array cell.

An attribute specifies a name and datatype for a particular value in each array cell. There are 3 supported attribute types:

Fundamental types, such as char, int, double, uint64, etc..
Fixed sized arrays: [N]T or make([]T, N), where T is a fundamental type
Variable length data: string, []T, where T is a fundamental type

func NewAttribute

func NewAttribute(context *Context, name string, datatype Datatype) (*Attribute, error)

NewAttribute alloc a new attribute

Example

Code:

config, err := NewConfig()
if err != nil {

	return
}

context, err := NewContext(config)
if err != nil {

	return
}

attribute, err := NewAttribute(context, "test", TILEDB_INT32)
if err != nil {

	return
}

filter, err := NewFilter(context, TILEDB_FILTER_GZIP)
if err != nil {

	return
}
filterList, err := NewFilterList(context)
if err != nil {

	return
}
err = filterList.AddFilter(filter)
if err != nil {

	return
}
err = attribute.SetFilterList(filterList)
if err != nil {

	return
}

err = attribute.SetCellValNum(10)
if err != nil {

	return
}

func (*Attribute) CellValNum

func (a *Attribute) CellValNum() (uint, error)

CellValNum returns number of values of one cell on this attribute. For variable-sized attributes returns TILEDB_VAR_NUM.

func (*Attribute) Dump

func (a *Attribute) Dump(path string) error

Dump Dumps the attribute in ASCII format in the selected output.

func (*Attribute) DumpSTDOUT

func (a *Attribute) DumpSTDOUT() error

DumpSTDOUT Dumps the attribute in ASCII format to stdout

func (*Attribute) FilterList

func (a *Attribute) FilterList() (*FilterList, error)

FilterList returns a copy of the filter list for attribute

func (*Attribute) Free

func (a *Attribute) Free()

Free tiledb_attribute_t that was allocated on heap in c

func (*Attribute) Name

func (a *Attribute) Name() (string, error)

Name returns name of attribute

func (*Attribute) SetCellValNum

func (a *Attribute) SetCellValNum(val uint) error

SetCellValNum Sets the number of attribute values per cell. This is inferred from the type parameter of the NewAttribute function, but can also be set manually.

func (*Attribute) SetFilterList

func (a *Attribute) SetFilterList(filterlist *FilterList) error

SetFilterList sets the attribute filterList

Example

Code:

config, err := NewConfig()
if err != nil {

	return
}

context, err := NewContext(config)
if err != nil {

	return
}

attribute, err := NewAttribute(context, "test", TILEDB_INT32)
if err != nil {

	return
}

filter, err := NewFilter(context, TILEDB_FILTER_GZIP)
if err != nil {

	return
}

err = filter.SetOption(TILEDB_COMPRESSION_LEVEL, int32(5))
if err != nil {

	return
}

filterList, err := NewFilterList(context)
if err != nil {

	return
}
err = filterList.AddFilter(filter)
if err != nil {

	return
}
err = attribute.SetFilterList(filterList)
if err != nil {

	return
}

func (*Attribute) Type

func (a *Attribute) Type() (Datatype, error)

Type returns the attribute datatype

type Buffer

type Buffer struct {
	// contains filtered or unexported fields
}

Buffer A generic Buffer object used by some TileDB APIs

func NewBuffer

func NewBuffer(context *Context) (*Buffer, error)

NewBuffer Allocs a new buffer

Example

Code:

context, err := NewContext(nil)
if err != nil {

	return
}

buffer, err := NewBuffer(context)
if err != nil {

	return
}

bytes, err := buffer.Data()
if err != nil {

	return
}
fmt.Println(bytes)
[]

func SerializeArrayMaxBufferSizes

func SerializeArrayMaxBufferSizes(a *Array, subarray interface{}, serializationType SerializationType) (*Buffer, error)

SerializeArrayMaxBufferSizes gets and serializes the array max buffer sizes for the given subarray

func SerializeArrayMetadata

func SerializeArrayMetadata(a *Array, serializationType SerializationType) (*Buffer, error)

SerializeArrayMetadata gets and serializes the array metadata

func SerializeArrayNonEmptyDomain

func SerializeArrayNonEmptyDomain(a *Array, serializationType SerializationType) (*Buffer, error)

SerializeArrayNonEmptyDomain gets and serializes the array nonempty domain

func SerializeArrayNonEmptyDomainAllDimensions

func SerializeArrayNonEmptyDomainAllDimensions(a *Array, serializationType SerializationType) (*Buffer, error)

SerializeArrayNonEmptyDomainAllDimensions gets and serializes the array nonempty domain

func SerializeArraySchema

func SerializeArraySchema(schema *ArraySchema, serializationType SerializationType, clientSide bool) (*Buffer, error)

SerializeArraySchema serializes an array schema

func SerializeQueryEstResultSizes

func SerializeQueryEstResultSizes(q *Query, serializationType SerializationType, clientSide bool) (*Buffer, error)

SerializeQueryEstResultSizes gets and serializes the query estimated result sizes

func (*Buffer) Data

func (b *Buffer) Data() ([]byte, error)

Data returns a byte slice backed by the underlying C memory region of the buffer

func (*Buffer) Free

func (b *Buffer) Free()

Free c-alloc'ed data types

func (*Buffer) SetBuffer

func (b *Buffer) SetBuffer(buffer []byte) error

SetType sets the data pointer and size on the Buffer to the given slice

func (*Buffer) SetType

func (b *Buffer) SetType(datatype Datatype) error

SetType sets buffer datatype

func (*Buffer) Type

func (b *Buffer) Type() (Datatype, error)

Type returns the buffer datatype

type BufferList

type BufferList struct {
	// contains filtered or unexported fields
}

BufferList A list of TileDB BufferList objects

func NewBufferList

func NewBufferList(context *Context) (*BufferList, error)

NewBufferList Allocs a new buffer list

Example

Code:

context, err := NewContext(nil)
if err != nil {

	return
}

bufferList, err := NewBufferList(context)
if err != nil {

	return
}

numBuffers, err := bufferList.NumBuffers()
if err != nil {

	return
}
fmt.Println(numBuffers)
0

func SerializeQuery

func SerializeQuery(query *Query, serializationType SerializationType, clientSide bool) (*BufferList, error)

SerializeQuery serializes a query

func (*BufferList) Flatten

func (b *BufferList) Flatten() (*Buffer, error)

Flatten copies and concatenates all buffers in the list into a new buffer

func (*BufferList) Free

func (b *BufferList) Free()

Free c-alloc'ed data types

func (*BufferList) GetBuffer

func (b *BufferList) GetBuffer(bufferIndex uint) (*Buffer, error)

GetBuffer returns a Buffer at the given index in the list

func (*BufferList) NumBuffers

func (b *BufferList) NumBuffers() (uint, error)

NumBuffers returns number of buffers in the list

func (*BufferList) TotalSize

func (b *BufferList) TotalSize() (uint64, error)

TotalSize returns total number of bytes in the buffers in the list

type Config

type Config struct {
	// contains filtered or unexported fields
}

Config Carries configuration parameters for a context.

func LoadConfig

func LoadConfig(uri string) (*Config, error)

LoadConfig reads a configuration from a given uri

func NewConfig

func NewConfig() (*Config, error)

NewConfig alloc a new configuration

func (*Config) Free

func (c *Config) Free()

Free tiledb_config_t that was allocated on heap in c

func (*Config) Get

func (c *Config) Get(param string) (string, error)

Get Get a parameter from the configuration by key.

Example

Code:

config, err := NewConfig()
if err != nil {

}

val, err := config.Get("sm.tile_cache_size")
if err != nil {

}
fmt.Println(val)
10000000

func (*Config) SaveToFile

func (c *Config) SaveToFile(file string) error

SaveToFile Saves the config parameters to a (local) text file.

func (*Config) Set

func (c *Config) Set(param string, value string) error

Set Sets a config parameter-value pair.

Example

Code:

config, err := NewConfig()
if err != nil {

}

err = config.Set("sm.tile_cache_size", "10")
if err != nil {

}

val, err := config.Get("sm.tile_cache_size")
if err != nil {

}
fmt.Println(val)
10

func (*Config) Unset

func (c *Config) Unset(param string) error

Unset Resets a config parameter to its default value.

type Context

type Context struct {
	// contains filtered or unexported fields
}

Context A TileDB context wraps a TileDB storage manager “instance.” Most objects and functions will require a Context. Internal error handling is also defined by the Context; the default error handler throws a TileDBError with a specific message.

func NewContext

func NewContext(config *Config) (*Context, error)

NewContext creates a TileDB context with the given configuration If the configuration passed is null it is created with default config

Example

ExampleNewContext example of creating a new context

Code:

context, err := NewContext(nil)

if err != nil {

	return
}

config, err := NewConfig()
if err != nil {

	return
}

context, err = NewContext(config)
if err != nil {

	return
}

isS3Supported, err := context.IsSupportedFS(TILEDB_S3)
if err != nil {

	return
}

fmt.Println(isS3Supported)
false

func (*Context) Config

func (c *Context) Config() (*Config, error)

Config retrieves a copy of the config from context

func (*Context) Free

func (c *Context) Free()

Free tiledb_ctx_t that was allocated on heap in c

func (*Context) IsSupportedFS

func (c *Context) IsSupportedFS(fs FS) (bool, error)

IsSupportedFS Return true if the given filesystem backend is supported.

func (*Context) LastError

func (c *Context) LastError() error

LastError returns the last error from this context

func (*Context) SetTag

func (c *Context) SetTag(key string, value string) error

SetTag, sets context tag

type Datatype

type Datatype int8

Datatype

const (
	// TILEDB_INT32 32-bit signed integer
	TILEDB_INT32 Datatype = C.TILEDB_INT32
	// TILEDB_INT64 64-bit signed integer
	TILEDB_INT64 Datatype = C.TILEDB_INT64
	// TILEDB_FLOAT32 32-bit floating point value
	TILEDB_FLOAT32 Datatype = C.TILEDB_FLOAT32
	// TILEDB_FLOAT64 64-bit floating point value
	TILEDB_FLOAT64 Datatype = C.TILEDB_FLOAT64
	// TILEDB_CHAR Character
	TILEDB_CHAR Datatype = C.TILEDB_CHAR
	// TILEDB_INT8 8-bit signed integer
	TILEDB_INT8 Datatype = C.TILEDB_INT8
	// TILEDB_UINT8 8-bit unsigned integer
	TILEDB_UINT8 Datatype = C.TILEDB_UINT8
	// TILEDB_INT16 16-bit signed integer
	TILEDB_INT16 Datatype = C.TILEDB_INT16
	// TILEDB_UINT16 16-bit unsigned integer
	TILEDB_UINT16 Datatype = C.TILEDB_UINT16
	// TILEDB_UINT32 32-bit unsigned integer
	TILEDB_UINT32 Datatype = C.TILEDB_UINT32
	// TILEDB_UINT64 64-bit unsigned integer
	TILEDB_UINT64 Datatype = C.TILEDB_UINT64
	// TILEDB_STRING_ASCII ASCII string
	TILEDB_STRING_ASCII Datatype = C.TILEDB_STRING_ASCII
	// TILEDB_STRING_UTF8 UTF-8 string
	TILEDB_STRING_UTF8 Datatype = C.TILEDB_STRING_UTF8
	// TILEDB_STRING_UTF16 UTF-16 string
	TILEDB_STRING_UTF16 Datatype = C.TILEDB_STRING_UTF16
	// TILEDB_STRING_UTF32 UTF-32 string
	TILEDB_STRING_UTF32 Datatype = C.TILEDB_STRING_UTF32
	// TILEDB_STRING_UCS2 UCS2 string
	TILEDB_STRING_UCS2 Datatype = C.TILEDB_STRING_UCS2
	// TILEDB_STRING_UCS4 UCS4 string
	TILEDB_STRING_UCS4 Datatype = C.TILEDB_STRING_UCS4
	// TILEDB_ANY This can be any datatype. Must store (type tag, value) pairs.
	TILEDB_ANY Datatype = C.TILEDB_ANY
	// TILEDB_DATETIME_YEAR 64-bit signed integer representing year
	TILEDB_DATETIME_YEAR Datatype = C.TILEDB_DATETIME_YEAR
	// TILEDB_DATETIME_MONTH 64-bit signed integer representing month
	TILEDB_DATETIME_MONTH Datatype = C.TILEDB_DATETIME_MONTH
	// TILEDB_DATETIME_WEEK 64-bit signed integer representing week
	TILEDB_DATETIME_WEEK Datatype = C.TILEDB_DATETIME_WEEK
	// TILEDB_DATETIME_DAY 64-bit signed integer representing day
	TILEDB_DATETIME_DAY Datatype = C.TILEDB_DATETIME_DAY
	// TILEDB_DATETIME_HR 64-bit signed integer representing hour
	TILEDB_DATETIME_HR Datatype = C.TILEDB_DATETIME_HR
	// TILEDB_DATETIME_MIN 64-bit signed integer representing minute
	TILEDB_DATETIME_MIN Datatype = C.TILEDB_DATETIME_MIN
	// TILEDB_DATETIME_SEC 64-bit signed integer representing second
	TILEDB_DATETIME_SEC Datatype = C.TILEDB_DATETIME_SEC
	// TILEDB_DATETIME_MS 64-bit signed integer representing ms
	TILEDB_DATETIME_MS Datatype = C.TILEDB_DATETIME_MS
	// TILEDB_DATETIME_US 64-bit signed integer representing us
	TILEDB_DATETIME_US Datatype = C.TILEDB_DATETIME_US
	// TILEDB_DATETIME_NS 64-bit signed integer representing ns
	TILEDB_DATETIME_NS Datatype = C.TILEDB_DATETIME_NS
	// TILEDB_DATETIME_PS 64-bit signed integer representing ps
	TILEDB_DATETIME_PS Datatype = C.TILEDB_DATETIME_PS
	// TILEDB_DATETIME_FS 64-bit signed integer representing fs
	TILEDB_DATETIME_FS Datatype = C.TILEDB_DATETIME_FS
	// TILEDB_DATETIME_AS 64-bit signed integer representing as
	TILEDB_DATETIME_AS Datatype = C.TILEDB_DATETIME_AS
)

func DatatypeFromString

func DatatypeFromString(s string) Datatype

DatatypeFromString converts from a datatype string to enum

func (*Datatype) FromString

func (d *Datatype) FromString(s string) error

FromString converts from a datatype string to enum

func (Datatype) MakeSlice

func (d Datatype) MakeSlice(numElements uint64) (interface{}, unsafe.Pointer, error)

MakeSlice makes a slice of the correct type corresponding to the datatype, with a given number of elements

func (Datatype) MarshalJSON

func (d Datatype) MarshalJSON() ([]byte, error)

MarshalJSON interface for marshaling to json

func (Datatype) ReflectKind

func (d Datatype) ReflectKind() reflect.Kind

ReflectKind returns the reflect kind given a datatype

func (Datatype) Size

func (d Datatype) Size() uint64

Size returns the datatype size in bytes

func (Datatype) String

func (d Datatype) String() string

String returns string representation

func (*Datatype) UnmarshalJSON

func (d *Datatype) UnmarshalJSON(bytes []byte) error

UnmarshalJSON interface for unmarshaling from json

type Dimension

type Dimension struct {
	// contains filtered or unexported fields
}

Dimension Describes one dimension of an Array. The dimension consists of a type, lower and upper bound, and tile-extent describing the memory ordering. Dimensions are added to a Domain.

func NewDimension

func NewDimension(context *Context, name string, domain interface{}, extent interface{}) (*Dimension, error)

NewDimension alloc a new dimension

Example

Code:

config, err := NewConfig()
if err != nil {

	return
}

context, err := NewContext(config)
if err != nil {

	return
}

_, err = NewDimension(context, "test", []int32{1, 10}, int32(5))
if err != nil {

	return
}

func NewStringDimension

func NewStringDimension(context *Context, name string) (*Dimension, error)

NewStringDimension alloc a new string dimension

func (*Dimension) CellValNum

func (d *Dimension) CellValNum() (uint, error)

CellValNum returns number of values of one cell on this attribute. For variable-sized attributes returns TILEDB_VAR_NUM.

func (*Dimension) Domain

func (d *Dimension) Domain() (interface{}, error)

Domain returns the dimension's domain

func (*Dimension) Dump

func (d *Dimension) Dump(path string) error

Dump Dumps the dimension in ASCII format in the selected output.

func (*Dimension) DumpSTDOUT

func (d *Dimension) DumpSTDOUT() error

DumpSTDOUT Dumps the dimension in ASCII format to stdout

func (*Dimension) Extent

func (d *Dimension) Extent() (interface{}, error)

Extent returns the dimension's extent

func (*Dimension) Free

func (d *Dimension) Free()

Free tiledb_dimension_t that was allocated on heap in c

func (*Dimension) Name

func (d *Dimension) Name() (string, error)

Name returns the name of the dimension

func (*Dimension) SetCellValNum

func (d *Dimension) SetCellValNum(val uint) error

SetCellValNum Sets the number of values per cell for a dimension. If this is not used, the default is `1`. This is inferred from the type parameter of the NewDimension function, but can also be set manually.

func (*Dimension) Type

func (d *Dimension) Type() (Datatype, error)

Type returns the type of the dimension

type Domain

type Domain struct {
	// contains filtered or unexported fields
}

Domain Represents the domain of an array. A Domain defines the set of Dimension objects for a given array. The properties of a Domain derive from the underlying dimensions. A Domain is a component of an ArraySchema.

func NewDomain

func NewDomain(ctx *Context) (*Domain, error)

NewDomain alloc a new domainuration

Example

Code:

config, err := NewConfig()
if err != nil {

	return
}

context, err := NewContext(config)
if err != nil {

	return
}

dimension, err := NewDimension(context, "test", []int32{1, 10}, int32(5))
if err != nil {

	return
}

domain, err := NewDomain(context)
if err != nil {

	return
}

err = domain.AddDimensions(dimension)
if err != nil {

	return
}

func (*Domain) AddDimensions

func (d *Domain) AddDimensions(dimensions ...*Dimension) error

AddDimensions adds one or more dimensions to a domain

func (*Domain) DimensionFromIndex

func (d *Domain) DimensionFromIndex(index uint) (*Dimension, error)

DimensionFromIndex retrieves a dimension object from a domain by index.

func (*Domain) DimensionFromName

func (d *Domain) DimensionFromName(name string) (*Dimension, error)

DimensionFromName retrieves a dimension object from a domain by name (key)

func (*Domain) Dump

func (d *Domain) Dump(path string) error

Dump Dumps the domain in ASCII format in the selected output.

func (*Domain) DumpSTDOUT

func (d *Domain) DumpSTDOUT() error

DumpSTDOUT Dumps the domain in ASCII format to stdout

func (*Domain) Free

func (d *Domain) Free()

Free tiledb_domain_t that was allocated on heap in c

func (*Domain) HasDimension

func (d *Domain) HasDimension(dimName string) (bool, error)

HasDimension returns true if dimension: `dimName` is part of the domain

func (*Domain) NDim

func (d *Domain) NDim() (uint, error)

NDim returns the number of dimensions

func (*Domain) Type

func (d *Domain) Type() (Datatype, error)

Type returns a domains type deduced from dimensions

type EncryptionType

type EncryptionType uint8

EncryptionType represents different encryption algorithms

const (
	// TILEDB_NO_ENCRYPTION No encryption
	TILEDB_NO_ENCRYPTION EncryptionType = C.TILEDB_NO_ENCRYPTION
	// TILEDB_AES_256_GCM AES-256-GCM encryption
	TILEDB_AES_256_GCM EncryptionType = C.TILEDB_AES_256_GCM
)

type FS

type FS int8

FS represents support fs types

const (
	// TILEDB_HDFS HDFS filesystem support
	TILEDB_HDFS FS = C.TILEDB_HDFS

	// TILEDB_S3 S3 filesystem support
	TILEDB_S3 FS = C.TILEDB_S3
)

type Filter

type Filter struct {
	// contains filtered or unexported fields
}

Filter represents

func NewFilter

func NewFilter(context *Context, filterType FilterType) (*Filter, error)

NewFilter Allocs a new filter

func (*Filter) Free

func (f *Filter) Free()

Free c-alloc'ed data types

func (*Filter) Option

func (f *Filter) Option(filterOption FilterOption) (interface{}, error)

Option fetchs the specified option set on a filter. Returns an interface{} dependent on the option being fetched var optionValue int32 optionValueInterface, err := filter.Option(TILEDB_FILTER_GZIP) optionValue = optionValueInterface.(int32)

func (*Filter) SetOption

func (f *Filter) SetOption(filterOption FilterOption, valueInterface interface{}) error

SetOption set an option on a filter. Options are filter dependent; this function returns an error if the given option is not valid for the given filter.

func (*Filter) Type

func (f *Filter) Type() (FilterType, error)

Type returns the filter type

type FilterList

type FilterList struct {
	// contains filtered or unexported fields
}

FilterList represents

func NewFilterList

func NewFilterList(context *Context) (*FilterList, error)

Alloc a new FilterList

func (*FilterList) AddFilter

func (f *FilterList) AddFilter(filter *Filter) error

AddFilter appends a filter to a filter list. Data is processed through each filter in the order the filters were added.

func (*FilterList) FilterFromIndex

func (f *FilterList) FilterFromIndex(index uint32) (*Filter, error)

FilterFromIndex Retrieves a filter object from a filter list by index.

func (*FilterList) Filters

func (f *FilterList) Filters() ([]*Filter, error)

Filters return slice of filters applied to filter list

func (*FilterList) Free

func (f *FilterList) Free()

Free c-alloc'ed data types

func (*FilterList) MaxChunkSize

func (f *FilterList) MaxChunkSize() (uint32, error)

MaxChunkSize Gets the maximum tile chunk size for a filter list.

func (*FilterList) NFilters

func (f *FilterList) NFilters() (uint32, error)

NFilters Retrieves the number of filters in a filter list.

func (*FilterList) SetMaxChunkSize

func (f *FilterList) SetMaxChunkSize(maxChunkSize uint32) error

SetMaxChunkSize sets the maximum tile chunk size for a filter list.

type FilterOption

type FilterOption uint8

FilterOption for a given filter

const (
	// TILEDB_COMPRESSION_LEVEL Compression level. Type: `int32_t`.
	TILEDB_COMPRESSION_LEVEL FilterOption = C.TILEDB_COMPRESSION_LEVEL
	// TILEDB_BIT_WIDTH_MAX_WINDOW Max window length for bit width reduction. Type: `uint32_t`.
	TILEDB_BIT_WIDTH_MAX_WINDOW FilterOption = C.TILEDB_BIT_WIDTH_MAX_WINDOW
	// TILEDB_POSITIVE_DELTA_MAX_WINDOW Max window length for positive-delta encoding. Type: `uint32_t`.
	TILEDB_POSITIVE_DELTA_MAX_WINDOW FilterOption = C.TILEDB_POSITIVE_DELTA_MAX_WINDOW
)

type FilterType

type FilterType uint8

FilterType for attribute/coordinates/offsets filters

const (
	// TILEDB_FILTER_NONE No-op filter
	TILEDB_FILTER_NONE FilterType = C.TILEDB_FILTER_NONE
	// TILEDB_FILTER_GZIP Gzip compressor
	TILEDB_FILTER_GZIP FilterType = C.TILEDB_FILTER_GZIP
	// TILEDB_FILTER_ZSTD Zstandard compressor
	TILEDB_FILTER_ZSTD FilterType = C.TILEDB_FILTER_ZSTD
	// TILEDB_FILTER_LZ4 LZ4 compressor
	TILEDB_FILTER_LZ4 FilterType = C.TILEDB_FILTER_LZ4
	// TILEDB_FILTER_RLE Run-length encoding compressor
	TILEDB_FILTER_RLE FilterType = C.TILEDB_FILTER_RLE
	// TILEDB_FILTER_BZIP2 Bzip2 compressor
	TILEDB_FILTER_BZIP2 FilterType = C.TILEDB_FILTER_BZIP2
	// TILEDB_FILTER_DOUBLE_DELTA Double-delta compressor
	TILEDB_FILTER_DOUBLE_DELTA FilterType = C.TILEDB_FILTER_DOUBLE_DELTA
	// TILEDB_FILTER_BIT_WIDTH_REDUCTION Bit width reduction filter.
	TILEDB_FILTER_BIT_WIDTH_REDUCTION FilterType = C.TILEDB_FILTER_BIT_WIDTH_REDUCTION
	// TILEDB_FILTER_BITSHUFFLE Bitshuffle filter.
	TILEDB_FILTER_BITSHUFFLE FilterType = C.TILEDB_FILTER_BITSHUFFLE
	// TILEDB_FILTER_BYTESHUFFLE Byteshuffle filter.
	TILEDB_FILTER_BYTESHUFFLE FilterType = C.TILEDB_FILTER_BYTESHUFFLE
	// TILEDB_FILTER_POSITIVE_DELTA Positive-delta encoding filter.
	TILEDB_FILTER_POSITIVE_DELTA FilterType = C.TILEDB_FILTER_POSITIVE_DELTA
)

type Layout

type Layout int8

Layout cell/tile layout

const (
	// TILEDB_ROW_MAJOR Row-major layout
	TILEDB_ROW_MAJOR Layout = C.TILEDB_ROW_MAJOR
	// TILEDB_COL_MAJOR Column-major layout
	TILEDB_COL_MAJOR Layout = C.TILEDB_COL_MAJOR
	// TILEDB_GLOBAL_ORDER Global-order layout
	TILEDB_GLOBAL_ORDER Layout = C.TILEDB_GLOBAL_ORDER
	// TILEDB_UNORDERED Unordered layout
	TILEDB_UNORDERED Layout = C.TILEDB_UNORDERED
)

type NonEmptyDomain

type NonEmptyDomain struct {
	DimensionName string
	Bounds        interface{}
}

NonEmptyDomain contains the non empty dimension bounds and dimension name

func DeserializeArrayNonEmptyDomain

func DeserializeArrayNonEmptyDomain(a *Array, buffer *Buffer, serializationType SerializationType) ([]NonEmptyDomain, bool, error)

DeserializeArrayNonEmptyDomain deserializes an array nonempty domain

type Query

type Query struct {
	// contains filtered or unexported fields
}

Query construct and execute read/write queries on a tiledb Array

func NewQuery

func NewQuery(ctx *Context, array *Array) (*Query, error)

NewQuery Creates a TileDB query object.

The query type (read or write) must be the same as the type used to open the array object.

The storage manager also acquires a shared lock on the array. This means multiple read and write queries to the same array can be made concurrently (in TileDB, only consolidation requires an exclusive lock for a short period of time).

Example

ExampleNewQuery shows a complete write and read example

Code:

config, err := NewConfig()
if err != nil {

	return
}

context, err := NewContext(config)
if err != nil {

	return
}

dimension, err := NewDimension(context, "dim1", []int8{0, 9}, int8(10))
if err != nil {

	return
}

domain, err := NewDomain(context)
if err != nil {

	return
}

err = domain.AddDimensions(dimension)
if err != nil {

	return
}

arraySchema, err := NewArraySchema(context, TILEDB_DENSE)
if err != nil {

	return
}

attribute, err := NewAttribute(context, "a1", TILEDB_INT32)
if err != nil {

	return
}

attribute2, err := NewAttribute(context, "a2", TILEDB_STRING_ASCII)
if err != nil {

	return
}

attribute3, err := NewAttribute(context, "a3", TILEDB_FLOAT32)
if err != nil {

	return
}

attribute4, err := NewAttribute(context, "a4", TILEDB_STRING_UTF8)
if err != nil {

	return
}

err = attribute3.SetCellValNum(TILEDB_VAR_NUM)
if err != nil {

	return
}

err = attribute4.SetCellValNum(TILEDB_VAR_NUM)
if err != nil {

	return
}

err = arraySchema.AddAttributes(attribute, attribute2, attribute3, attribute4)
if err != nil {

	return
}

err = arraySchema.SetDomain(domain)
if err != nil {

	return
}

err = arraySchema.Check()
if err != nil {

	return
}

tmpArrayPath := os.TempDir() + string(os.PathSeparator) + "tiledb_test_array"

defer os.RemoveAll(tmpArrayPath)
if _, err = os.Stat(tmpArrayPath); err == nil {
	os.RemoveAll(tmpArrayPath)
}

array, err := NewArray(context, tmpArrayPath)
if err != nil {

	return
}

err = array.Create(arraySchema)
if err != nil {

	return
}

err = array.Open(TILEDB_WRITE)
if err != nil {

	return
}

query, err := NewQuery(context, array)
if err != nil {

	return
}

err = query.SetSubArray([]int8{0, 1})
if err != nil {

	return
}

err = query.SetLayout(TILEDB_ROW_MAJOR)
if err != nil {

	return
}

bufferA1 := []int32{1, 2}
_, err = query.SetBuffer("a1", bufferA1)
if err != nil {

	return
}

bufferA2 := []byte("ab")
_, err = query.SetBuffer("a2", bufferA2)
if err != nil {

	return
}

bufferA3 := []float32{1.0, 2.0, 3.0, 4.0, 5.0}
offsetBufferA3 := []uint64{0, 3}
_, _, err = query.SetBufferVar("a3", offsetBufferA3, bufferA3)
if err != nil {

	return
}

bufferA4 := []byte("hello" + "world")
offsetBufferA4 := []uint64{0, 5}
_, _, err = query.SetBufferVar("a4", offsetBufferA4, bufferA4)
if err != nil {

	return
}

err = query.Submit()
if err != nil {

	return
}

status, err := query.Status()
if err != nil {

	return
}
if status != TILEDB_COMPLETED {

}

err = query.Finalize()
if err != nil {

	return
}

err = array.Close()
if err != nil {

	return
}

err = array.Open(TILEDB_READ)
if err != nil {

	return
}

query, err = NewQuery(context, array)
if err != nil {

	return
}

err = query.SetSubArray([]int8{0, 1})
if err != nil {

	return
}

readBufferA1 := make([]int32, 2)
_, err = query.SetBuffer("a1", readBufferA1)
if err != nil {

	return
}

readBufferA2 := make([]byte, 2)
_, err = query.SetBuffer("a2", readBufferA2)
if err != nil {

	return
}

readBufferA3 := make([]float32, 5)
readOffsetBufferA3 := make([]uint64, 2)
_, _, err = query.SetBufferVar("a3", readOffsetBufferA3, readBufferA3)
if err != nil {

	return
}
readBufferA4 := make([]byte, 10)
readOffsetBufferA4 := make([]uint64, 2)
_, _, err = query.SetBufferVar("a4", readOffsetBufferA4, readBufferA4)
if err != nil {

	return
}

err = query.SetLayout(TILEDB_ROW_MAJOR)
if err != nil {

	return
}

err = query.SubmitAsync()
if err != nil {

	return
}

for status, err = query.Status(); status == TILEDB_INPROGRESS && err == nil; status, err = query.Status() {

}
if err != nil {

	return
}

hasResults, err := query.HasResults()
if err != nil {

	return
}
if hasResults {

}

func (*Query) AddRange

func (q *Query) AddRange(dimIdx uint32, start interface{}, end interface{}) error

AddRange adds a 1D range along a subarray dimension, which is in the form (start, end, stride). The datatype of the range components must be the same as the type of the domain of the array in the query. The stride is currently unsupported and set to nil.

func (*Query) AddRangeVar

func (q *Query) AddRangeVar(dimIdx uint32, start interface{}, end interface{}) error

AddRangeVar adds a range applicable to variable-sized dimensions Applicable only to string dimensions

func (*Query) Buffer

func (q *Query) Buffer(attributeOrDimension string) (interface{}, error)

Buffer returns a slice backed by the underlying c buffer from tiledb

func (*Query) BufferSize

func (q *Query) BufferSize(attributeNameOrDimension string) (uint64, error)

BufferSize returns the size (in num elements) of the backing C buffer for the given attribute

func (*Query) BufferSizeVar

func (q *Query) BufferSizeVar(attributeOrDimension string) (uint64, uint64, error)

BufferSizeVar returns the size (in num elements) of the backing C buffers for the given variable-length attribute

func (*Query) BufferVar

func (q *Query) BufferVar(attributeOrDimension string) ([]uint64, interface{}, error)

BufferVar returns a slice backed by the underlying c buffer from tiledb for offets and values

func (*Query) Finalize

func (q *Query) Finalize() error

Finalize Flushes all internal state of a query object and finalizes the query. This is applicable only to global layout writes. It has no effect for any other query type.

func (*Query) Free

func (q *Query) Free()

Free tiledb_query_t that was allocated on heap in c

func (*Query) GetRange

func (q *Query) GetRange(dimIdx uint32, rangeNum uint64) (interface{}, interface{}, error)

GetRange retrieves a specific range of the query subarray along a given dimension. Returns (start, end, error) Stride is not supported at the moment, always nil

func (*Query) GetRangeNum

func (q *Query) GetRangeNum(dimIdx uint32) (*uint64, error)

GetRangeNum retrieves the number of ranges of the query subarray along a given dimension.

func (*Query) GetRangeVar

func (q *Query) GetRangeVar(dimIdx uint32, rangeNum uint64) (interface{}, interface{}, error)

GetRangeVar exists for continuinity with other TileDB APIs GetRange in Golang supports the variable length attribute also The function retrieves a specific range of the query subarray along a given dimension. Returns (start, end, error)

func (*Query) GetRanges

func (q *Query) GetRanges() (map[string][]RangeLimits, error)

GetRanges gets the number of dimensions from the array under current query and builds an array of dimensions that have as memmbers arrays of ranges

func (*Query) HasResults

func (q *Query) HasResults() (bool, error)

HasResults Returns true if the query has results Applicable only to read queries (it returns false for write queries)

func (*Query) ResultBufferElements

func (q *Query) ResultBufferElements() (map[string][2]uint64, error)

ResultBufferElements returns the number of elements in the result buffers from a read query. This is a map from the attribute name to a pair of values. The first is number of elements (offsets) for var size attributes, and the second is number of elements in the data buffer. For fixed sized attributes (and coordinates), the first is always 0.

func (*Query) SetBuffer

func (q *Query) SetBuffer(attributeOrDimension string, buffer interface{}) (*uint64,
	error)

SetBuffer Sets the buffer for a fixed-sized attribute to a query The buffer must be an initialized slice

func (*Query) SetBufferUnsafe

func (q *Query) SetBufferUnsafe(attribute string, buffer unsafe.Pointer, bufferSize uint64) (*uint64, error)

SetBufferUnsafe Sets the buffer for a fixed-sized attribute to a query This takes an unsafe pointer which is passsed straight to tiledb c_api for advanced useage

func (*Query) SetBufferVar

func (q *Query) SetBufferVar(attributeOrDimension string, offset []uint64, buffer interface{}) (*uint64, *uint64, error)

SetBufferVar Sets the buffer for a variable sized attribute/dimension to a query The buffer must be an initialized slice

func (*Query) SetBufferVarUnsafe

func (q *Query) SetBufferVarUnsafe(attribute string, offset unsafe.Pointer, offsetSize uint64, buffer unsafe.Pointer, bufferSize uint64) (*uint64, *uint64, error)

SetBufferVarUnsafe Sets the buffer for a variable sized attribute to a query This takes unsafe pointers which is passsed straight to tiledb c_api for advanced useage

func (*Query) SetCoordinates

func (q *Query) SetCoordinates(coordinates interface{}) (*uint64, error)

SetCoordinates sets the coordinate buffer

func (*Query) SetLayout

func (q *Query) SetLayout(layout Layout) error

SetLayout sets the layout of the cells to be written or read

func (*Query) SetSubArray

func (q *Query) SetSubArray(subArray interface{}) error

SetSubArray Sets a subarray, defined in the order dimensions were added. Coordinates are inclusive. For the case of writes, this is meaningful only for dense arrays, and specifically dense writes.

func (*Query) Status

func (q *Query) Status() (QueryStatus, error)

Status returns the status of a query

func (*Query) Submit

func (q *Query) Submit() error

Submit a TileDB query This will block until query is completed

Note: Finalize() must be invoked after finish writing in global layout (via repeated invocations of Submit()), in order to flush any internal state. For the case of reads, if the returned status is TILEDB_INCOMPLETE, TileDB could not fit the entire result in the user’s buffers. In this case, the user should consume the read results (if any), optionally reset the buffers with SetBuffer(), and then resubmit the query until the status becomes TILEDB_COMPLETED. If all buffer sizes after the termination of this function become 0, then this means that no useful data was read into the buffers, implying that the larger buffers are needed for the query to proceed. In this case, the users must reallocate their buffers (increasing their size), reset the buffers with set_buffer(), and resubmit the query.

func (*Query) SubmitAsync

func (q *Query) SubmitAsync() error

SubmitAsync a TileDB query

Async does not currently support the callback function parameter To monitor progress of a query in a non blocking manner the status can be polled:

// Start goroutine for background monitoring
go func(query Query) {
 var status QueryStatus
 var err error
  for status, err = query.Status(); status == TILEDB_INPROGRESS && err == nil; status, err = query.Status() {
    // Do something while query is running
  }
  // Do something when query is finished
}(query)

func (*Query) Type

func (q *Query) Type() (QueryType, error)

Type returns the query type

type QueryStatus

type QueryStatus int8

QueryStatus status of a query

const (
	// TILEDB_FAILED Query failed
	TILEDB_FAILED QueryStatus = C.TILEDB_FAILED
	// TILEDB_COMPLETED Query completed (all data has been read)
	TILEDB_COMPLETED QueryStatus = C.TILEDB_COMPLETED
	// TILEDB_INPROGRESS Query is in progress
	TILEDB_INPROGRESS QueryStatus = C.TILEDB_INPROGRESS
	//TILEDB_INCOMPLETE Query completed (but not all data has been read)
	TILEDB_INCOMPLETE QueryStatus = C.TILEDB_INCOMPLETE
	// TILEDB_UNINITIALIZED Query not initialized.
	TILEDB_UNINITIALIZED QueryStatus = C.TILEDB_UNINITIALIZED
)

type QueryType

type QueryType int8

QueryType read or write query

const (
	// TILEDB_READ Read query
	TILEDB_READ QueryType = C.TILEDB_READ
	// TILEDB_WRITE Write query
	TILEDB_WRITE QueryType = C.TILEDB_WRITE
)

type RangeLimits

type RangeLimits struct {
	// contains filtered or unexported fields
}

RangeLimits defines a query range

func (RangeLimits) MarshalJSON

func (r RangeLimits) MarshalJSON() ([]byte, error)

MarshalJSON implements the Marshaler interface for RangeLimits

type SerializationType

type SerializationType int8

SerializationType how data is serialized

const (
	// TILEDB_JSON Serialization to/from json
	TILEDB_JSON SerializationType = C.TILEDB_JSON

	// TILEDB_JSON Serialization to/from capnp
	TILEDB_CAPNP SerializationType = C.TILEDB_CAPNP
)

type VFS

type VFS struct {
	// contains filtered or unexported fields
}

VFS Implements a virtual filesystem that enables performing directory/file operations with a unified API on different filesystems, such as local posix/windows, HDFS, AWS S3, etc.

func NewVFS

func NewVFS(context *Context, config *Config) (*VFS, error)

NewVFS alloc a new context using tiledb_vfs_alloc. This also registers the `runtime.SetFinalizer` for handling the free'ing of the c data structure on garbage collection

Example

ExampleNewVFS show basic usage of tiledb's vfs functionality

Code:

config, err := NewConfig()
if err != nil {

}

context, err := NewContext(config)
if err != nil {

}

vfs, err := NewVFS(context, config)
if err != nil {

}

uri := "file:///tmp/tiledb_example_folder"

if isDir, err := vfs.IsDir(uri); err != nil {
	fmt.Println(err)
} else {

	if isDir {
		fmt.Println("URI is a directory")

	} else {
		fmt.Println("URI is not a directory")

	}
}
URI is not a directory

func (*VFS) Close

func (v *VFS) Close(fh *VFSfh) error

Close a file. This is flushes the buffered data into the file when the file was opened in write (or append) mode. It is particularly important to be called after S3 writes, as otherwise the writes will not take effect.

func (*VFS) Config

func (v *VFS) Config() (*Config, error)

Config retrieves a copy of the config from vfs

func (*VFS) CreateBucket

func (v *VFS) CreateBucket(uri string) error

CreateBucket creates an object-store bucket with the input URI.

func (*VFS) CreateDir

func (v *VFS) CreateDir(uri string) error

CreateDir creates a directory with the input URI.

func (*VFS) DirSize

func (v *VFS) DirSize(uri string) (uint64, error)

DirSize retrieves the size of a directory.

func (*VFS) EmptyBucket

func (v *VFS) EmptyBucket(uri string) error

EmptyBucket empty a bucket

func (*VFS) FileSize

func (v *VFS) FileSize(uri string) (uint64, error)

FileSize retrieves the size of a file.

func (*VFS) Free

func (v *VFS) Free()

Free tiledb_vfs_t c structure that was allocated on the heap

func (*VFS) IsBucket

func (v *VFS) IsBucket(uri string) (bool, error)

IsBucket checks if an object-store bucket with the input URI exists.

func (*VFS) IsDir

func (v *VFS) IsDir(uri string) (bool, error)

IsDir checks if a directory with the input URI exists.

func (*VFS) IsEmptyBucket

func (v *VFS) IsEmptyBucket(uri string) (bool, error)

IsEmptyBucket check if a bucket is empty

func (*VFS) IsFile

func (v *VFS) IsFile(uri string) (bool, error)

IsFile checks if a file with the input URI exists.

func (*VFS) MoveDir

func (v *VFS) MoveDir(oldURI string, newURI string) error

MoveDir menames a TileDB directory from an old URI to a new URI.

func (*VFS) MoveFile

func (v *VFS) MoveFile(oldURI string, newURI string) error

MoveFile renames a TileDB file from an old URI to a new URI.

func (*VFS) Open

func (v *VFS) Open(uri string, mode VFSMode) (*VFSfh, error)

Open prepares a file for reading/writing.

func (*VFS) Read

func (v *VFS) Read(fh *VFSfh, offset uint64, nbytes uint64) ([]byte, error)

Read part of a file

func (*VFS) RemoveBucket

func (v *VFS) RemoveBucket(uri string) error

RemoveBucket deletes an object-store bucket with the input URI.

func (*VFS) RemoveDir

func (v *VFS) RemoveDir(uri string) error

RemoveDir removes a directory (recursively) with the input URI.

func (*VFS) RemoveFile

func (v *VFS) RemoveFile(uri string) error

RemoveFile deletes a file with the input URI.

func (*VFS) Sync

func (v *VFS) Sync(fh *VFSfh) error

Sync (flushes) a file.

func (*VFS) Touch

func (v *VFS) Touch(uri string) error

Touch a file, i.e., creates a new empty file.

func (*VFS) Write

func (v *VFS) Write(fh *VFSfh, bytes []byte) error

Write the contents of a buffer into a file. Note that this function only appends data at the end of the file. If the file does not exist, it will be created

type VFSMode

type VFSMode int8

VFSMode is virtual file system file open mode

const (
	// TILEDB_VFS_READ open file in read mode
	TILEDB_VFS_READ VFSMode = C.TILEDB_VFS_READ

	// TILEDB_VFS_WRITE open file in write mode
	TILEDB_VFS_WRITE VFSMode = C.TILEDB_VFS_WRITE

	// TILEDB_VFS_APPENDopen file in write append mode
	TILEDB_VFS_APPEND VFSMode = C.TILEDB_VFS_APPEND
)

type VFSfh

type VFSfh struct {
	// contains filtered or unexported fields
}

VFSfh is a virtual file system file handler

func (*VFSfh) Free

func (v *VFSfh) Free()

Free a tiledb c vfs file handler

func (*VFSfh) IsClosed

func (v *VFSfh) IsClosed() (bool, error)

IsClosed checks a vfs file handler to see if it is closed. Return true if file handler is closed, false if its not closed and error is non-nil on error

Documentation was rendered with GOOS=linux and GOARCH=amd64.

Jump to identifier

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to identifier