Documentation ¶
Overview ¶
Package tiledb is a idomatic go binding to tiledb's c_api. Go structs are used for object style access to tiledb types, such as Config and ArraySchema. Tiledb c objects that are alloc'ed are set to be freeded on garbage collection using `runtime.SetFinalizer`.
For more information on TileDB see official docs, https://docs.tiledb.io/en/stable .
Semantic Versioning is followed for this package and for compatibility with upcoming vgo. See compatibility section of Readme for a mapping of TileDB-Go package to tiledb core library versions, https://github.com/TileDB-Inc/TileDB-Go/blob/master/README.md#compatibility .
Installation ¶
See readme for installation requirements and instructions: https://github.com/TileDB-Inc/TileDB-Go/blob/master/README.md#installation .
Quickstart ¶
See quickstart_dense_test.go and quickstart_sparse_test.go for examples. Also checkout the official tiledb quickstart docs https://docs.tiledb.io/en/latest/quickstart.html
Index ¶
- Variables
- func GroupCreate(context *Context, group string) error
- func StatsDisable() error
- func StatsDump(path string) error
- func StatsDumpSTDOUT() error
- func StatsEnable() error
- func StatsReset() error
- func Version() (major int, minor int, rev int)
- type Array
- func (a *Array) Close() error
- func (a *Array) Consolidate(config *Config) error
- func (a *Array) ConsolidateWithKey(encryptionType EncryptionType, key string, config *Config) error
- func (a *Array) Create(arraySchema *ArraySchema) error
- func (a *Array) CreateWithKey(arraySchema *ArraySchema, encryptionType EncryptionType, key string) error
- func (a *Array) Free()
- func (a *Array) MaxBufferElements(subarray interface{}) (map[string][2]uint64, error)
- func (a *Array) MaxBufferSize(attributeName string, subarray interface{}) (uint64, error)
- func (a *Array) MaxBufferSizeVar(attributeName string, subarray interface{}) (uint64, uint64, error)
- func (a *Array) NonEmptyDomain() ([]NonEmptyDomain, bool, error)
- func (a *Array) Open(queryType QueryType) error
- func (a *Array) OpenAt(queryType QueryType, timestamp uint64) error
- func (a *Array) OpenAtWithKey(queryType QueryType, encryptionType EncryptionType, key string, ...) error
- func (a *Array) OpenWithKey(queryType QueryType, encryptionType EncryptionType, key string) error
- func (a *Array) QueryType() (QueryType, error)
- func (a *Array) Reopen() error
- func (a *Array) Schema() (*ArraySchema, error)
- func (a *Array) URI() (string, error)
- type ArraySchema
- func (a *ArraySchema) AddAttributes(attributes ...*Attribute) error
- func (a *ArraySchema) AttributeFromIndex(index uint) (*Attribute, error)
- func (a *ArraySchema) AttributeFromName(attrName string) (*Attribute, error)
- func (a *ArraySchema) AttributeNum() (uint, error)
- func (a *ArraySchema) Attributes() ([]*Attribute, error)
- func (a *ArraySchema) Capacity() (uint64, error)
- func (a *ArraySchema) CellOrder() (Layout, error)
- func (a *ArraySchema) Check() error
- func (a *ArraySchema) CoordsFilterList() (*FilterList, error)
- func (a *ArraySchema) Domain() (*Domain, error)
- func (a *ArraySchema) Dump(path string) error
- func (a *ArraySchema) DumpSTDOUT() error
- func (a *ArraySchema) Free()
- func (a *ArraySchema) OffsetsFilterList() (*FilterList, error)
- func (a *ArraySchema) SetCapacity(capacity uint64) error
- func (a *ArraySchema) SetCellOrder(cellOrder Layout) error
- func (a *ArraySchema) SetCoordsFilterList(filterList *FilterList) error
- func (a *ArraySchema) SetDomain(domain *Domain) error
- func (a *ArraySchema) SetOffsetsFilterList(filterList *FilterList) error
- func (a *ArraySchema) SetTileOrder(tileOrder Layout) error
- func (a *ArraySchema) TileOrder() (Layout, error)
- type ArrayType
- type Attribute
- func (a *Attribute) CellValNum() (uint, error)
- func (a *Attribute) Dump(path string) error
- func (a *Attribute) DumpSTDOUT() error
- func (a *Attribute) FilterList() (*FilterList, error)
- func (a *Attribute) Free()
- func (a *Attribute) Name() (string, error)
- func (a *Attribute) SetCellValNum(val uint) error
- func (a *Attribute) SetFilterList(filterlist *FilterList) error
- func (a *Attribute) Type() (Datatype, error)
- type Config
- type Context
- type Datatype
- type Dimension
- type Domain
- func (d *Domain) AddDimensions(dimensions ...*Dimension) error
- func (d *Domain) DimensionFromIndex(index uint) (*Dimension, error)
- func (d *Domain) DimensionFromName(name string) (*Dimension, error)
- func (d *Domain) Dump(path string) error
- func (d *Domain) DumpSTDOUT() error
- func (d *Domain) Free()
- func (d *Domain) NDim() (uint, error)
- func (d *Domain) Type() (Datatype, error)
- type EncryptionType
- type FS
- type Filter
- type FilterList
- func (f *FilterList) AddFilter(filter *Filter) error
- func (f *FilterList) FilterFromIndex(index uint32) (*Filter, error)
- func (f *FilterList) Filters() ([]*Filter, error)
- func (f *FilterList) Free()
- func (f *FilterList) MaxChunkSize() (uint32, error)
- func (f *FilterList) NFilters() (uint32, error)
- func (f *FilterList) SetMaxChunkSize(maxChunkSize uint32) error
- type FilterOption
- type FilterType
- type KV
- func (k *KV) AddItem(item *KVItem) error
- func (k *KV) Close() error
- func (k *KV) Consolidate(config *Config) error
- func (k *KV) ConsolidateWithKey(encryptionType EncryptionType, key string, config *Config) error
- func (k *KV) Create(kvSchema *KVSchema) error
- func (k *KV) CreateWithKey(kvSchema *KVSchema, encryptionType EncryptionType, key string) error
- func (k *KV) Flush() error
- func (k *KV) Free()
- func (k *KV) IsDirty() (bool, error)
- func (k *KV) IsOpen() (bool, error)
- func (k *KV) Item(key interface{}) (*KVItem, error)
- func (k *KV) Iterate() (*KVIter, error)
- func (k *KV) Open(queryType QueryType) error
- func (k *KV) OpenAt(queryType QueryType, timestamp uint64) error
- func (k *KV) OpenAtWithKey(queryType QueryType, encryptionType EncryptionType, key string, ...) error
- func (k *KV) OpenWithKey(queryType QueryType, encryptionType EncryptionType, key string) error
- func (k *KV) Reopen() error
- func (k *KV) ReopenAt(timestamp uint64) error
- func (k *KV) Schema() (*KVSchema, error)
- func (k *KV) Timestamp() (uint64, error)
- type KVItem
- type KVIter
- type KVSchema
- func (k *KVSchema) AddAttributes(attributes ...*Attribute) error
- func (k *KVSchema) AttributeFromIndex(index uint) (*Attribute, error)
- func (k *KVSchema) AttributeFromName(attrName string) (*Attribute, error)
- func (k *KVSchema) AttributeNum() (uint, error)
- func (k *KVSchema) Attributes() ([]*Attribute, error)
- func (k *KVSchema) Capacity() (uint64, error)
- func (k *KVSchema) Check() error
- func (k *KVSchema) Dump(path string) error
- func (k *KVSchema) DumpSTDOUT() error
- func (k *KVSchema) Free()
- func (k *KVSchema) SetCapacity(capacity uint64) error
- type Layout
- type NonEmptyDomain
- type Query
- func (q *Query) Buffer(attributeName string) (interface{}, error)
- func (q *Query) BufferVar(attributeName string) ([]uint64, interface{}, error)
- func (q *Query) Finalize() error
- func (q *Query) Free()
- func (q *Query) HasResults() (bool, error)
- func (q *Query) ResultBufferElements() (map[string][2]uint64, error)
- func (q *Query) SetBuffer(attribute string, buffer interface{}) (*uint64, error)
- func (q *Query) SetBufferVar(attribute string, offset []uint64, buffer interface{}) (*uint64, *uint64, error)
- func (q *Query) SetCoordinates(coordinates interface{}) (*uint64, error)
- func (q *Query) SetLayout(layout Layout) error
- func (q *Query) SetSubArray(subArray interface{}) error
- func (q *Query) Status() (QueryStatus, error)
- func (q *Query) Submit() error
- func (q *Query) SubmitAsync() error
- func (q *Query) Type() (QueryType, error)
- type QueryStatus
- type QueryType
- type VFS
- func (v *VFS) Close(fh *VFSfh) error
- func (v *VFS) Config() (*Config, error)
- func (v *VFS) CreateBucket(uri string) error
- func (v *VFS) CreateDir(uri string) error
- func (v *VFS) DirSize(uri string) (uint64, error)
- func (v *VFS) EmptyBucket(uri string) error
- func (v *VFS) FileSize(uri string) (uint64, error)
- func (v *VFS) Free()
- func (v *VFS) IsBucket(uri string) (bool, error)
- func (v *VFS) IsDir(uri string) (bool, error)
- func (v *VFS) IsEmptyBucket(uri string) (bool, error)
- func (v *VFS) IsFile(uri string) (bool, error)
- func (v *VFS) MoveDir(oldURI string, newURI string) error
- func (v *VFS) MoveFile(oldURI string, newURI string) error
- func (v *VFS) Open(uri string, mode VFSMode) (*VFSfh, error)
- func (v *VFS) Read(fh *VFSfh, offset uint64, nbytes uint64) ([]byte, error)
- func (v *VFS) RemoveBucket(uri string) error
- func (v *VFS) RemoveDir(uri string) error
- func (v *VFS) RemoveFile(uri string) error
- func (v *VFS) Sync(fh *VFSfh) error
- func (v *VFS) Touch(uri string) error
- func (v *VFS) Write(fh *VFSfh, bytes []byte) error
- type VFSMode
- type VFSfh
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var TILEDB_COORDS = C.GoString(C.TILEDB_COORDS)
TILEDB_COORDS A special name indicating the coordinates attribute.
var TILEDB_VAR_NUM = uint(C.TILEDB_VAR_NUM)
TILEDB_VAR_NUM indicates variable sized attributes for cell values
Functions ¶
func GroupCreate ¶
GroupCreate creates a new tiledb group. A Group is a logical grouping of Objects on the storage system (a directory).
Example ¶
// Create context without config context, err := NewContext(nil) if err != nil { // Handle error return } // Create Group err = GroupCreate(context, "my_group") if err != nil { // Handle error return }
Output:
func StatsEnable ¶
func StatsEnable() error
StatsEnable enable internal statistics gathering
Example ¶
Example usage of tiledb statistics
err := StatsEnable() if err != nil { // Handle error } // Perform tile operations err = StatsDumpSTDOUT() if err != nil { // Handle error }
Output:
Types ¶
type Array ¶
type Array struct {
// contains filtered or unexported fields
}
Array struct representing a TileDB array object.
An Array object represents array data in TileDB at some persisted location, e.g. on disk, in an S3 bucket, etc. Once an array has been opened for reading or writing, interact with the data through Query objects.
func NewArray ¶
NewArray alloc a new array
Example ¶
// Create Config, this is optional config, err := NewConfig() if err != nil { // Handle error return } // Test context with config context, err := NewContext(config) if err != nil { // Handle error return } // Create Dimension dimension, err := NewDimension(context, "test", []int32{1, 10}, 5) if err != nil { // Handle error return } // Create Domain domain, err := NewDomain(context) if err != nil { // Handle error return } // Add dimension to domain err = domain.AddDimensions(dimension) if err != nil { // Handle error return } arraySchema, err := NewArraySchema(context, TILEDB_DENSE) if err != nil { // Handle error return } // Crete attribute to add to schema attribute, err := NewAttribute(context, "a1", TILEDB_INT32) if err != nil { // Handle error return } err = arraySchema.AddAttributes(attribute) if err != nil { // Handle error return } array, err := NewArray(context, "my_array") if err != nil { // Handle error return } err = array.Create(arraySchema) if err != nil { // Handle error return }
Output:
func (*Array) Consolidate ¶
Consolidate Consolidates the fragments of an array into a single fragment. You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).
func (*Array) ConsolidateWithKey ¶ added in v0.2.0
func (a *Array) ConsolidateWithKey(encryptionType EncryptionType, key string, config *Config) error
ConsolidateWithKey Consolidates the fragments of an encrypted array into a single fragment. You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).
func (*Array) Create ¶
func (a *Array) Create(arraySchema *ArraySchema) error
Create a new TileDB array given an input schema.
func (*Array) CreateWithKey ¶ added in v0.2.0
func (a *Array) CreateWithKey(arraySchema *ArraySchema, encryptionType EncryptionType, key string) error
CreateWithKey a new TileDB array given an input schema.
func (*Array) MaxBufferElements ¶
MaxBufferElements compute an upper bound on the buffer elements needed to read a subarray. Returns A map of attribute name (including TILEDB_COORDS) to the maximum number of elements that can be read in the given subarray. For each attribute, a pair of numbers are returned. The first, for variable-length attributes, is the maximum number of offsets for that attribute in the given subarray. For fixed-length attributes and coordinates, the first is always 0. The second is the maximum number of elements for that attribute in the given subarray.
func (*Array) MaxBufferSize ¶
MaxBufferSize computes the upper bound on the buffer size (in bytes) required for a read query for a given fixed attribute and subarray
func (*Array) MaxBufferSizeVar ¶
func (a *Array) MaxBufferSizeVar(attributeName string, subarray interface{}) (uint64, uint64, error)
MaxBufferSizeVar computes the upper bound on the buffer size (in bytes) required for a read query for a given variable sized attribute and subarray
func (*Array) NonEmptyDomain ¶
func (a *Array) NonEmptyDomain() ([]NonEmptyDomain, bool, error)
NonEmptyDomain retrieves the non-empty domain from an array This returns the bounding coordinates for each dimension
func (*Array) Open ¶
Open the array. The array is opened using a query type as input. This is to indicate that queries created for this Array object will inherit the query type. In other words, Array objects are opened to receive only one type of queries. They can always be closed and be re-opened with another query type. Also there may be many different Array objects created and opened with different query types. For instance, one may create and open an array object array_read for reads and another one array_write for writes, and interleave creation and submission of queries for both these array objects.
func (*Array) OpenAt ¶ added in v0.2.0
OpenAt Similar to tiledb_array_open, but this function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This function is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.
func (*Array) OpenAtWithKey ¶ added in v0.2.0
func (a *Array) OpenAtWithKey(queryType QueryType, encryptionType EncryptionType, key string, timestamp uint64) error
OpenAtWithKey Similar to tiledb_array_open_with_key, but this function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This function is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.
func (*Array) OpenWithKey ¶ added in v0.2.0
func (a *Array) OpenWithKey(queryType QueryType, encryptionType EncryptionType, key string) error
OpenWithKey Opens an encrypted array using the given encryption key. This function has the same semantics as tiledb_array_open() but is used for encrypted arrays.
An encrypted array must be opened with this function before queries can be issued to it.
func (*Array) Reopen ¶
Reopen the array (the array must be already open). This is useful when the array got updated after it got opened and the Array object got created. To sync-up with the updates, the user must either close the array and open with open(), or just use reopen() without closing. This function will be generally faster than the former alternative.
func (*Array) Schema ¶
func (a *Array) Schema() (*ArraySchema, error)
Schema returns the ArraySchema for the array
type ArraySchema ¶
type ArraySchema struct {
// contains filtered or unexported fields
}
ArraySchema Schema describing an array.
The schema is an independent description of an array. A schema can be used to create multiple array’s, and stores information about its domain, cell types, and compression details. An array schema is composed of:
A Domain A set of Attributes Memory layout definitions: tile and cell Compression details for Array level factors like offsets and coordinates
func LoadArraySchema ¶
func LoadArraySchema(context *Context, path string) (*ArraySchema, error)
LoadArraySchema reads a directory for a ArraySchema
func LoadArraySchemaWithKey ¶ added in v0.2.0
func LoadArraySchemaWithKey(context *Context, path string, encryptionType EncryptionType, key string) (*ArraySchema, error)
LoadArraySchemaWithKey retrieves the schema of an encrypted array from the disk, creating an array schema struct.
func NewArraySchema ¶
func NewArraySchema(ctx *Context, arrayType ArrayType) (*ArraySchema, error)
NewArraySchema alloc a new ArraySchema
Example ¶
// Create Config, this is optional config, err := NewConfig() if err != nil { // Handle error return } // Test context with config context, err := NewContext(config) if err != nil { // Handle error return } // Create Dimension dimension, err := NewDimension(context, "test", []int32{1, 10}, 5) if err != nil { // Handle error return } // Create Domain domain, err := NewDomain(context) if err != nil { // Handle error return } // Add dimension to domain err = domain.AddDimensions(dimension) if err != nil { // Handle error return } arraySchema, err := NewArraySchema(context, TILEDB_DENSE) if err != nil { // Handle error return } // Crete attribute to add to schema attribute, err := NewAttribute(context, "a1", TILEDB_INT32) if err != nil { // Handle error return } err = arraySchema.AddAttributes(attribute) if err != nil { // Handle error return }
Output:
func (*ArraySchema) AddAttributes ¶
func (a *ArraySchema) AddAttributes(attributes ...*Attribute) error
AddAttributes add one or more attributes to the array
func (*ArraySchema) AttributeFromIndex ¶
func (a *ArraySchema) AttributeFromIndex(index uint) (*Attribute, error)
AttributeFromIndex get a copy of an Attribute in the schema by name.
func (*ArraySchema) AttributeFromName ¶
func (a *ArraySchema) AttributeFromName(attrName string) (*Attribute, error)
AttributeFromName Get a copy of an Attribute in the schema by index. Attributes are ordered the same way they were defined when constructing the array schema.
func (*ArraySchema) AttributeNum ¶
func (a *ArraySchema) AttributeNum() (uint, error)
AttributeNum returns the number of attributes
func (*ArraySchema) Attributes ¶
func (a *ArraySchema) Attributes() ([]*Attribute, error)
Attributes gets all attributes in the array.
func (*ArraySchema) Capacity ¶
func (a *ArraySchema) Capacity() (uint64, error)
Capacity returns the tile capacity.
func (*ArraySchema) CellOrder ¶
func (a *ArraySchema) CellOrder() (Layout, error)
CellOrder return the cell order
func (*ArraySchema) CoordsFilterList ¶ added in v0.2.0
func (a *ArraySchema) CoordsFilterList() (*FilterList, error)
CoordsFilterList Returns a copy of the filter list of the coordinates.
func (*ArraySchema) Domain ¶
func (a *ArraySchema) Domain() (*Domain, error)
Domain returns the array's domain
func (*ArraySchema) Dump ¶
func (a *ArraySchema) Dump(path string) error
Dump Dumps the array schema in ASCII format in the selected output.
func (*ArraySchema) DumpSTDOUT ¶
func (a *ArraySchema) DumpSTDOUT() error
DumpSTDOUT Dumps the array schema in ASCII format to stdout
func (*ArraySchema) Free ¶
func (a *ArraySchema) Free()
Free tiledb_array_schema_t that was allocated on heap in c
func (*ArraySchema) OffsetsFilterList ¶ added in v0.2.0
func (a *ArraySchema) OffsetsFilterList() (*FilterList, error)
OffsetsFilterList returns a copy of the FilterList of the offsets for variable-length attributes.
func (*ArraySchema) SetCapacity ¶
func (a *ArraySchema) SetCapacity(capacity uint64) error
SetCapacity sets the tile capacity.
func (*ArraySchema) SetCellOrder ¶
func (a *ArraySchema) SetCellOrder(cellOrder Layout) error
SetCellOrder set the cell order
func (*ArraySchema) SetCoordsFilterList ¶ added in v0.2.0
func (a *ArraySchema) SetCoordsFilterList(filterList *FilterList) error
SetCoordsFilterList sets the filter list used for coordinates
func (*ArraySchema) SetDomain ¶
func (a *ArraySchema) SetDomain(domain *Domain) error
SetDomain sets the array domain
func (*ArraySchema) SetOffsetsFilterList ¶ added in v0.2.0
func (a *ArraySchema) SetOffsetsFilterList(filterList *FilterList) error
SetOffsetsFilterList sets the filter list for the offsets of variable-length attributes
func (*ArraySchema) SetTileOrder ¶
func (a *ArraySchema) SetTileOrder(tileOrder Layout) error
SetTileOrder set the tile order
func (*ArraySchema) TileOrder ¶
func (a *ArraySchema) TileOrder() (Layout, error)
TileOrder return the tile order
type ArrayType ¶
type ArrayType int8
ArrayType enum for tiledb arrays
const ( // TILEDB_DENSE dense array TILEDB_DENSE ArrayType = C.TILEDB_DENSE // TILEDB_SPARSE dense array TILEDB_SPARSE ArrayType = C.TILEDB_SPARSE )
type Attribute ¶
type Attribute struct {
// contains filtered or unexported fields
}
Attribute describes an attribute of an Array cell.
An attribute specifies a name and datatype for a particular value in each array cell. There are 3 supported attribute types:
Fundamental types, such as char, int, double, uint64, etc.. Fixed sized arrays: [N]T or make([]T, N), where T is a fundamental type Variable length data: string, []T, where T is a fundamental type
func NewAttribute ¶
NewAttribute alloc a new attribute
Example ¶
// Create Config, this is optional config, err := NewConfig() if err != nil { // Handle error return } // Test context with config context, err := NewContext(config) if err != nil { // Handle error return } // Create Attribute attribute, err := NewAttribute(context, "test", TILEDB_INT32) if err != nil { // Handle error return } // Set Filter List filter, err := NewFilter(context, TILEDB_FILTER_GZIP) if err != nil { // Handle error return } filterList, err := NewFilterList(context) if err != nil { // Handle error return } err = filterList.AddFilter(filter) if err != nil { // Handle error return } err = attribute.SetFilterList(filterList) if err != nil { // Handle error return } // Set Cell Value Number err = attribute.SetCellValNum(10) if err != nil { // Handle error return }
Output:
func (*Attribute) CellValNum ¶
CellValNum returns number of values of one cell on this attribute. For variable-sized attributes returns TILEDB_VAR_NUM.
func (*Attribute) DumpSTDOUT ¶
DumpSTDOUT Dumps the attribute in ASCII format to stdout
func (*Attribute) FilterList ¶ added in v0.2.0
func (a *Attribute) FilterList() (*FilterList, error)
FilterList returns a copy of the filter list for attribute
func (*Attribute) Free ¶
func (a *Attribute) Free()
Free tiledb_attribute_t that was allocated on heap in c
func (*Attribute) SetCellValNum ¶
SetCellValNum Sets the number of attribute values per cell. This is inferred from the type parameter of the NewAttribute function, but can also be set manually.
func (*Attribute) SetFilterList ¶ added in v0.2.0
func (a *Attribute) SetFilterList(filterlist *FilterList) error
SetFilterList sets the attribute filterList
Example ¶
// Create configuration config, err := NewConfig() if err != nil { // Handle error return } // Test context with config context, err := NewContext(config) if err != nil { // Handle error return } attribute, err := NewAttribute(context, "test", TILEDB_INT32) if err != nil { // Handle error return } // Set Filter List filter, err := NewFilter(context, TILEDB_FILTER_GZIP) if err != nil { // Handle error return } err = filter.SetOption(TILEDB_COMPRESSION_LEVEL, int32(5)) if err != nil { // Handle error return } filterList, err := NewFilterList(context) if err != nil { // Handle error return } err = filterList.AddFilter(filter) if err != nil { // Handle error return } err = attribute.SetFilterList(filterList) if err != nil { // Handle error return }
Output:
type Config ¶
type Config struct {
// contains filtered or unexported fields
}
Config Carries configuration parameters for a context.
func LoadConfig ¶
LoadConfig reads a configuration from a given uri
func (*Config) Get ¶
Get Get a parameter from the configuration by key.
Example ¶
config, err := NewConfig() if err != nil { // handle error } val, err := config.Get("sm.tile_cache_size") if err != nil { // handle error } fmt.Println(val)
Output: 10000000
func (*Config) SaveToFile ¶
SaveToFile Saves the config parameters to a (local) text file.
func (*Config) Set ¶
Set Sets a config parameter-value pair.
Example ¶
config, err := NewConfig() if err != nil { // handle error } err = config.Set("sm.tile_cache_size", "10") if err != nil { // handle error } val, err := config.Get("sm.tile_cache_size") if err != nil { // handle error } fmt.Println(val)
Output: 10
type Context ¶
type Context struct {
// contains filtered or unexported fields
}
Context A TileDB context wraps a TileDB storage manager “instance.” Most objects and functions will require a Context. Internal error handling is also defined by the Context; the default error handler throws a TileDBError with a specific message.
func NewContext ¶
NewContext creates a TileDB context with the given configuration If the configuration passed is null it is created with default config
Example ¶
ExampleNewContext example of creating a new context
// Create Context with default configuration context, err := NewContext(nil) if err != nil { // handle error return } // Create a config config, err := NewConfig() if err != nil { // handle error return } // Use created config to create a new Context context, err = NewContext(config) if err != nil { // handle error return } // Check if S3 is supported isS3Supported, err := context.IsSupportedFS(TILEDB_S3) if err != nil { // handle error return }
Output: false
func (*Context) IsSupportedFS ¶
IsSupportedFS Return true if the given filesystem backend is supported.
type Datatype ¶
type Datatype int8
Datatype
const ( // TILEDB_INT32 32-bit signed integer TILEDB_INT32 Datatype = C.TILEDB_INT32 // TILEDB_INT64 64-bit signed integer TILEDB_INT64 Datatype = C.TILEDB_INT64 // TILEDB_FLOAT32 32-bit floating point value TILEDB_FLOAT32 Datatype = C.TILEDB_FLOAT32 // TILEDB_FLOAT64 64-bit floating point value TILEDB_FLOAT64 Datatype = C.TILEDB_FLOAT64 // TILEDB_CHAR Character TILEDB_CHAR Datatype = C.TILEDB_CHAR // TILEDB_INT8 8-bit signed integer TILEDB_INT8 Datatype = C.TILEDB_INT8 // TILEDB_UINT8 8-bit unsigned integer TILEDB_UINT8 Datatype = C.TILEDB_UINT8 // TILEDB_INT16 16-bit signed integer TILEDB_INT16 Datatype = C.TILEDB_INT16 // TILEDB_UINT16 16-bit unsigned integer TILEDB_UINT16 Datatype = C.TILEDB_UINT16 // TILEDB_UINT32 32-bit unsigned integer TILEDB_UINT32 Datatype = C.TILEDB_UINT32 // TILEDB_UINT64 64-bit unsigned integer TILEDB_UINT64 Datatype = C.TILEDB_UINT64 // TILEDB_STRING_ASCII ASCII string TILEDB_STRING_ASCII Datatype = C.TILEDB_STRING_ASCII // TILEDB_STRING_UTF8 UTF-8 string TILEDB_STRING_UTF8 Datatype = C.TILEDB_STRING_UTF8 // TILEDB_STRING_UTF16 UTF-16 string TILEDB_STRING_UTF16 Datatype = C.TILEDB_STRING_UTF16 // TILEDB_STRING_UTF32 UTF-32 string TILEDB_STRING_UTF32 Datatype = C.TILEDB_STRING_UTF32 // TILEDB_STRING_UCS2 UCS2 string TILEDB_STRING_UCS2 Datatype = C.TILEDB_STRING_UCS2 // TILEDB_STRING_UCS4 UCS4 string TILEDB_STRING_UCS4 Datatype = C.TILEDB_STRING_UCS4 // TILEDB_ANY This can be any datatype. Must store (type tag, value) pairs. TILEDB_ANY Datatype = C.TILEDB_ANY )
func (Datatype) ReflectKind ¶
ReflectKind returns the reflect kind given a datatype
type Dimension ¶
type Dimension struct {
// contains filtered or unexported fields
}
Dimension Describes one dimension of an Array. The dimension consists of a type, lower and upper bound, and tile-extent describing the memory ordering. Dimensions are added to a Domain.
func NewDimension ¶
func NewDimension(context *Context, name string, domain interface{}, extent interface{}) (*Dimension, error)
NewDimension alloc a new dimension
Example ¶
// Create Config, this is optional config, err := NewConfig() if err != nil { // Handle error return } // Test context with config context, err := NewContext(config) if err != nil { // Handle error return } // Create Dimension _, err = NewDimension(context, "test", []int32{1, 10}, int32(5)) if err != nil { // Handle error return }
Output:
func (*Dimension) DumpSTDOUT ¶
DumpSTDOUT Dumps the dimension in ASCII format to stdout
func (*Dimension) Free ¶
func (d *Dimension) Free()
Free tiledb_dimension_t that was allocated on heap in c
type Domain ¶
type Domain struct {
// contains filtered or unexported fields
}
Domain Represents the domain of an array. A Domain defines the set of Dimension objects for a given array. The properties of a Domain derive from the underlying dimensions. A Domain is a component of an ArraySchema.
func NewDomain ¶
NewDomain alloc a new domainuration
Example ¶
// Create Config, this is optional config, err := NewConfig() if err != nil { // Handle error return } // Test context with config context, err := NewContext(config) if err != nil { // Handle error return } // Create Dimension dimension, err := NewDimension(context, "test", []int32{1, 10}, int32(5)) if err != nil { // Handle error return } // Create Domain domain, err := NewDomain(context) if err != nil { // Handle error return } // Add dimension to domain err = domain.AddDimensions(dimension) if err != nil { // Handle error return }
Output:
func (*Domain) AddDimensions ¶
AddDimensions adds one or more dimensions to a domain
func (*Domain) DimensionFromIndex ¶
DimensionFromIndex retrieves a dimension object from a domain by index.
func (*Domain) DimensionFromName ¶
DimensionFromName retrieves a dimension object from a domain by name (key)
func (*Domain) DumpSTDOUT ¶
DumpSTDOUT Dumps the domain in ASCII format to stdout
type EncryptionType ¶ added in v0.2.0
type EncryptionType uint8
EncryptionType represents different encryption algorithms
const ( // TILEDB_NO_ENCRYPTION No encryption TILEDB_NO_ENCRYPTION EncryptionType = C.TILEDB_NO_ENCRYPTION // TILEDB_AES_256_GCM AES-256-GCM encryption TILEDB_AES_256_GCM EncryptionType = C.TILEDB_AES_256_GCM )
type Filter ¶ added in v0.2.0
type Filter struct {
// contains filtered or unexported fields
}
Filter represents
func NewFilter ¶ added in v0.2.0
func NewFilter(context *Context, filterType FilterType) (*Filter, error)
NewFilter Allocs a new filter
func (*Filter) Option ¶ added in v0.2.0
func (f *Filter) Option(filterOption FilterOption) (interface{}, error)
Option fetchs the specified option set on a filter. Returns an interface{} dependent on the option being fetched var optionValue int32 optionValueInterface, err := filter.Option(TILEDB_FILTER_GZIP) optionValue = optionValueInterface.(int32)
func (*Filter) SetOption ¶ added in v0.2.0
func (f *Filter) SetOption(filterOption FilterOption, valueInterface interface{}) error
SetOption set an option on a filter. Options are filter dependent; this function returns an error if the given option is not valid for the given filter.
func (*Filter) Type ¶ added in v0.2.0
func (f *Filter) Type() (FilterType, error)
Type returns the filter type
type FilterList ¶ added in v0.2.0
type FilterList struct {
// contains filtered or unexported fields
}
FilterList represents
func NewFilterList ¶ added in v0.2.0
func NewFilterList(context *Context) (*FilterList, error)
Alloc a new FilterList
func (*FilterList) AddFilter ¶ added in v0.2.0
func (f *FilterList) AddFilter(filter *Filter) error
AddFilter appends a filter to a filter list. Data is processed through each filter in the order the filters were added.
func (*FilterList) FilterFromIndex ¶ added in v0.2.0
func (f *FilterList) FilterFromIndex(index uint32) (*Filter, error)
FilterFromIndex Retrieves a filter object from a filter list by index.
func (*FilterList) Filters ¶ added in v0.2.0
func (f *FilterList) Filters() ([]*Filter, error)
Filters return slice of filters applied to filter list
func (*FilterList) MaxChunkSize ¶ added in v0.2.0
func (f *FilterList) MaxChunkSize() (uint32, error)
MaxChunkSize Gets the maximum tile chunk size for a filter list.
func (*FilterList) NFilters ¶ added in v0.2.0
func (f *FilterList) NFilters() (uint32, error)
NFilters Retrieves the number of filters in a filter list.
func (*FilterList) SetMaxChunkSize ¶ added in v0.2.0
func (f *FilterList) SetMaxChunkSize(maxChunkSize uint32) error
SetMaxChunkSize sets the maximum tile chunk size for a filter list.
type FilterOption ¶ added in v0.2.0
type FilterOption uint8
FilterOption for a given filter
const ( // TILEDB_COMPRESSION_LEVEL Compression level. Type: `int32_t`. TILEDB_COMPRESSION_LEVEL FilterOption = C.TILEDB_COMPRESSION_LEVEL // TILEDB_BIT_WIDTH_MAX_WINDOW Max window length for bit width reduction. Type: `uint32_t`. TILEDB_BIT_WIDTH_MAX_WINDOW FilterOption = C.TILEDB_BIT_WIDTH_MAX_WINDOW // TILEDB_POSITIVE_DELTA_MAX_WINDOW Max window length for positive-delta encoding. Type: `uint32_t`. TILEDB_POSITIVE_DELTA_MAX_WINDOW FilterOption = C.TILEDB_POSITIVE_DELTA_MAX_WINDOW )
type FilterType ¶ added in v0.2.0
type FilterType uint8
FilterType for attribute/coordinates/offsets filters
const ( // TILEDB_FILTER_NONE No-op filter TILEDB_FILTER_NONE FilterType = C.TILEDB_FILTER_NONE // TILEDB_FILTER_GZIP Gzip compressor TILEDB_FILTER_GZIP FilterType = C.TILEDB_FILTER_GZIP // TILEDB_FILTER_ZSTD Zstandard compressor TILEDB_FILTER_ZSTD FilterType = C.TILEDB_FILTER_ZSTD // TILEDB_FILTER_LZ4 LZ4 compressor TILEDB_FILTER_LZ4 FilterType = C.TILEDB_FILTER_LZ4 // TILEDB_FILTER_RLE Run-length encoding compressor TILEDB_FILTER_RLE FilterType = C.TILEDB_FILTER_RLE // TILEDB_FILTER_BZIP2 Bzip2 compressor TILEDB_FILTER_BZIP2 FilterType = C.TILEDB_FILTER_BZIP2 // TILEDB_FILTER_DOUBLE_DELTA Double-delta compressor TILEDB_FILTER_DOUBLE_DELTA FilterType = C.TILEDB_FILTER_DOUBLE_DELTA // TILEDB_FILTER_BIT_WIDTH_REDUCTION Bit width reduction filter. TILEDB_FILTER_BIT_WIDTH_REDUCTION FilterType = C.TILEDB_FILTER_BIT_WIDTH_REDUCTION // TILEDB_FILTER_BITSHUFFLE Bitshuffle filter. TILEDB_FILTER_BITSHUFFLE FilterType = C.TILEDB_FILTER_BITSHUFFLE // TILEDB_FILTER_BYTESHUFFLE Byteshuffle filter. TILEDB_FILTER_BYTESHUFFLE FilterType = C.TILEDB_FILTER_BYTESHUFFLE // TILEDB_FILTER_POSITIVE_DELTA Positive-delta encoding filter. TILEDB_FILTER_POSITIVE_DELTA FilterType = C.TILEDB_FILTER_POSITIVE_DELTA )
type KV ¶ added in v0.3.0
type KV struct {
// contains filtered or unexported fields
}
KV Creates a key-value store object.
Example ¶
ExampleKV example of creating and using a Key-Value store
// Create Context with default configuration context, err := NewContext(nil) if err != nil { // handle error return } // Create a KV schema kvSchema, err := NewKVSchema(context) if err != nil { // handle error return } // Add a attribute to the key value store a1, err := NewAttribute(context, "a1", TILEDB_INT8) if err != nil { // handle error return } err = kvSchema.AddAttributes(a1) if err != nil { // handle error return } // create temp KV folder tmpKVPath := os.TempDir() + string(os.PathSeparator) + "tiledb_test_kv" defer os.RemoveAll(tmpKVPath) if _, err = os.Stat(tmpKVPath); err == nil { os.RemoveAll(tmpKVPath) } // Create new KV Struct kv, err := NewKV(context, tmpKVPath) if err != nil { // handle error return } // Create new KV err = kv.Create(kvSchema) if err != nil { // handle error return } // Create new KV err = kv.Open(TILEDB_WRITE) if err != nil { // handle error return } // Create a new KVItem to add kvItem, err := NewKVItem(context) if err != nil { // handle error return } err = kvItem.SetKey("string_key") if err != nil { // handle error return } err = kvItem.SetValue("a1", int8(1)) if err != nil { // handle error return } // Add the item to the kv store err = kv.AddItem(kvItem) if err != nil { // handle error return } // Flush will flush any in-memory kvitems to disk for persistance // Flush can be performance impacting if called to often due to syncing to disk err = kv.Flush() if err != nil { // handle error return } err = kv.Close() if err != nil { // handle error return } // Read key-value store err = kv.Open(TILEDB_READ) if err != nil { // handle error return } // Iterate of the kv store iter, err := kv.Iterate() if err != nil { // handle error return } for ; !iter.IsDone(); err = iter.Next() { if err != nil { // handle error from call to iter.Next() return } // Get current KVItem from iterator kvItemNew, err := iter.Here() if err != nil { // handle error return } key, err := kvItemNew.Key() if err != nil { // handle error return } a1, err := kvItemNew.Value("a1") if err != nil { // handle error return }
Output: string_key 1
func (*KV) AddItem ¶ added in v0.3.0
AddItem Adds a key-value item to a key-value store. The item is buffered internally and periodically flushed to persistent storage. tiledb_kv_flush forces flushing the buffered items to storage.
func (*KV) Close ¶ added in v0.3.0
Close a tiledb kv, this is called on garbage collection automatically All buffered written items will be flushed to persistent storage.
func (*KV) Consolidate ¶ added in v0.3.0
Consolidate Consolidates the fragments of a key-value store into a single fragment.
func (*KV) ConsolidateWithKey ¶ added in v0.3.0
func (k *KV) ConsolidateWithKey(encryptionType EncryptionType, key string, config *Config) error
ConsolidateWithKey Consolidates the fragments of an encrypted key-value store into a single fragment.
func (*KV) CreateWithKey ¶ added in v0.3.0
func (k *KV) CreateWithKey(kvSchema *KVSchema, encryptionType EncryptionType, key string) error
CreateWithKey a new TileDB kv given an input schemk.
func (*KV) Free ¶ added in v0.3.0
func (k *KV) Free()
Free tiledb_kv_t that was allocated on heap in c
func (*KV) IsDirty ¶ added in v0.3.0
IsDirty Checks if the key-value store is dirty, i.e., if the user added items to the key-value store that are kept in main-memory and have not been flushed to persistent storage.
func (*KV) Item ¶ added in v0.3.0
Item Retrieves a key-value item based on the input key. If the item with the input key does not exist, kv_item is set to NULL.
func (*KV) Iterate ¶ added in v0.3.0
Iterate of the kv store
for iter, err := kv.Iterate(); !iter.IsDone(); iter.Next(){ kvItem, err := iter.Here() }
func (*KV) OpenAt ¶ added in v0.3.0
OpenAt Similar to tiledb_kv_open, but this function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This function is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.
func (*KV) OpenAtWithKey ¶ added in v0.3.0
func (k *KV) OpenAtWithKey(queryType QueryType, encryptionType EncryptionType, key string, timestamp uint64) error
OpenAtWithKey Similar to tiledb_kv_open_with_key, but this function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the kv at a timestamp provides a view of the kv with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This function is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the k.
func (*KV) OpenWithKey ¶ added in v0.3.0
func (k *KV) OpenWithKey(queryType QueryType, encryptionType EncryptionType, key string) error
OpenWithKey Prepares an encrypted key-value store for reading/writing.
An encrypted key-value store must be opened with this function before queries can be issued to it.
func (*KV) Reopen ¶ added in v0.3.0
Reopen the kv Reopens a key-value store. This is useful when there were updates to the key-value store after it got opened. This function reopens the key-value store so that it can “see” the new fragments.
func (*KV) ReopenAt ¶ added in v0.3.0
ReopenAt the kv Reopens a key-value store at a specific timestamp.
type KVItem ¶ added in v0.3.0
type KVItem struct {
// contains filtered or unexported fields
}
KV Creates a key-value store object.
func (*KVItem) Free ¶ added in v0.3.0
func (k *KVItem) Free()
Free tiledb_kv_t that was allocated on heap in c
func (*KVItem) Key ¶ added in v0.3.0
Key Gets the key in the key-value item. This copies the value from a c pointer into a golang type
type KVIter ¶ added in v0.3.0
type KVIter struct {
// contains filtered or unexported fields
}
KVIter Creates a key-value store iterator object.
func NewKVIter ¶ added in v0.3.0
NewKVIter Creates an iterator for a key-value store. This can be used only for reading. This sets the pointer to the first key-value item.
func (*KVIter) Free ¶ added in v0.3.0
func (k *KVIter) Free()
Free tiledb_kv_t that was allocated on heap in c
func (*KVIter) Here ¶ added in v0.3.0
Here Retrieves the key-value item currently pointed by the iterator. Note that this function creates a new key-value item.
type KVSchema ¶ added in v0.3.0
type KVSchema struct {
// contains filtered or unexported fields
}
KV Schema describing an array.
The schema is an independent description of an array. A schema can be used to create multiple array’s, and stores information about its domain, cell types, and compression details. An array schema is composed of:
A Domain A set of Attributes Memory layout definitions: tile and cell Compression details for Array level factors like offsets and coordinates
func LoadKVSchema ¶ added in v0.3.0
LoadKVSchema reads a directory for a KVSchema
func LoadKVSchemaWithKey ¶ added in v0.3.0
func LoadKVSchemaWithKey(context *Context, path string, encryptionType EncryptionType, key string) (*KVSchema, error)
LoadKVSchemaWithKey retrieves the schema of an encrypted array from the disk, creating an array schema struct.
func NewKVSchema ¶ added in v0.3.0
NewKVSchema alloc a new KVSchema
func (*KVSchema) AddAttributes ¶ added in v0.3.0
AddAttributes add one or more attributes to the array
func (*KVSchema) AttributeFromIndex ¶ added in v0.3.0
AttributeFromIndex get a copy of an Attribute in the schema by name.
func (*KVSchema) AttributeFromName ¶ added in v0.3.0
AttributeFromName Get a copy of an Attribute in the schema by index. Attributes are ordered the same way they were defined when constructing the array schemk.
func (*KVSchema) AttributeNum ¶ added in v0.3.0
AttributeNum returns the number of attributes
func (*KVSchema) Attributes ¶ added in v0.3.0
Attributes gets all attributes in the array.
func (*KVSchema) Dump ¶ added in v0.3.0
Dump Dumps the array schema in ASCII format in the selected output.
func (*KVSchema) DumpSTDOUT ¶ added in v0.3.0
DumpSTDOUT Dumps the array schema in ASCII format to stdout
func (*KVSchema) Free ¶ added in v0.3.0
func (k *KVSchema) Free()
Free tiledb_kv_schema_t that was allocated on heap in c
func (*KVSchema) SetCapacity ¶ added in v0.3.0
SetCapacity sets the tile capacity.
type Layout ¶
type Layout int8
Layout cell/tile layout
const ( // TILEDB_ROW_MAJOR Row-major layout TILEDB_ROW_MAJOR Layout = C.TILEDB_ROW_MAJOR // TILEDB_COL_MAJOR Column-major layout TILEDB_COL_MAJOR Layout = C.TILEDB_COL_MAJOR // TILEDB_GLOBAL_ORDER Global-order layout TILEDB_GLOBAL_ORDER Layout = C.TILEDB_GLOBAL_ORDER // TILEDB_UNORDERED Unordered layout TILEDB_UNORDERED Layout = C.TILEDB_UNORDERED )
type NonEmptyDomain ¶ added in v0.4.0
type NonEmptyDomain struct { DimensionName string Bounds interface{} }
NonEmptyDomain contains the non empty dimension bounds and dimension name
type Query ¶
type Query struct {
// contains filtered or unexported fields
}
Query construct and execute read/write queries on a tiledb Array
func NewQuery ¶
NewQuery Creates a TileDB query object.
The query type (read or write) must be the same as the type used to open the array object.
The storage manager also acquires a shared lock on the array. This means multiple read and write queries to the same array can be made concurrently (in TileDB, only consolidation requires an exclusive lock for a short period of time).
Example ¶
ExampleNewQuery shows a complete write and read example
// Create configuration config, err := NewConfig() if err != nil { // Handle error return } // Test context with config context, err := NewContext(config) if err != nil { // Handle error return } // Test create dimension dimension, err := NewDimension(context, "dim1", []int8{0, 9}, int8(10)) if err != nil { // Handle error return } // Test creating domain domain, err := NewDomain(context) if err != nil { // Handle error return } // Add dimension err = domain.AddDimensions(dimension) if err != nil { // Handle error return } // Create array schema arraySchema, err := NewArraySchema(context, TILEDB_DENSE) if err != nil { // Handle error return } // Crete attribute to add to schema attribute, err := NewAttribute(context, "a1", TILEDB_INT32) if err != nil { // Handle error return } // Crete attribute to add to schema attribute2, err := NewAttribute(context, "a2", TILEDB_STRING_ASCII) if err != nil { // Handle error return } // Crete attribute to add to schema attribute3, err := NewAttribute(context, "a3", TILEDB_FLOAT32) if err != nil { // Handle error return } // Crete attribute to add to schema attribute4, err := NewAttribute(context, "a4", TILEDB_STRING_UTF8) if err != nil { // Handle error return } // Set a3 to be variable length err = attribute3.SetCellValNum(TILEDB_VAR_NUM) if err != nil { // Handle error return } // Set a4 to be variable length err = attribute4.SetCellValNum(TILEDB_VAR_NUM) if err != nil { // Handle error return } // Add Attribute err = arraySchema.AddAttributes(attribute, attribute2, attribute3, attribute4) if err != nil { // Handle error return } // Set Domain err = arraySchema.SetDomain(domain) if err != nil { // Handle error return } // Validate Schema err = arraySchema.Check() if err != nil { // Handle error return } // create temp array name and path // normal usage would be "my_array" uri // Temp path is used here so unit test can clean up after itself tmpArrayPath := os.TempDir() + string(os.PathSeparator) + "tiledb_test_array" // Cleanup group when test ends defer os.RemoveAll(tmpArrayPath) if _, err = os.Stat(tmpArrayPath); err == nil { os.RemoveAll(tmpArrayPath) } // Create new array struct array, err := NewArray(context, tmpArrayPath) if err != nil { // Handle error return } // Create array on disk err = array.Create(arraySchema) if err != nil { // Handle error return } // Open array for writting err = array.Open(TILEDB_WRITE) if err != nil { // Handle error return } // Create write query query, err := NewQuery(context, array) if err != nil { // Handle error return } // Limit writting to subarray err = query.SetSubArray([]int8{0, 1}) if err != nil { // Handle error return } // Set write layout err = query.SetLayout(TILEDB_ROW_MAJOR) if err != nil { // Handle error return } // Create write buffers bufferA1 := []int32{1, 2} _, err = query.SetBuffer("a1", bufferA1) if err != nil { // Handle error return } bufferA2 := []byte("ab") _, err = query.SetBuffer("a2", bufferA2) if err != nil { // Handle error return } bufferA3 := []float32{1.0, 2.0, 3.0, 4.0, 5.0} offsetBufferA3 := []uint64{0, 3} _, _, err = query.SetBufferVar("a3", offsetBufferA3, bufferA3) if err != nil { // Handle error return } bufferA4 := []byte("hello" + "world") offsetBufferA4 := []uint64{0, 5} _, _, err = query.SetBufferVar("a4", offsetBufferA4, bufferA4) if err != nil { // Handle error return } err = query.Submit() if err != nil { // Handle error return } // Validate status, since query was used this is should be complete status, err := query.Status() if err != nil { // Handle error return } if status != TILEDB_COMPLETED { // handle non-complete query // If applicable read partial data in buffer // and re-submit for remaining results } // Finalize Write err = query.Finalize() if err != nil { // Handle error return } // Close and prepare to read err = array.Close() if err != nil { // Handle error return } // Reopen array for reading err = array.Open(TILEDB_READ) if err != nil { // Handle error return } // Create query for reading query, err = NewQuery(context, array) if err != nil { // Handle error return } // Set read subarray to only data that was written err = query.SetSubArray([]int8{0, 1}) if err != nil { // Handle error return } // Set empty buffers for reading readBufferA1 := make([]int32, 2) _, err = query.SetBuffer("a1", readBufferA1) if err != nil { // Handle error return } readBufferA2 := make([]byte, 2) _, err = query.SetBuffer("a2", readBufferA2) if err != nil { // Handle error return } readBufferA3 := make([]float32, 5) readOffsetBufferA3 := make([]uint64, 2) _, _, err = query.SetBufferVar("a3", readOffsetBufferA3, readBufferA3) if err != nil { // Handle error return } readBufferA4 := make([]byte, 10) readOffsetBufferA4 := make([]uint64, 2) _, _, err = query.SetBufferVar("a4", readOffsetBufferA4, readBufferA4) if err != nil { // Handle error return } // Set read layout err = query.SetLayout(TILEDB_ROW_MAJOR) if err != nil { // Handle error return } // Submit read query async // Async submits do not block err = query.SubmitAsync() if err != nil { // Handle error return } // Wait for status to return complete or to error // Loop while status is inprogress for status, err = query.Status(); status == TILEDB_INPROGRESS && err == nil; status, err = query.Status() { // Do something while query is running } if err != nil { // Handle error return } // Results should be returned hasResults, err := query.HasResults() if err != nil { // Handle error return } if hasResults { // Do something with read buffer }
Output:
func (*Query) Buffer ¶ added in v0.2.0
Buffer returns a slice backed by the underlying c buffer from tiledb
func (*Query) BufferVar ¶ added in v0.2.0
BufferVar returns a slice backed by the underlying c buffer from tiledb for offets and values
func (*Query) Finalize ¶
Finalize Flushes all internal state of a query object and finalizes the query. This is applicable only to global layout writes. It has no effect for any other query type.
func (*Query) HasResults ¶
HasResults Returns true if the query has results Applicable only to read queries (it returns false for write queries)
func (*Query) ResultBufferElements ¶ added in v0.5.0
ResultBufferElements returns the number of elements in the result buffers from a read query. This is a map from the attribute name to a pair of values. The first is number of elements (offsets) for var size attributes, and the second is number of elements in the data buffer. For fixed sized attributes (and coordinates), the first is always 0.
func (*Query) SetBuffer ¶
SetBuffer Sets the buffer for a fixed-sized attribute to a query The buffer must be an initialized slice
func (*Query) SetBufferVar ¶
func (q *Query) SetBufferVar(attribute string, offset []uint64, buffer interface{}) (*uint64, *uint64, error)
SetBufferVar Sets the buffer for a fixed-sized attribute to a query The buffer must be an initialized slice
func (*Query) SetCoordinates ¶
SetCoordinates sets the coordinate buffer
func (*Query) SetSubArray ¶
SetSubArray Sets a subarray, defined in the order dimensions were added. Coordinates are inclusive. For the case of writes, this is meaningful only for dense arrays, and specifically dense writes.
func (*Query) Status ¶
func (q *Query) Status() (QueryStatus, error)
Status returns the status of a query
func (*Query) Submit ¶
Submit a TileDB query This will block until query is completed
Note: Finalize() must be invoked after finish writing in global layout (via repeated invocations of Submit()), in order to flush any internal state. For the case of reads, if the returned status is TILEDB_INCOMPLETE, TileDB could not fit the entire result in the user’s buffers. In this case, the user should consume the read results (if any), optionally reset the buffers with SetBuffer(), and then resubmit the query until the status becomes TILEDB_COMPLETED. If all buffer sizes after the termination of this function become 0, then this means that no useful data was read into the buffers, implying that the larger buffers are needed for the query to proceed. In this case, the users must reallocate their buffers (increasing their size), reset the buffers with set_buffer(), and resubmit the query.
func (*Query) SubmitAsync ¶
SubmitAsync a TileDB query
Async does not currently support the callback function parameter To monitor progress of a query in a non blocking manner the status can be polled:
// Start goroutine for background monitoring go func(query Query) { var status QueryStatus var err error for status, err = query.Status(); status == TILEDB_INPROGRESS && err == nil; status, err = query.Status() { // Do something while query is running } // Do something when query is finished }(query)
type QueryStatus ¶
type QueryStatus int8
QueryStatus status of a query
const ( // TILEDB_FAILED Query failed TILEDB_FAILED QueryStatus = C.TILEDB_FAILED // TILEDB_COMPLETED Query completed (all data has been read) TILEDB_COMPLETED QueryStatus = C.TILEDB_COMPLETED // TILEDB_INPROGRESS Query is in progress TILEDB_INPROGRESS QueryStatus = C.TILEDB_INPROGRESS //TILEDB_INCOMPLETE Query completed (but not all data has been read) TILEDB_INCOMPLETE QueryStatus = C.TILEDB_INCOMPLETE // TILEDB_UNINITIALIZED Query not initialized. TILEDB_UNINITIALIZED QueryStatus = C.TILEDB_UNINITIALIZED )
type QueryType ¶
type QueryType int8
QueryType read or write query
const ( // TILEDB_READ Read query TILEDB_READ QueryType = C.TILEDB_READ // TILEDB_WRITE Write query TILEDB_WRITE QueryType = C.TILEDB_WRITE )
type VFS ¶
type VFS struct {
// contains filtered or unexported fields
}
VFS Implements a virtual filesystem that enables performing directory/file operations with a unified API on different filesystems, such as local posix/windows, HDFS, AWS S3, etc.
func NewVFS ¶
NewVFS alloc a new context using tiledb_vfs_alloc. This also registers the `runtime.SetFinalizer` for handling the free'ing of the c data structure on garbage collection
Example ¶
ExampleNewVFS show basic usage of tiledb's vfs functionality
// Create a new config config, err := NewConfig() if err != nil { // return err } // Optionally set config settings here // config.Set("key", "value") // Create a context context, err := NewContext(config) if err != nil { // return err } // Create a VFS instance vfs, err := NewVFS(context, config) if err != nil { // return err } uri := "file:///tmp/tiledb_example_folder" // Check if directory exists if isDir, err := vfs.IsDir(uri); err != nil { fmt.Println(err) } else { // Directory exists if isDir { fmt.Println("URI is a directory") // Output: URI is a directory } else { fmt.Println("URI is not a directory")
Output: URI is not a directory
func (*VFS) Close ¶
Close a file. This is flushes the buffered data into the file when the file was opened in write (or append) mode. It is particularly important to be called after S3 writes, as otherwise the writes will not take effect.
func (*VFS) CreateBucket ¶
CreateBucket creates an object-store bucket with the input URI.
func (*VFS) Free ¶
func (v *VFS) Free()
Free tiledb_vfs_t c structure that was allocated on the heap
func (*VFS) IsEmptyBucket ¶
IsEmptyBucket check if a bucket is empty
func (*VFS) RemoveBucket ¶
RemoveBucket deletes an object-store bucket with the input URI.
func (*VFS) RemoveFile ¶
RemoveFile deletes a file with the input URI.
type VFSMode ¶
type VFSMode int8
VFSMode is virtual file system file open mode
const ( // TILEDB_VFS_READ open file in read mode TILEDB_VFS_READ VFSMode = C.TILEDB_VFS_READ // TILEDB_VFS_WRITE open file in write mode TILEDB_VFS_WRITE VFSMode = C.TILEDB_VFS_WRITE // TILEDB_VFS_APPENDopen file in write append mode TILEDB_VFS_APPEND VFSMode = C.TILEDB_VFS_APPEND )