inmem

package
Version: v0.0.0-...-d6600b6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 28, 2020 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package inmem implements a shared, in-memory index for each database.

The in-memory index is the original index implementation and provides fast access to index data. However, it also forces high memory usage for large datasets and can cause OOM errors.

Index is the shared index structure that provides most of the functionality. However, ShardIndex is a light per-shard wrapper that adapts this original shared index format to the new per-shard format.

Index

Constants

View Source
const IndexName = tsdb.InmemIndexName

IndexName is the name of this index.

Variables

This section is empty.

Functions

func NewShardIndex

func NewShardIndex(id uint64, seriesIDSet *tsdb.SeriesIDSet, opt tsdb.EngineOptions) tsdb.Index

NewShardIndex returns a new index for a shard.

Types

type FilterExprs

type FilterExprs map[uint64]influxql.Expr

FilterExprs represents a map of series IDs to filter expressions.

func (FilterExprs) DeleteBoolLiteralTrues

func (fe FilterExprs) DeleteBoolLiteralTrues()

DeleteBoolLiteralTrues deletes all elements whose filter expression is a boolean literal true.

func (FilterExprs) Len

func (fe FilterExprs) Len() int

Len returns the number of elements.

type Index

type Index struct {
	// contains filtered or unexported fields
}

Index is the in memory index of a collection of measurements, time series, and their tags. Exported functions are goroutine safe while un-exported functions assume the caller will use the appropriate locks.

func NewIndex

func NewIndex(database string, sfile *tsdb.SeriesFile) *Index

NewIndex returns a new initialized Index.

func (*Index) Bytes

func (i *Index) Bytes() int

Bytes estimates the memory footprint of this Index, in bytes.

func (*Index) Close

func (i *Index) Close() error

func (*Index) CreateMeasurementIndexIfNotExists

func (i *Index) CreateMeasurementIndexIfNotExists(name []byte) *measurement

CreateMeasurementIndexIfNotExists creates or retrieves an in memory index object for the measurement

func (*Index) CreateSeriesListIfNotExists

func (i *Index) CreateSeriesListIfNotExists(seriesIDSet *tsdb.SeriesIDSet, keys, names [][]byte, tagsSlice []models.Tags, opt *tsdb.EngineOptions, ignoreLimits bool) error

CreateSeriesListIfNotExists adds the series for the given measurement to the index and sets its ID or returns the existing series object

func (*Index) Database

func (i *Index) Database() string

Database returns the name of the database the index was initialized with.

func (*Index) DiskSizeBytes

func (i *Index) DiskSizeBytes() int64

DiskSizeBytes always returns zero bytes, since this is an in-memory index.

func (*Index) DropMeasurement

func (i *Index) DropMeasurement(name []byte) error

DropMeasurement removes the measurement and all of its underlying series from the database index

func (*Index) DropMeasurementIfSeriesNotExist

func (i *Index) DropMeasurementIfSeriesNotExist(name []byte) error

DropMeasurementIfSeriesNotExist drops a measurement only if there are no more series for the measurment.

func (*Index) DropSeriesGlobal

func (i *Index) DropSeriesGlobal(key []byte) error

DropSeriesGlobal removes the series key and its tags from the index.

func (*Index) FieldSet

func (i *Index) FieldSet() *tsdb.MeasurementFieldSet

FieldSet returns the assigned fieldset.

func (*Index) ForEachMeasurementName

func (i *Index) ForEachMeasurementName(fn func(name []byte) error) error

ForEachMeasurementName iterates over each measurement name.

func (*Index) ForEachMeasurementTagKey

func (i *Index) ForEachMeasurementTagKey(name []byte, fn func(key []byte) error) error

ForEachMeasurementTagKey iterates over all tag keys for a measurement.

func (*Index) HasTagKey

func (i *Index) HasTagKey(name, key []byte) (bool, error)

HasTagKey returns true if tag key exists.

func (*Index) HasTagValue

func (i *Index) HasTagValue(name, key, value []byte) (bool, error)

HasTagValue returns true if tag value exists.

func (*Index) Measurement

func (i *Index) Measurement(name []byte) (*measurement, error)

Measurement returns the measurement object from the index by the name

func (*Index) MeasurementExists

func (i *Index) MeasurementExists(name []byte) (bool, error)

MeasurementExists returns true if the measurement exists.

func (*Index) MeasurementIterator

func (i *Index) MeasurementIterator() (tsdb.MeasurementIterator, error)

MeasurementIterator returns an iterator over all measurements in the index. MeasurementIterator does not support authorization.

func (*Index) MeasurementNamesByExpr

func (i *Index) MeasurementNamesByExpr(auth query.Authorizer, expr influxql.Expr) ([][]byte, error)

MeasurementNamesByExpr takes an expression containing only tags and returns a list of matching measurement names.

TODO(edd): Remove authorisation from these methods. There shouldn't need to be any auth passed down into the index.

func (*Index) MeasurementNamesByRegex

func (i *Index) MeasurementNamesByRegex(re *regexp.Regexp) ([][]byte, error)

MeasurementNamesByRegex returns the measurements that match the regex.

func (*Index) MeasurementSeriesIDIterator

func (i *Index) MeasurementSeriesIDIterator(name []byte) (tsdb.SeriesIDIterator, error)

func (*Index) MeasurementSeriesKeysByExpr

func (i *Index) MeasurementSeriesKeysByExpr(name []byte, condition influxql.Expr) ([][]byte, error)

func (*Index) MeasurementSeriesKeysByExprIterator

func (i *Index) MeasurementSeriesKeysByExprIterator(name []byte, condition influxql.Expr) (tsdb.SeriesIDIterator, error)

func (*Index) MeasurementTagKeyValuesByExpr

func (i *Index) MeasurementTagKeyValuesByExpr(auth query.Authorizer, name []byte, keys []string, expr influxql.Expr, keysSorted bool) ([][]string, error)

MeasurementTagKeyValuesByExpr returns a set of tag values filtered by an expression.

See tsm1.Engine.MeasurementTagKeyValuesByExpr for a fuller description of this method.

func (*Index) MeasurementTagKeysByExpr

func (i *Index) MeasurementTagKeysByExpr(name []byte, expr influxql.Expr) (map[string]struct{}, error)

MeasurementTagKeysByExpr returns an ordered set of tag keys filtered by an expression.

func (*Index) MeasurementsByName

func (i *Index) MeasurementsByName(names [][]byte) ([]*measurement, error)

MeasurementsByName returns a list of measurements.

func (*Index) MeasurementsSketches

func (i *Index) MeasurementsSketches() (estimator.Sketch, estimator.Sketch, error)

MeasurementsSketches returns the sketches for the measurements.

func (*Index) Open

func (i *Index) Open() (err error)

func (*Index) Rebuild

func (i *Index) Rebuild()

Rebuild recreates the measurement indexes to allow deleted series to be removed and garbage collected.

func (*Index) Series

func (i *Index) Series(key []byte) (*series, error)

Series returns a series by key.

func (*Index) SeriesIDIterator

func (i *Index) SeriesIDIterator(opt query.IteratorOptions) (tsdb.SeriesIDIterator, error)

SeriesIDIterator returns an influxql iterator over matching series ids.

func (*Index) SeriesKeys

func (i *Index) SeriesKeys() []string

func (*Index) SeriesSketches

func (i *Index) SeriesSketches() (estimator.Sketch, estimator.Sketch, error)

SeriesSketches returns the sketches for the series.

func (*Index) SetFieldName

func (i *Index) SetFieldName(measurement []byte, name string)

SetFieldName adds a field name to a measurement.

func (*Index) SetFieldSet

func (i *Index) SetFieldSet(fieldset *tsdb.MeasurementFieldSet)

SetFieldSet sets a shared field set from the engine.

func (*Index) TagKeyCardinality

func (i *Index) TagKeyCardinality(name, key []byte) int

TagKeyCardinality returns the number of values for a measurement/tag key.

func (*Index) TagKeyHasAuthorizedSeries

func (i *Index) TagKeyHasAuthorizedSeries(auth query.Authorizer, name []byte, key string) bool

TagKeyHasAuthorizedSeries determines if there exists an authorized series for the provided measurement name and tag key.

func (*Index) TagKeyIterator

func (i *Index) TagKeyIterator(name []byte) (tsdb.TagKeyIterator, error)

func (*Index) TagKeySeriesIDIterator

func (i *Index) TagKeySeriesIDIterator(name, key []byte) (tsdb.SeriesIDIterator, error)

func (*Index) TagSets

func (i *Index) TagSets(shardSeriesIDs *tsdb.SeriesIDSet, name []byte, opt query.IteratorOptions) ([]*query.TagSet, error)

TagSets returns a list of tag sets.

func (*Index) TagValueIterator

func (i *Index) TagValueIterator(name, key []byte) (tsdb.TagValueIterator, error)

TagValueIterator provides an iterator over all the tag values belonging to series with the provided measurement name and tag key.

TagValueIterator does not currently support authorization.

func (*Index) TagValueN

func (i *Index) TagValueN(name, key []byte) int

TagValueN returns the cardinality of a tag value.

func (*Index) TagValueSeriesIDIterator

func (i *Index) TagValueSeriesIDIterator(name, key, value []byte) (tsdb.SeriesIDIterator, error)

func (*Index) TagsForSeries

func (i *Index) TagsForSeries(key string) (models.Tags, error)

TagsForSeries returns the tag map for the passed in series

func (*Index) Type

func (i *Index) Type() string

func (*Index) UniqueReferenceID

func (i *Index) UniqueReferenceID() uintptr

func (*Index) WithLogger

func (i *Index) WithLogger(*zap.Logger)

type ShardIndex

type ShardIndex struct {
	*Index // Shared reference to global database-wide index.
	// contains filtered or unexported fields
}

ShardIndex represents a shim between the TSDB index interface and the shared in-memory index. This is required because per-shard in-memory indexes will grow the heap size too large.

func (*ShardIndex) CreateSeriesIfNotExists

func (idx *ShardIndex) CreateSeriesIfNotExists(key, name []byte, tags models.Tags) error

CreateSeriesIfNotExists creates the provided series on the index if it is not already present.

func (*ShardIndex) CreateSeriesListIfNotExists

func (idx *ShardIndex) CreateSeriesListIfNotExists(keys, names [][]byte, tagsSlice []models.Tags) error

CreateSeriesListIfNotExists creates a list of series if they doesn't exist in bulk.

func (*ShardIndex) DropMeasurementIfSeriesNotExist

func (idx *ShardIndex) DropMeasurementIfSeriesNotExist(name []byte) error

DropMeasurementIfSeriesNotExist drops a measurement only if there are no more series for the measurment.

func (*ShardIndex) DropSeries

func (idx *ShardIndex) DropSeries(seriesID uint64, _ []byte, _ bool) error

DropSeries removes the provided series id from the local bitset that tracks series in this shard only.

func (*ShardIndex) InitializeSeries

func (idx *ShardIndex) InitializeSeries(keys, names [][]byte, tags []models.Tags) error

InitializeSeries is called during start-up. This works the same as CreateSeriesListIfNotExists except it ignore limit errors.

func (*ShardIndex) SeriesIDSet

func (idx *ShardIndex) SeriesIDSet() *tsdb.SeriesIDSet

SeriesIDSet returns the bitset associated with the series ids.

func (*ShardIndex) SeriesN

func (idx *ShardIndex) SeriesN() int64

SeriesN returns the number of unique non-tombstoned series local to this shard.

func (*ShardIndex) TagSets

func (idx *ShardIndex) TagSets(name []byte, opt query.IteratorOptions) ([]*query.TagSet, error)

TagSets returns a list of tag sets based on series filtering.

type TagFilter

type TagFilter struct {
	Op    influxql.Token
	Key   string
	Value string
	Regex *regexp.Regexp
}

TagFilter represents a tag filter when looking up other tags or measurements.

Source Files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL