storage

package
v1.118.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 23, 2025 License: Apache-2.0 Imports: 48 Imported by: 10

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrDeadlineExceeded = fmt.Errorf("deadline exceeded")

ErrDeadlineExceeded is returned when the request times out.

Functions

func DeduplicateSamples added in v1.33.0 

func DeduplicateSamples(srcTimestamps []int64, srcValues []float64, dedupInterval int64) ([]int64, []float64)

DeduplicateSamples removes samples from src* if they are closer to each other than dedupInterval in milliseconds. DeduplicateSamples treats StaleNaN (Prometheus stale markers) as values and doesn't skip them on purpose - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5587

func GetDedupInterval added in v1.71.0 

func GetDedupInterval() int64

GetDedupInterval returns the dedup interval in milliseconds, which has been set via SetDedupInterval.

func MarshalMetricNameRaw 

func MarshalMetricNameRaw(dst []byte, labels []prompbmarshal.Label) []byte

MarshalMetricNameRaw marshals labels to dst and returns the result.

The result must be unmarshaled with MetricName.UnmarshalRaw

func PutMetricName 

func PutMetricName(mn *MetricName)

PutMetricName returns mn to the pool.

func RegexpCacheMaxSizeBytes added in v1.76.0 

func RegexpCacheMaxSizeBytes() int

RegexpCacheMaxSizeBytes returns the maximum size in bytes for the cached regexps for tag filters.

func RegexpCacheMisses 

func RegexpCacheMisses() uint64

RegexpCacheMisses returns the number of cache misses for regexp cache for tag filters.

func RegexpCacheRequests 

func RegexpCacheRequests() uint64

RegexpCacheRequests returns the number of requests to regexp cache for tag filters.

func RegexpCacheSize 

func RegexpCacheSize() int

RegexpCacheSize returns the number of cached regexps for tag filters.

func RegexpCacheSizeBytes added in v1.76.0 

func RegexpCacheSizeBytes() int

RegexpCacheSizeBytes returns an approximate size in bytes for the cached regexps for tag filters.

func RegexpPrefixesCacheMaxSizeBytes added in v1.76.0 

func RegexpPrefixesCacheMaxSizeBytes() int

RegexpPrefixesCacheMaxSizeBytes returns the maximum size in bytes for cached regexp prefixes for tag filters in bytes.

func RegexpPrefixesCacheMisses added in v1.76.0 

func RegexpPrefixesCacheMisses() uint64

RegexpPrefixesCacheMisses returns the number of cache misses for regexp prefixes cache.

func RegexpPrefixesCacheRequests added in v1.76.0 

func RegexpPrefixesCacheRequests() uint64

RegexpPrefixesCacheRequests returns the number of requests to regexp prefixes cache.

func RegexpPrefixesCacheSize added in v1.76.0 

func RegexpPrefixesCacheSize() int

RegexpPrefixesCacheSize returns the number of cached regexp prefixes for tag filters.

func RegexpPrefixesCacheSizeBytes added in v1.76.0 

func RegexpPrefixesCacheSizeBytes() int

RegexpPrefixesCacheSizeBytes returns an approximate size in bytes for cached regexp prefixes for tag filters.

func SetDataFlushInterval added in v1.85.0 

func SetDataFlushInterval(d time.Duration)

SetDataFlushInterval sets the interval for guaranteed flush of recently ingested data from memory to disk.

The data can be flushed from memory to disk more frequently if it doesn't fit the memory limit.

This function must be called before initializing the storage.

func SetDedupInterval added in v1.71.0 

func SetDedupInterval(dedupInterval time.Duration)

SetDedupInterval sets the deduplication interval, which is applied to raw samples during data ingestion and querying.

De-duplication is disabled if dedupInterval is 0.

This function must be called before initializing the storage.

func SetFinalDedupScheduleInterval added in v1.97.15 

func SetFinalDedupScheduleInterval(d time.Duration)

SetFinalDedupScheduleInterval configures the interval for checking when the final deduplication process should start.

func SetFreeDiskSpaceLimit added in v1.67.0 

func SetFreeDiskSpaceLimit(bytes int64)

SetFreeDiskSpaceLimit sets the minimum free disk space size of current storage path

The function must be called before opening or creating any storage.

func SetLogNewSeries added in v1.56.0 

func SetLogNewSeries(ok bool)

SetLogNewSeries updates new series logging.

This function must be called before any calling any storage functions.

func SetMetricNamesStatsCacheSize added in v1.102.16 

func SetMetricNamesStatsCacheSize(size int)

SetMetricNamesStatsCacheSize overrides the default size of storage/metricNamesStatsTracker

func SetRetentionTimezoneOffset added in v1.78.0 

func SetRetentionTimezoneOffset(offset time.Duration)

SetRetentionTimezoneOffset sets the offset, which is used for calculating the time for indexdb rotation. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2574

func SetTSIDCacheSize added in v1.73.1 

func SetTSIDCacheSize(size int)

SetTSIDCacheSize overrides the default size of storage/tsid cache

func SetTagFiltersCacheSize added in v1.83.0 

func SetTagFiltersCacheSize(size int)

SetTagFiltersCacheSize overrides the default size of tagFiltersToMetricIDsCache

func TimestampToHumanReadableFormat added in v1.79.0 

func TimestampToHumanReadableFormat(timestamp int64) string

TimestampToHumanReadableFormat converts the given timestamp to human-readable format.

Types

type Block 

type Block struct {
	// contains filtered or unexported fields
}

Block represents a block of time series values for a single TSID.

func (*Block) AppendRowsWithTimeRangeFilter added in v1.42.0 

func (b *Block) AppendRowsWithTimeRangeFilter(dstTimestamps []int64, dstValues []float64, tr TimeRange) ([]int64, []float64)

AppendRowsWithTimeRangeFilter filters samples from b according to tr and appends them to dst*.

It is expected that UnmarshalData has been already called on b.

func (*Block) CopyFrom 

func (b *Block) CopyFrom(src *Block)

CopyFrom copies src to b.

func (*Block) Init 

func (b *Block) Init(tsid *TSID, timestamps, values []int64, scale int16, precisionBits uint8)

Init initializes b with the given tsid, timestamps, values and scale.

func (*Block) MarshalData 

func (b *Block) MarshalData(timestampsBlockOffset, valuesBlockOffset uint64) ([]byte, []byte, []byte)

MarshalData marshals the block into binary representation.

func (*Block) MarshalPortable added in v1.42.0 

func (b *Block) MarshalPortable(dst []byte) []byte

MarshalPortable marshals b to dst, so it could be portably migrated to other VictoriaMetrics instance.

The marshaled value must be unmarshaled with UnmarshalPortable function.

func (*Block) Reset 

func (b *Block) Reset()

Reset resets b.

func (*Block) RowsCount 

func (b *Block) RowsCount() int

RowsCount returns the number of rows in the block.

func (*Block) UnmarshalData 

func (b *Block) UnmarshalData() error

UnmarshalData unmarshals block data.

func (*Block) UnmarshalPortable added in v1.42.0 

func (b *Block) UnmarshalPortable(src []byte) ([]byte, error)

UnmarshalPortable unmarshals block from src to b and returns the remaining tail.

It is assumed that the block has been marshaled with MarshalPortable.

type BlockRef added in v1.35.0 

type BlockRef struct {
	// contains filtered or unexported fields
}

BlockRef references a Block.

BlockRef is valid only until the corresponding Search is valid, i.e. it becomes invalid after Search.MustClose is called.

func (*BlockRef) Init added in v1.46.0 

func (br *BlockRef) Init(pr PartRef, data []byte) error

Init initializes br from pr and data

func (*BlockRef) Marshal added in v1.46.0 

func (br *BlockRef) Marshal(dst []byte) []byte

Marshal marshals br to dst.

func (*BlockRef) MustReadBlock added in v1.35.0 

func (br *BlockRef) MustReadBlock(dst *Block)

MustReadBlock reads block from br to dst.

func (*BlockRef) PartRef added in v1.46.0 

func (br *BlockRef) PartRef() PartRef

PartRef returns PartRef from br.

func (*BlockRef) RowsCount added in v1.64.0 

func (br *BlockRef) RowsCount() int

RowsCount returns the number of rows in br.

type IndexDBMetrics 

type IndexDBMetrics struct {
	TagFiltersToMetricIDsCacheSize         uint64
	TagFiltersToMetricIDsCacheSizeBytes    uint64
	TagFiltersToMetricIDsCacheSizeMaxBytes uint64
	TagFiltersToMetricIDsCacheRequests     uint64
	TagFiltersToMetricIDsCacheMisses       uint64

	DeletedMetricsCount uint64

	IndexDBRefCount uint64

	MissingTSIDsForMetricID uint64

	RecentHourMetricIDsSearchCalls uint64
	RecentHourMetricIDsSearchHits  uint64

	DateRangeSearchCalls uint64
	DateRangeSearchHits  uint64
	GlobalSearchCalls    uint64

	MissingMetricNamesForMetricID uint64

	IndexBlocksWithMetricIDsProcessed      uint64
	IndexBlocksWithMetricIDsIncorrectOrder uint64

	MinTimestampForCompositeIndex     uint64
	CompositeFilterSuccessConversions uint64
	CompositeFilterMissingConversions uint64

	mergeset.TableMetrics
}

IndexDBMetrics contains essential metrics for indexDB.

type MetricBlockRef added in v1.35.0 

type MetricBlockRef struct {
	// The metric name
	MetricName []byte

	// The block reference. Call BlockRef.MustReadBlock in order to obtain the block.
	BlockRef *BlockRef
}

MetricBlockRef contains reference to time series block for a single metric.

type MetricName 

type MetricName struct {
	MetricGroup []byte

	// Tags are optional. They must be sorted by tag Key for canonical view.
	// Use sortTags method.
	Tags []Tag
}

MetricName represents a metric name.

func GetMetricName 

func GetMetricName() *MetricName

GetMetricName returns a MetricName from pool.

func (*MetricName) AddTag 

func (mn *MetricName) AddTag(key, value string)

AddTag adds new tag to mn with the given key and value.

func (*MetricName) AddTagBytes 

func (mn *MetricName) AddTagBytes(key, value []byte)

AddTagBytes adds new tag to mn with the given key and value.

func (*MetricName) CopyFrom 

func (mn *MetricName) CopyFrom(src *MetricName)

CopyFrom copies src to mn.

func (*MetricName) GetTagValue 

func (mn *MetricName) GetTagValue(tagKey string) []byte

GetTagValue returns tag value for the given tagKey.

func (*MetricName) Marshal 

func (mn *MetricName) Marshal(dst []byte) []byte

Marshal appends marshaled mn to dst and returns the result.

mn.sortTags must be called before calling this function in order to sort and de-duplcate tags.

func (*MetricName) MoveFrom added in v1.86.0 

func (mn *MetricName) MoveFrom(src *MetricName)

MoveFrom moves src to mn.

The src is reset after the call.

func (*MetricName) RemoveTag 

func (mn *MetricName) RemoveTag(tagKey string)

RemoveTag removes a tag with the given tagKey

func (*MetricName) RemoveTagsIgnoring 

func (mn *MetricName) RemoveTagsIgnoring(ignoringTags []string)

RemoveTagsIgnoring removes all the tags included in ignoringTags.

func (*MetricName) RemoveTagsOn 

func (mn *MetricName) RemoveTagsOn(onTags []string)

RemoveTagsOn removes all the tags not included to onTags.

func (*MetricName) Reset 

func (mn *MetricName) Reset()

Reset resets the mn.

func (*MetricName) ResetMetricGroup 

func (mn *MetricName) ResetMetricGroup()

ResetMetricGroup resets mn.MetricGroup

func (*MetricName) SetTagBytes added in v1.92.0 

func (mn *MetricName) SetTagBytes(key, value []byte)

SetTagBytes sets tag with the given key to the given value.

func (*MetricName) SetTags added in v1.37.3 

func (mn *MetricName) SetTags(addTags []string, prefix string, skipTags []string, src *MetricName)

SetTags sets tags from src with keys matching addTags.

It adds prefix to copied label names. skipTags contains a list of tags, which must be skipped.

func (*MetricName) String 

func (mn *MetricName) String() string

String returns user-readable representation of the metric name.

func (*MetricName) Unmarshal 

func (mn *MetricName) Unmarshal(src []byte) error

Unmarshal unmarshals mn from src.

func (*MetricName) UnmarshalRaw added in v1.60.0 

func (mn *MetricName) UnmarshalRaw(src []byte) error

UnmarshalRaw unmarshals mn encoded with MarshalMetricNameRaw.

func (*MetricName) UnmarshalString added in v1.79.0 

func (mn *MetricName) UnmarshalString(s string) error

UnmarshalString unmarshals mn from s

type MetricNamesStatsRecord added in v1.102.19 

type MetricNamesStatsRecord = metricnamestats.StatRecord

MetricNamesStatsRecord represents record at MetricNamesStatsResponse

type MetricNamesStatsResponse added in v1.102.16 

type MetricNamesStatsResponse = metricnamestats.StatsResult

MetricNamesStatsResponse contains metric names usage stats API response

type MetricRow 

type MetricRow struct {
	// MetricNameRaw contains raw metric name, which must be decoded
	// with MetricName.UnmarshalRaw.
	MetricNameRaw []byte

	Timestamp int64
	Value     float64
}

MetricRow is a metric to insert into storage.

func (*MetricRow) CopyFrom 

func (mr *MetricRow) CopyFrom(src *MetricRow)

CopyFrom copies src to mr.

func (*MetricRow) Marshal 

func (mr *MetricRow) Marshal(dst []byte) []byte

Marshal appends marshaled mr to dst and returns the result.

func (*MetricRow) String 

func (mr *MetricRow) String() string

String returns string representation of the mr.

func (*MetricRow) UnmarshalX added in v1.60.0 

func (mr *MetricRow) UnmarshalX(src []byte) ([]byte, error)

UnmarshalX unmarshals mr from src and returns the remaining tail from src.

mr refers to src, so it remains valid until src changes.

type Metrics 

type Metrics struct {
	RowsReceivedTotal uint64
	RowsAddedTotal    uint64
	DedupsDuringMerge uint64
	SnapshotsCount    uint64

	TooSmallTimestampRows uint64
	TooBigTimestampRows   uint64
	InvalidRawMetricNames uint64

	TimeseriesRepopulated  uint64
	TimeseriesPreCreated   uint64
	NewTimeseriesCreated   uint64
	SlowRowInserts         uint64
	SlowPerDayIndexInserts uint64
	SlowMetricNameLoads    uint64

	HourlySeriesLimitRowsDropped   uint64
	HourlySeriesLimitMaxSeries     uint64
	HourlySeriesLimitCurrentSeries uint64

	DailySeriesLimitRowsDropped   uint64
	DailySeriesLimitMaxSeries     uint64
	DailySeriesLimitCurrentSeries uint64

	TimestampsBlocksMerged uint64
	TimestampsBytesSaved   uint64

	TSIDCacheSize         uint64
	TSIDCacheSizeBytes    uint64
	TSIDCacheSizeMaxBytes uint64
	TSIDCacheRequests     uint64
	TSIDCacheMisses       uint64
	TSIDCacheCollisions   uint64

	MetricIDCacheSize         uint64
	MetricIDCacheSizeBytes    uint64
	MetricIDCacheSizeMaxBytes uint64
	MetricIDCacheRequests     uint64
	MetricIDCacheMisses       uint64
	MetricIDCacheCollisions   uint64

	MetricNameCacheSize         uint64
	MetricNameCacheSizeBytes    uint64
	MetricNameCacheSizeMaxBytes uint64
	MetricNameCacheRequests     uint64
	MetricNameCacheMisses       uint64
	MetricNameCacheCollisions   uint64

	DateMetricIDCacheSize        uint64
	DateMetricIDCacheSizeBytes   uint64
	DateMetricIDCacheSyncsCount  uint64
	DateMetricIDCacheResetsCount uint64

	HourMetricIDCacheSize      uint64
	HourMetricIDCacheSizeBytes uint64

	NextDayMetricIDCacheSize      uint64
	NextDayMetricIDCacheSizeBytes uint64

	PrefetchedMetricIDsSize      uint64
	PrefetchedMetricIDsSizeBytes uint64

	NextRetentionSeconds uint64

	MetricNamesUsageTrackerSize         uint64
	MetricNamesUsageTrackerSizeBytes    uint64
	MetricNamesUsageTrackerSizeMaxBytes uint64

	IndexDBMetrics IndexDBMetrics
	TableMetrics   TableMetrics
}

Metrics contains essential metrics for the Storage.

func (*Metrics) Reset 

func (m *Metrics) Reset()

Reset resets m.

type OpenOptions added in v1.102.14 

type OpenOptions struct {
	Retention             time.Duration
	MaxHourlySeries       int
	MaxDailySeries        int
	DisablePerDayIndex    bool
	TrackMetricNamesStats bool
}

OpenOptions optional args for MustOpenStorage

type PartRef added in v1.46.0 

type PartRef struct {
	// contains filtered or unexported fields
}

PartRef is Part reference. 

type Search struct {
	// MetricBlockRef is updated with each Search.NextMetricBlock call.
	MetricBlockRef MetricBlockRef
	// contains filtered or unexported fields
}

Search is a search for time series.

func (*Search) Error 

func (s *Search) Error() error

Error returns the last error from s.

func (*Search) Init 

func (s *Search) Init(qt *querytracer.Tracer, storage *Storage, tfss []*TagFilters, tr TimeRange, maxMetrics int, deadline uint64) int

Init initializes s from the given storage, tfss and tr.

MustClose must be called when the search is done.

Init returns the upper bound on the number of found time series.

func (*Search) MustClose 

func (s *Search) MustClose()

MustClose closes the Search.

func (*Search) NextMetricBlock 

func (s *Search) NextMetricBlock() bool

NextMetricBlock proceeds to the next MetricBlockRef.

type SearchQuery 

type SearchQuery struct {
	// The time range for searching time series
	MinTimestamp int64
	MaxTimestamp int64

	// Tag filters for the search query
	TagFilterss [][]TagFilter

	// The maximum number of time series the search query can return.
	MaxMetrics int
}

SearchQuery is used for sending search queries from vmselect to vmstorage.

func NewSearchQuery added in v1.47.0 

func NewSearchQuery(start, end int64, tagFilterss [][]TagFilter, maxMetrics int) *SearchQuery

NewSearchQuery creates new search query for the given args.

func (*SearchQuery) GetTimeRange added in v1.79.0 

func (sq *SearchQuery) GetTimeRange() TimeRange

GetTimeRange returns time range for the given sq.

func (*SearchQuery) Marshal 

func (sq *SearchQuery) Marshal(dst []byte) []byte

Marshal appends marshaled sq to dst and returns the result.

func (*SearchQuery) String 

func (sq *SearchQuery) String() string

String returns string representation of the search query.

func (*SearchQuery) Unmarshal 

func (sq *SearchQuery) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals sq from src and returns the tail.

type Storage 

type Storage struct {
	// contains filtered or unexported fields
}

Storage represents TSDB storage.

func MustOpenStorage added in v1.91.0 

func MustOpenStorage(path string, opts OpenOptions) *Storage

MustOpenStorage opens storage on the given path with the given retentionMsecs.

func (*Storage) AddRows 

func (s *Storage) AddRows(mrs []MetricRow, precisionBits uint8)

AddRows adds the given mrs to s.

The caller should limit the number of concurrent AddRows calls to the number of available CPU cores in order to limit memory usage.

func (*Storage) DebugFlush added in v1.47.0 

func (s *Storage) DebugFlush()

DebugFlush makes sure all the recently added data is visible to search.

Note: this function doesn't store all the in-memory data to disk - it just converts recently added items to searchable parts, which can be stored either in memory (if they are quite small) or to persistent disk.

This function is for debugging and testing purposes only, since it may slow down data ingestion when used frequently.

func (*Storage) DeleteSeries added in v1.79.0 

func (s *Storage) DeleteSeries(qt *querytracer.Tracer, tfss []*TagFilters, maxMetrics int) (int, error)

DeleteSeries deletes the series matching the given tfss.

If the number of the series exceeds maxMetrics, no series will be deleted and an error will be returned. Otherwise, the function returns the number of metrics deleted.

func (*Storage) DeleteSnapshot 

func (s *Storage) DeleteSnapshot(snapshotName string) error

DeleteSnapshot deletes the given snapshot.

func (*Storage) ForceMergePartitions added in v1.41.1 

func (s *Storage) ForceMergePartitions(partitionNamePrefix string) error

ForceMergePartitions force-merges partitions in s with names starting from the given partitionNamePrefix.

Partitions are merged sequentially in order to reduce load on the system.

func (*Storage) GetMetricNamesStats added in v1.102.16 

func (s *Storage) GetMetricNamesStats(_ *querytracer.Tracer, limit, le int, matchPattern string) MetricNamesStatsResponse

GetMetricNamesStats returns metric names usage stats with given limit and le predicate

func (*Storage) GetSeriesCount 

func (s *Storage) GetSeriesCount(deadline uint64) (uint64, error)

GetSeriesCount returns the approximate number of unique time series.

It includes the deleted series too and may count the same series up to two times - in db and extDB.

func (*Storage) GetTSDBStatus added in v1.78.0 

func (s *Storage) GetTSDBStatus(qt *querytracer.Tracer, tfss []*TagFilters, date uint64, focusLabel string, topN, maxMetrics int, deadline uint64) (*TSDBStatus, error)

GetTSDBStatus returns TSDB status data for /api/v1/status/tsdb

If -disablePerDayIndex flag is not set, the status is calculated for the given date, i.e. the per-day index are used for calculation.

Otherwise, the date is ignored and the status is calculated for the entire retention period, i.e. the global index are used for calculation.

func (*Storage) IsReadOnly added in v1.67.0 

func (s *Storage) IsReadOnly() bool

IsReadOnly returns information is storage in read only mode

func (*Storage) MustClose 

func (s *Storage) MustClose()

MustClose closes the storage.

It is expected that the s is no longer used during the close.

func (*Storage) MustCreateSnapshot added in v1.102.18 

func (s *Storage) MustCreateSnapshot() string

MustCreateSnapshot creates snapshot for s and returns the snapshot name.

The method panics in case of any error since it does not accept any user input and therefore the error is not recoverable.

func (*Storage) MustDeleteStaleSnapshots added in v1.102.18 

func (s *Storage) MustDeleteStaleSnapshots(maxAge time.Duration)

MustDeleteStaleSnapshots deletes snapshot older than given maxAge

The method panics in case of any error since it is unrelated to the user input and indicates a bug in storage or a problem with the underlying file system.

func (*Storage) MustListSnapshots added in v1.102.18 

func (s *Storage) MustListSnapshots() []string

MustListSnapshots returns sorted list of existing snapshots for s.

The method panics in case of any error since it does not accept any user input and therefore the error is not recoverable.

func (*Storage) RegisterMetricNames added in v1.47.0 

func (s *Storage) RegisterMetricNames(qt *querytracer.Tracer, mrs []MetricRow)

RegisterMetricNames registers all the metric names from mrs in the indexdb, so they can be queried later.

The the MetricRow.Timestamp is used for registering the metric name at the given day according to the timestamp. Th MetricRow.Value field is ignored.

func (*Storage) ResetMetricNamesStats added in v1.102.16 

func (s *Storage) ResetMetricNamesStats(_ *querytracer.Tracer)

ResetMetricNamesStats resets state for metric names usage tracker

func (*Storage) SearchGraphitePaths added in v1.53.0 

func (s *Storage) SearchGraphitePaths(qt *querytracer.Tracer, tr TimeRange, query []byte, maxPaths int, deadline uint64) ([]string, error)

SearchGraphitePaths returns all the matching paths for the given graphite query on the given tr.

If -disablePerDayIndex flag is not set, the graphite paths are searched within the given time range (as long as the time range is no more than 40 days), i.e. the per-day index are used for searching.

If -disablePerDayIndex is set or the time range is more than 40 days, the time range is ignored and the graphite paths are searched within the entire retention period, i.e. global index are used for searching.

func (*Storage) SearchLabelNames added in v1.102.14 

func (s *Storage) SearchLabelNames(qt *querytracer.Tracer, tfss []*TagFilters, tr TimeRange, maxLabelNames, maxMetrics int, deadline uint64) ([]string, error)

SearchLabelNames searches for label names matching the given tfss on tr.

If -disablePerDayIndex flag is not set, the label names are searched within the given time range (as long as the time range is no more than 40 days), i.e. the per-day index are used for searching.

If -disablePerDayIndex is set or the time range is more than 40 days, the time range is ignored and the label names are searched within the entire retention period, i.e. the global index are used for searching.

func (*Storage) SearchLabelValues added in v1.102.14 

func (s *Storage) SearchLabelValues(qt *querytracer.Tracer, labelName string, tfss []*TagFilters, tr TimeRange, maxLabelValues, maxMetrics int, deadline uint64) ([]string, error)

SearchLabelValues searches for label values for the given labelName, filters and tr.

If -disablePerDayIndex flag is not set, the label values are searched within the given time range (as long as the time range is no more than 40 days), i.e. the per-day index are used for searching.

If -disablePerDayIndex is set or the time range is more than 40 days, the time range is ignored and the label values are searched within the entire retention period, i.e. the global index are used for searching.

func (*Storage) SearchMetricNames added in v1.47.0 

func (s *Storage) SearchMetricNames(qt *querytracer.Tracer, tfss []*TagFilters, tr TimeRange, maxMetrics int, deadline uint64) ([]string, error)

SearchMetricNames returns marshaled metric names matching the given tfss on the given tr.

The marshaled metric names must be unmarshaled via MetricName.UnmarshalString().

If -disablePerDayIndex flag is not set, the metric names are searched within the given time range (as long as the time range is no more than 40 days), i.e. the per-day index are used for searching.

If -disablePerDayIndex is set or the time range is more than 40 days, the time range is ignored and the metrics are searched within the entire retention period, i.e. the global index are used for searching.

func (*Storage) SearchTagValueSuffixes added in v1.41.0 

func (s *Storage) SearchTagValueSuffixes(qt *querytracer.Tracer, tr TimeRange, tagKey, tagValuePrefix string,
	delimiter byte, maxTagValueSuffixes int, deadline uint64,
) ([]string, error)

SearchTagValueSuffixes returns all the tag value suffixes for the given tagKey and tagValuePrefix on the given tr.

This allows implementing https://graphite-api.readthedocs.io/en/latest/api.html#metrics-find or similar APIs.

If more than maxTagValueSuffixes suffixes is found, then only the first maxTagValueSuffixes suffixes is returned.

If -disablePerDayIndex flag is not set, the tag value suffixes are searched within the given time range (as long as the time range is no more than 40 days), i.e. the per-day index are used for searching.

If -disablePerDayIndex is set or the time range is more than 40 days, the time range is ignored and the tag value suffixes are searched within the entire retention period, i.e. the global index are used for searching.

func (*Storage) UpdateMetrics 

func (s *Storage) UpdateMetrics(m *Metrics)

UpdateMetrics updates m with metrics from s.

type TSDBStatus added in v1.35.0 

type TSDBStatus struct {
	TotalSeries                  uint64
	TotalLabelValuePairs         uint64
	SeriesCountByMetricName      []TopHeapEntry
	SeriesCountByLabelName       []TopHeapEntry
	SeriesCountByFocusLabelValue []TopHeapEntry
	SeriesCountByLabelValuePair  []TopHeapEntry
	LabelValueCountByLabelName   []TopHeapEntry
	SeriesQueryStatsByMetricName []MetricNamesStatsRecord
}

TSDBStatus contains TSDB status data for /api/v1/status/tsdb.

See https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-stats

type TSID 

type TSID struct {
	// MetricGroupID is the id of metric group.
	//
	// MetricGroupID must be unique.
	//
	// Metric group contains metrics with the identical name like
	// 'memory_usage', 'http_requests', but with different
	// labels. For instance, the following metrics belong
	// to a metric group 'memory_usage':
	//
	//   memory_usage{datacenter="foo1", job="bar1", instance="baz1:1234"}
	//   memory_usage{datacenter="foo1", job="bar1", instance="baz2:1234"}
	//   memory_usage{datacenter="foo1", job="bar2", instance="baz1:1234"}
	//   memory_usage{datacenter="foo2", job="bar1", instance="baz2:1234"}
	MetricGroupID uint64

	// JobID is the id of an individual job (aka service).
	//
	// JobID must be unique.
	//
	// Service may consist of multiple instances.
	// See https://prometheus.io/docs/concepts/jobs_instances/ for details.
	JobID uint32

	// InstanceID is the id of an instance (aka process).
	//
	// InstanceID must be unique.
	//
	// See https://prometheus.io/docs/concepts/jobs_instances/ for details.
	InstanceID uint32

	// MetricID is the unique id of the metric (time series).
	//
	// All the other TSID fields may be obtained by MetricID.
	MetricID uint64
}

TSID is unique id for a time series.

Time series blocks are sorted by TSID.

All the fields except MetricID are optional. They exist solely for better grouping of related metrics. It is OK if their meaning differ from their naming.

func (*TSID) Less 

func (t *TSID) Less(b *TSID) bool

Less return true if t < b.

func (*TSID) Marshal 

func (t *TSID) Marshal(dst []byte) []byte

Marshal appends marshaled t to dst and returns the result.

func (*TSID) Unmarshal 

func (t *TSID) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals t from src and returns the rest of src.

type TableMetrics 

type TableMetrics struct {

	// LastPartition contains metrics for the last partition.
	// These metrics are important, since the majority of data ingestion
	// and querying goes to the last partition.
	LastPartition partitionMetrics

	PartitionsRefCount uint64
	// contains filtered or unexported fields
}

TableMetrics contains essential metrics for the table.

func (*TableMetrics) TotalRowsCount added in v1.85.0 

func (pm *TableMetrics) TotalRowsCount() uint64

TotalRowsCount returns total number of rows in tm.

type Tag 

type Tag struct {
	Key   []byte
	Value []byte
}

Tag represents a (key, value) tag for metric.

func (*Tag) Equal added in v1.28.0 

func (tag *Tag) Equal(t *Tag) bool

Equal returns true if tag equals t

func (*Tag) Marshal 

func (tag *Tag) Marshal(dst []byte) []byte

Marshal appends marshaled tag to dst and returns the result.

func (*Tag) Reset added in v1.28.0 

func (tag *Tag) Reset()

Reset resets the tag.

func (*Tag) Unmarshal 

func (tag *Tag) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals tag from src and returns the remaining data from src.

type TagFilter 

type TagFilter struct {
	Key        []byte
	Value      []byte
	IsNegative bool
	IsRegexp   bool
}

TagFilter represents a single tag filter from SearchQuery.

func (*TagFilter) Marshal 

func (tf *TagFilter) Marshal(dst []byte) []byte

Marshal appends marshaled tf to dst and returns the result.

func (*TagFilter) String 

func (tf *TagFilter) String() string

String returns string representation of tf.

func (*TagFilter) Unmarshal 

func (tf *TagFilter) Unmarshal(src []byte) ([]byte, error)

Unmarshal unmarshals tf from src and returns the tail.

type TagFilters 

type TagFilters struct {
	// contains filtered or unexported fields
}

TagFilters represents filters used for filtering tags.

func NewTagFilters 

func NewTagFilters() *TagFilters

NewTagFilters returns new TagFilters.

func (*TagFilters) Add 

func (tfs *TagFilters) Add(key, value []byte, isNegative, isRegexp bool) error

Add adds the given tag filter to tfs.

MetricGroup must be encoded with nil key.

func (*TagFilters) AddGraphiteQuery added in v1.53.0 

func (tfs *TagFilters) AddGraphiteQuery(query []byte, paths []string, isNegative bool)

AddGraphiteQuery adds the given Graphite query that matches the given paths to tfs.

func (*TagFilters) Reset 

func (tfs *TagFilters) Reset()

Reset resets the tf

func (*TagFilters) String 

func (tfs *TagFilters) String() string

String returns human-readable value for tfs.

type TimeRange 

type TimeRange struct {
	MinTimestamp int64
	MaxTimestamp int64
}

TimeRange is time range.

func (*TimeRange) DateRange added in v1.102.14 

func (tr *TimeRange) DateRange() (uint64, uint64)

DateRange returns the date range for the given time range.

func (*TimeRange) String 

func (tr *TimeRange) String() string

type TopHeapEntry added in v1.35.0 

type TopHeapEntry struct {
	Name  string
	Count uint64
}

TopHeapEntry represents an entry from `top heap` used in stats.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.