Version: v1.9.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: May 14, 2021 License: Apache-2.0 Imports: 17 Imported by: 13




View Source
const (
	LocalIngestionRateStrategy  = "local"
	GlobalIngestionRateStrategy = "global"

Supported values for enum limits

View Source
const (

	// ErrQueryTooLong is used in chunk store, querier and query frontend.
	ErrQueryTooLong = "the query time range exceeds the limit (query length: %s, limit: %s)"

	// RateLimited is one of the values for the reason to discard samples.
	// Declared here to avoid duplication in ingester and distributor.
	RateLimited = "rate_limited"

	// Too many HA clusters is one of the reasons for discarding samples.
	TooManyHAClusters = "too_many_ha_clusters"


View Source
var DiscardedMetadata = prometheus.NewCounterVec(
		Name: "cortex_discarded_metadata_total",
		Help: "The total number of metadata that were discarded.",
	[]string{discardReasonLabel, "user"},

DiscardedMetadata is a metric of the number of discarded metadata, by reason.

View Source
var DiscardedSamples = prometheus.NewCounterVec(
		Name: "cortex_discarded_samples_total",
		Help: "The total number of samples that were discarded.",
	[]string{discardReasonLabel, "user"},

DiscardedSamples is a metric of the number of discarded samples, by reason.


func DeletePerUserValidationMetrics added in v1.8.0

func DeletePerUserValidationMetrics(userID string, log log.Logger)

func MaxDurationPerTenant added in v1.7.0

func MaxDurationPerTenant(tenantIDs []string, f func(string) time.Duration) time.Duration

MaxDurationPerTenant is returning the maximum duration per tenant. Without tenants given it will return a time.Duration(0).

func SetDefaultLimitsForYAMLUnmarshalling added in v0.6.0

func SetDefaultLimitsForYAMLUnmarshalling(defaults Limits)

SetDefaultLimitsForYAMLUnmarshalling sets global default limits, used when loading Limits from YAML files. This is used to ensure per-tenant limits are defaulted to those values.

func SmallestPositiveIntPerTenant added in v1.7.0

func SmallestPositiveIntPerTenant(tenantIDs []string, f func(string) int) int

SmallestPositiveIntPerTenant is returning the minimal positive value of the supplied limit function for all given tenants.

func SmallestPositiveNonZeroDurationPerTenant added in v1.7.0

func SmallestPositiveNonZeroDurationPerTenant(tenantIDs []string, f func(string) time.Duration) time.Duration

SmallestPositiveNonZeroDurationPerTenant is returning the minimal positive and non-zero value of the supplied limit function for all given tenants. In many limits a value of 0 means unlimted so the method will return 0 only if all inputs have a limit of 0 or an empty tenant list is given.

func SmallestPositiveNonZeroIntPerTenant added in v1.7.0

func SmallestPositiveNonZeroIntPerTenant(tenantIDs []string, f func(string) int) int

SmallestPositiveNonZeroIntPerTenant is returning the minimal positive and non-zero value of the supplied limit function for all given tenants. In many limits a value of 0 means unlimted so the method will return 0 only if all inputs have a limit of 0 or an empty tenant list is given.

func ValidateMetadata added in v1.1.0

func ValidateMetadata(cfg MetadataValidationConfig, userID string, metadata *cortexpb.MetricMetadata) error

ValidateMetadata returns an err if a metric metadata is invalid.


type LabelValidationConfig added in v0.2.0

type LabelValidationConfig interface {
	EnforceMetricName(userID string) bool
	MaxLabelNamesPerSeries(userID string) int
	MaxLabelNameLength(userID string) int
	MaxLabelValueLength(userID string) int

LabelValidationConfig helps with getting required config to validate labels.

type LimitError added in v1.4.0

type LimitError string

LimitError are errors that do not comply with the limits specified.

func (LimitError) Error added in v1.4.0

func (e LimitError) Error() string

type Limits

type Limits struct {
	// Distributor enforced limits.
	IngestionRate             float64             `yaml:"ingestion_rate" json:"ingestion_rate"`
	IngestionRateStrategy     string              `yaml:"ingestion_rate_strategy" json:"ingestion_rate_strategy"`
	IngestionBurstSize        int                 `yaml:"ingestion_burst_size" json:"ingestion_burst_size"`
	AcceptHASamples           bool                `yaml:"accept_ha_samples" json:"accept_ha_samples"`
	HAClusterLabel            string              `yaml:"ha_cluster_label" json:"ha_cluster_label"`
	HAReplicaLabel            string              `yaml:"ha_replica_label" json:"ha_replica_label"`
	HAMaxClusters             int                 `yaml:"ha_max_clusters" json:"ha_max_clusters"`
	DropLabels                flagext.StringSlice `yaml:"drop_labels" json:"drop_labels"`
	MaxLabelNameLength        int                 `yaml:"max_label_name_length" json:"max_label_name_length"`
	MaxLabelValueLength       int                 `yaml:"max_label_value_length" json:"max_label_value_length"`
	MaxLabelNamesPerSeries    int                 `yaml:"max_label_names_per_series" json:"max_label_names_per_series"`
	MaxMetadataLength         int                 `yaml:"max_metadata_length" json:"max_metadata_length"`
	RejectOldSamples          bool                `yaml:"reject_old_samples" json:"reject_old_samples"`
	RejectOldSamplesMaxAge    model.Duration      `yaml:"reject_old_samples_max_age" json:"reject_old_samples_max_age"`
	CreationGracePeriod       model.Duration      `yaml:"creation_grace_period" json:"creation_grace_period"`
	EnforceMetadataMetricName bool                `yaml:"enforce_metadata_metric_name" json:"enforce_metadata_metric_name"`
	EnforceMetricName         bool                `yaml:"enforce_metric_name" json:"enforce_metric_name"`
	IngestionTenantShardSize  int                 `yaml:"ingestion_tenant_shard_size" json:"ingestion_tenant_shard_size"`
	MetricRelabelConfigs      []*relabel.Config   `` /* 295-byte string literal not displayed */

	// Ingester enforced limits.
	// Series
	MaxSeriesPerQuery        int `yaml:"max_series_per_query" json:"max_series_per_query"`
	MaxSamplesPerQuery       int `yaml:"max_samples_per_query" json:"max_samples_per_query"`
	MaxLocalSeriesPerUser    int `yaml:"max_series_per_user" json:"max_series_per_user"`
	MaxLocalSeriesPerMetric  int `yaml:"max_series_per_metric" json:"max_series_per_metric"`
	MaxGlobalSeriesPerUser   int `yaml:"max_global_series_per_user" json:"max_global_series_per_user"`
	MaxGlobalSeriesPerMetric int `yaml:"max_global_series_per_metric" json:"max_global_series_per_metric"`
	MinChunkLength           int `yaml:"min_chunk_length" json:"min_chunk_length"`
	// Metadata
	MaxLocalMetricsWithMetadataPerUser  int `yaml:"max_metadata_per_user" json:"max_metadata_per_user"`
	MaxLocalMetadataPerMetric           int `yaml:"max_metadata_per_metric" json:"max_metadata_per_metric"`
	MaxGlobalMetricsWithMetadataPerUser int `yaml:"max_global_metadata_per_user" json:"max_global_metadata_per_user"`
	MaxGlobalMetadataPerMetric          int `yaml:"max_global_metadata_per_metric" json:"max_global_metadata_per_metric"`

	// Querier enforced limits.
	MaxChunksPerQuery    int            `yaml:"max_chunks_per_query" json:"max_chunks_per_query"`
	MaxQueryLookback     model.Duration `yaml:"max_query_lookback" json:"max_query_lookback"`
	MaxQueryLength       model.Duration `yaml:"max_query_length" json:"max_query_length"`
	MaxQueryParallelism  int            `yaml:"max_query_parallelism" json:"max_query_parallelism"`
	CardinalityLimit     int            `yaml:"cardinality_limit" json:"cardinality_limit"`
	MaxCacheFreshness    model.Duration `yaml:"max_cache_freshness" json:"max_cache_freshness"`
	MaxQueriersPerTenant int            `yaml:"max_queriers_per_tenant" json:"max_queriers_per_tenant"`

	// Ruler defaults and limits.
	RulerEvaluationDelay        model.Duration `yaml:"ruler_evaluation_delay_duration" json:"ruler_evaluation_delay_duration"`
	RulerTenantShardSize        int            `yaml:"ruler_tenant_shard_size" json:"ruler_tenant_shard_size"`
	RulerMaxRulesPerRuleGroup   int            `yaml:"ruler_max_rules_per_rule_group" json:"ruler_max_rules_per_rule_group"`
	RulerMaxRuleGroupsPerTenant int            `yaml:"ruler_max_rule_groups_per_tenant" json:"ruler_max_rule_groups_per_tenant"`

	// Store-gateway.
	StoreGatewayTenantShardSize int `yaml:"store_gateway_tenant_shard_size" json:"store_gateway_tenant_shard_size"`

	// Compactor.
	CompactorBlocksRetentionPeriod model.Duration `yaml:"compactor_blocks_retention_period" json:"compactor_blocks_retention_period"`

	// This config doesn't have a CLI flag registered here because they're registered in
	// their own original config struct.
	S3SSEType                 string `` /* 221-byte string literal not displayed */
	S3SSEKMSKeyID             string `` /* 156-byte string literal not displayed */
	S3SSEKMSEncryptionContext string `` /* 284-byte string literal not displayed */

Limits describe all the limits for users; can be used to describe global default limits via flags, or per-user limits via yaml config.

func (*Limits) RegisterFlags

func (l *Limits) RegisterFlags(f *flag.FlagSet)

RegisterFlags adds the flags required to config this to the given FlagSet

func (*Limits) UnmarshalJSON added in v1.8.0

func (l *Limits) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*Limits) UnmarshalYAML

func (l *Limits) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML implements the yaml.Unmarshaler interface.

func (*Limits) Validate added in v0.4.0

func (l *Limits) Validate(shardByAllLabels bool) error

Validate the limits config and returns an error if the validation doesn't pass

type MetadataValidationConfig added in v1.1.0

type MetadataValidationConfig interface {
	EnforceMetadataMetricName(userID string) bool
	MaxMetadataLength(userID string) int

MetadataValidationConfig helps with getting required config to validate metadata.

type Overrides

type Overrides struct {
	// contains filtered or unexported fields

Overrides periodically fetch a set of per-user overrides, and provides convenience functions for fetching the correct value.

func NewOverrides

func NewOverrides(defaults Limits, tenantLimits TenantLimits) (*Overrides, error)

NewOverrides makes a new Overrides.

func (*Overrides) AcceptHASamples

func (o *Overrides) AcceptHASamples(userID string) bool

AcceptHASamples returns whether the distributor should track and accept samples from HA replicas for this user.

func (*Overrides) CardinalityLimit

func (o *Overrides) CardinalityLimit(userID string) int

CardinalityLimit returns the maximum number of timeseries allowed in a query.

func (*Overrides) CompactorBlocksRetentionPeriod added in v1.8.0

func (o *Overrides) CompactorBlocksRetentionPeriod(userID string) time.Duration

CompactorBlocksRetentionPeriod returns the retention period for a given user.

func (*Overrides) CreationGracePeriod

func (o *Overrides) CreationGracePeriod(userID string) time.Duration

CreationGracePeriod is misnamed, and actually returns how far into the future we should accept samples.

func (*Overrides) DropLabels added in v0.6.0

func (o *Overrides) DropLabels(userID string) flagext.StringSlice

DropLabels returns the list of labels to be dropped when ingesting HA samples for the user.

func (*Overrides) EnforceMetadataMetricName added in v1.1.0

func (o *Overrides) EnforceMetadataMetricName(userID string) bool

EnforceMetadataMetricName whether to enforce the presence of a metric name on metadata.

func (*Overrides) EnforceMetricName

func (o *Overrides) EnforceMetricName(userID string) bool

EnforceMetricName whether to enforce the presence of a metric name.

func (*Overrides) EvaluationDelay added in v1.4.0

func (o *Overrides) EvaluationDelay(userID string) time.Duration

EvaluationDelay returns the rules evaluation delay for a given user.

func (*Overrides) HAClusterLabel

func (o *Overrides) HAClusterLabel(userID string) string

HAClusterLabel returns the cluster label to look for when deciding whether to accept a sample from a Prometheus HA replica.

func (*Overrides) HAReplicaLabel

func (o *Overrides) HAReplicaLabel(userID string) string

HAReplicaLabel returns the replica label to look for when deciding whether to accept a sample from a Prometheus HA replica.

func (*Overrides) IngestionBurstSize

func (o *Overrides) IngestionBurstSize(userID string) int

IngestionBurstSize returns the burst size for ingestion rate.

func (*Overrides) IngestionRate

func (o *Overrides) IngestionRate(userID string) float64

IngestionRate returns the limit on ingester rate (samples per second).

func (*Overrides) IngestionRateStrategy added in v0.6.0

func (o *Overrides) IngestionRateStrategy() string

IngestionRateStrategy returns whether the ingestion rate limit should be individually applied to each distributor instance (local) or evenly shared across the cluster (global).

func (*Overrides) IngestionTenantShardSize added in v1.5.0

func (o *Overrides) IngestionTenantShardSize(userID string) int

IngestionTenantShardSize returns the ingesters shard size for a given user.

func (*Overrides) MaxCacheFreshness added in v1.2.0

func (o *Overrides) MaxCacheFreshness(userID string) time.Duration

MaxCacheFreshness returns the period after which results are cacheable, to prevent caching of very recent results.

func (*Overrides) MaxChunksPerQuery

func (o *Overrides) MaxChunksPerQuery(userID string) int

MaxChunksPerQuery returns the maximum number of chunks allowed per query.

func (*Overrides) MaxGlobalMetadataPerMetric added in v1.1.0

func (o *Overrides) MaxGlobalMetadataPerMetric(userID string) int

MaxGlobalMetadataPerMetric returns the maximum number of metadata allowed per metric across the cluster.

func (*Overrides) MaxGlobalMetricsWithMetadataPerUser added in v1.1.0

func (o *Overrides) MaxGlobalMetricsWithMetadataPerUser(userID string) int

MaxGlobalMetricsWithMetadataPerUser returns the maximum number of metrics with metadata a user is allowed to store across the cluster.

func (*Overrides) MaxGlobalSeriesPerMetric added in v0.4.0

func (o *Overrides) MaxGlobalSeriesPerMetric(userID string) int

MaxGlobalSeriesPerMetric returns the maximum number of series allowed per metric across the cluster.

func (*Overrides) MaxGlobalSeriesPerUser added in v0.4.0

func (o *Overrides) MaxGlobalSeriesPerUser(userID string) int

MaxGlobalSeriesPerUser returns the maximum number of series a user is allowed to store across the cluster.

func (*Overrides) MaxHAClusters added in v1.7.0

func (o *Overrides) MaxHAClusters(user string) int

MaxHAClusters returns maximum number of clusters that HA tracker will track for a user.

func (*Overrides) MaxLabelNameLength

func (o *Overrides) MaxLabelNameLength(userID string) int

MaxLabelNameLength returns maximum length a label name can be.

func (*Overrides) MaxLabelNamesPerSeries

func (o *Overrides) MaxLabelNamesPerSeries(userID string) int

MaxLabelNamesPerSeries returns maximum number of label/value pairs timeseries.

func (*Overrides) MaxLabelValueLength

func (o *Overrides) MaxLabelValueLength(userID string) int

MaxLabelValueLength returns maximum length a label value can be. This also is the maximum length of a metric name.

func (*Overrides) MaxLocalMetadataPerMetric added in v1.1.0

func (o *Overrides) MaxLocalMetadataPerMetric(userID string) int

MaxLocalMetadataPerMetric returns the maximum number of metadata allowed per metric in a single ingester.

func (*Overrides) MaxLocalMetricsWithMetadataPerUser added in v1.1.0

func (o *Overrides) MaxLocalMetricsWithMetadataPerUser(userID string) int

MaxLocalMetricsWithMetadataPerUser returns the maximum number of metrics with metadata a user is allowed to store in a single ingester.

func (*Overrides) MaxLocalSeriesPerMetric added in v0.4.0

func (o *Overrides) MaxLocalSeriesPerMetric(userID string) int

MaxLocalSeriesPerMetric returns the maximum number of series allowed per metric in a single ingester.

func (*Overrides) MaxLocalSeriesPerUser added in v0.4.0

func (o *Overrides) MaxLocalSeriesPerUser(userID string) int

MaxLocalSeriesPerUser returns the maximum number of series a user is allowed to store in a single ingester.

func (*Overrides) MaxMetadataLength added in v1.1.0

func (o *Overrides) MaxMetadataLength(userID string) int

MaxMetadataLength returns maximum length metadata can be. Metadata refers to the Metric Name, HELP and UNIT.

func (*Overrides) MaxQueriersPerUser added in v1.5.0

func (o *Overrides) MaxQueriersPerUser(userID string) int

MaxQueriersPerUser returns the maximum number of queriers that can handle requests for this user.

func (*Overrides) MaxQueryLength

func (o *Overrides) MaxQueryLength(userID string) time.Duration

MaxQueryLength returns the limit of the length (in time) of a query.

func (*Overrides) MaxQueryLookback added in v1.6.0

func (o *Overrides) MaxQueryLookback(userID string) time.Duration

MaxQueryLookback returns the max lookback period of queries.

func (*Overrides) MaxQueryParallelism

func (o *Overrides) MaxQueryParallelism(userID string) int

MaxQueryParallelism returns the limit to the number of split queries the frontend will process in parallel.

func (*Overrides) MaxSamplesPerQuery

func (o *Overrides) MaxSamplesPerQuery(userID string) int

MaxSamplesPerQuery returns the maximum number of samples in a query (from the ingester).

func (*Overrides) MaxSeriesPerQuery

func (o *Overrides) MaxSeriesPerQuery(userID string) int

MaxSeriesPerQuery returns the maximum number of series a query is allowed to hit.

func (*Overrides) MetricRelabelConfigs added in v1.5.0

func (o *Overrides) MetricRelabelConfigs(userID string) []*relabel.Config

MetricRelabelConfigs returns the metric relabel configs for a given user.

func (*Overrides) MinChunkLength added in v0.2.0

func (o *Overrides) MinChunkLength(userID string) int

MinChunkLength returns the minimum size of chunk that will be saved by ingesters

func (*Overrides) RejectOldSamples

func (o *Overrides) RejectOldSamples(userID string) bool

RejectOldSamples returns true when we should reject samples older than certain age.

func (*Overrides) RejectOldSamplesMaxAge

func (o *Overrides) RejectOldSamplesMaxAge(userID string) time.Duration

RejectOldSamplesMaxAge returns the age at which samples should be rejected.

func (*Overrides) RulerMaxRuleGroupsPerTenant added in v1.5.0

func (o *Overrides) RulerMaxRuleGroupsPerTenant(userID string) int

RulerMaxRuleGroupsPerTenant returns the maximum number of rule groups for a given user.

func (*Overrides) RulerMaxRulesPerRuleGroup added in v1.5.0

func (o *Overrides) RulerMaxRulesPerRuleGroup(userID string) int

RulerMaxRulesPerRuleGroup returns the maximum number of rules per rule group for a given user.

func (*Overrides) RulerTenantShardSize added in v1.5.0

func (o *Overrides) RulerTenantShardSize(userID string) int

RulerTenantShardSize returns shard size (number of rulers) used by this tenant when using shuffle-sharding strategy.

func (*Overrides) S3SSEKMSEncryptionContext added in v1.8.0

func (o *Overrides) S3SSEKMSEncryptionContext(user string) string

S3SSEKMSEncryptionContext returns the per-tenant S3 KMS-SSE encryption context.

func (*Overrides) S3SSEKMSKeyID added in v1.8.0

func (o *Overrides) S3SSEKMSKeyID(user string) string

S3SSEKMSKeyID returns the per-tenant S3 KMS-SSE key id.

func (*Overrides) S3SSEType added in v1.8.0

func (o *Overrides) S3SSEType(user string) string

S3SSEType returns the per-tenant S3 SSE type.

func (*Overrides) StoreGatewayTenantShardSize added in v1.4.0

func (o *Overrides) StoreGatewayTenantShardSize(userID string) int

StoreGatewayTenantShardSize returns the store-gateway shard size for a given user.

type OverridesExporter added in v1.8.0

type OverridesExporter struct {
	// contains filtered or unexported fields

OverridesExporter exposes per-tenant resource limit overrides as Prometheus metrics

func NewOverridesExporter added in v1.8.0

func NewOverridesExporter(tenantLimits TenantLimits) *OverridesExporter

NewOverridesExporter creates an OverridesExporter that reads updates to per-tenant limits using the provided function.

func (*OverridesExporter) Collect added in v1.8.0

func (oe *OverridesExporter) Collect(ch chan<- prometheus.Metric)

func (*OverridesExporter) Describe added in v1.8.0

func (oe *OverridesExporter) Describe(ch chan<- *prometheus.Desc)

type SampleValidationConfig added in v0.2.0

type SampleValidationConfig interface {
	RejectOldSamples(userID string) bool
	RejectOldSamplesMaxAge(userID string) time.Duration
	CreationGracePeriod(userID string) time.Duration

SampleValidationConfig helps with getting required config to validate sample.

type TenantLimits added in v0.6.0

type TenantLimits interface {
	// ByUserID gets limits specific to a particular tenant or nil if there are none
	ByUserID(userID string) *Limits

	// AllByUserID gets a mapping of all tenant IDs and limits for that user
	AllByUserID() map[string]*Limits

TenantLimits exposes per-tenant limit overrides to various resource usage limits

type ValidationError added in v1.9.0

type ValidationError error

ValidationError is an error returned by series validation.

nolint:golint ignore stutter warning

func ValidateLabels added in v0.2.0

func ValidateLabels(cfg LabelValidationConfig, userID string, ls []cortexpb.LabelAdapter, skipLabelNameValidation bool) ValidationError

ValidateLabels returns an err if the labels are invalid. The returned error may retain the provided series labels.

func ValidateSample added in v0.2.0

ValidateSample returns an err if the sample is invalid. The returned error may retain the provided series labels.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL