util

package
v1.3.0-rc.1.0...-65594da Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 29, 2021 License: Apache-2.0 Imports: 44 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// Sharding strategies.
	ShardingStrategyDefault = "default"
	ShardingStrategyShuffle = "shuffle-sharding"
)

Variables

View Source
var ErrStopProcess = errors.New("stop process")

ErrStopProcess is the error returned by a service as a hint to stop the server entirely.

View Source
var (
	// Logger is a shared go-kit logger.
	// TODO: Change all components to take a non-global logger via their constructors.
	Logger = log.NewNopLogger()
)

Functions

func AddSourceIPsToIncomingContext

func AddSourceIPsToIncomingContext(ctx context.Context, source string) context.Context

AddSourceIPsToIncomingContext adds the given source to the GRPC context

func AddSourceIPsToOutgoingContext

func AddSourceIPsToOutgoingContext(ctx context.Context, source string) context.Context

AddSourceIPsToOutgoingContext adds the given source to the GRPC context

func CheckFatal

func CheckFatal(location string, err error)

CheckFatal prints an error and exits with error code 1 if err is non-nil

func DurationWithJitter

func DurationWithJitter(input time.Duration, variancePerc float64) time.Duration

func Event

func Event() log.Logger

Event is the log-like API for event sampling

func GetFirstAddressOf

func GetFirstAddressOf(names []string) (string, error)

GetFirstAddressOf returns the first IPv4 address of the supplied interface names.

func GetSourceIPsFromIncomingCtx

func GetSourceIPsFromIncomingCtx(ctx context.Context) string

GetSourceIPsFromIncomingCtx extracts the source field from the GRPC context

func GetSourceIPsFromOutgoingCtx

func GetSourceIPsFromOutgoingCtx(ctx context.Context) string

GetSourceIPsFromOutgoingCtx extracts the source field from the GRPC context

func HashFP

func HashFP(fp model.Fingerprint) uint32

HashFP simply moves entropy from the most significant 48 bits of the fingerprint into the least significant 16 bits (by XORing) so that a simple MOD on the result can be used to pick a mutex while still making use of changes in more significant bits of the fingerprint. (The fast fingerprinting function we use is prone to only change a few bits for similar metrics. We really want to make use of every change in the fingerprint to vary mutex selection.)

func InitEvents

func InitEvents(freq int)

InitEvents initializes event sampling, with the given frequency. Zero=off.

func InitLogger

func InitLogger(cfg *server.Config)

InitLogger initialises the global gokit logger (util.Logger) and overrides the default logger for the server.

func LabelsToMetric

func LabelsToMetric(ls labels.Labels) model.Metric

LabelsToMetric converts a Labels to Metric Don't do this on any performance sensitive paths.

func Max

func Max(a, b int) int

Max returns the maximum of two ints

func Max64

func Max64(a, b int64) int64

Max64 returns the maximum of two int64s

func MergeNSampleSets

func MergeNSampleSets(sampleSets ...[]model.SamplePair) []model.SamplePair

MergeNSampleSets merges and dedupes n sets of already sorted sample pairs.

func MergeSampleSets

func MergeSampleSets(a, b []model.SamplePair) []model.SamplePair

MergeSampleSets merges and dedupes two sets of already sorted sample pairs.

func Min

func Min(a, b int) int

Min returns the minimum of two ints

func Min64

func Min64(a, b int64) int64

Min64 returns the minimum of two int64s

func NewModuleService

func NewModuleService(name string, service services.Service, startDeps, stopDeps func(string) map[string]services.Service) services.Service

NewModuleService wraps a module service, and makes sure that dependencies are started/stopped before module service starts or stops. If any dependency fails to start, this service fails as well. On stop, errors from failed dependencies are ignored.

func NewPrometheusLogger

func NewPrometheusLogger(l logging.Level, format logging.Format) (log.Logger, error)

NewPrometheusLogger creates a new instance of PrometheusLogger which exposes Prometheus counters for various log levels.

func ParseProtoReader

func ParseProtoReader(ctx context.Context, reader io.Reader, expectedSize, maxSize int, req proto.Message, compression CompressionType) error

ParseProtoReader parses a compressed proto from an io.Reader.

func ParseTime

func ParseTime(s string) (int64, error)

ParseTime parses the string into an int64, milliseconds since epoch.

func RenderHTTPResponse

func RenderHTTPResponse(w http.ResponseWriter, v interface{}, t *template.Template, r *http.Request)

RenderHTTPResponse either responds with json or a rendered html page using the passed in template by checking the Accepts header

func SerializeProtoResponse

func SerializeProtoResponse(w http.ResponseWriter, resp proto.Message, compression CompressionType) error

SerializeProtoResponse serializes a protobuf response into an HTTP response.

func ShuffleShardExpectedInstances

func ShuffleShardExpectedInstances(shardSize, numZones int) int

ShuffleShardExpectedInstances returns the total number of instances that should be selected for a given tenant. If zone-aware replication is disabled, the input numZones should be 1.

func ShuffleShardExpectedInstancesPerZone

func ShuffleShardExpectedInstancesPerZone(shardSize, numZones int) int

ShuffleShardExpectedInstancesPerZone returns the number of instances that should be selected for each zone when zone-aware replication is enabled. The algorithm expects the shard size to be divisible by the number of zones, in order to have nodes balanced across zones. If it's not, we do round up.

func ShuffleShardSeed

func ShuffleShardSeed(identifier, zone string) int64

ShuffleShardSeed returns seed for random number generator, computed from provided identifier.

func SplitFiltersAndMatchers

func SplitFiltersAndMatchers(allMatchers []*labels.Matcher) (filters, matchers []*labels.Matcher)

SplitFiltersAndMatchers splits empty matchers off, which are treated as filters, see #220

func StringsContain

func StringsContain(values []string, search string) bool

StringsContain returns true if the search value is within the list of input values.

func TimeFromMillis

func TimeFromMillis(ms int64) time.Time

TimeFromMillis is a helper to turn milliseconds -> time.Time

func TimeToMillis

func TimeToMillis(t time.Time) int64

func WaitGroup

func WaitGroup(ctx context.Context, wg *sync.WaitGroup) error

WaitGroup calls Wait() on a sync.WaitGroup and return once the Wait() completed or the context is cancelled or times out, whatever occurs first. Returns the specific context error if the context is cancelled or times out before Wait() completes.

func WarnExperimentalUse

func WarnExperimentalUse(feature string)

WarnExperimentalUse logs a warning and increments the experimental features metric.

func WithContext

func WithContext(ctx context.Context, l log.Logger) log.Logger

WithContext returns a Logger that has information about the current user in its details.

e.g.

log := util.WithContext(ctx)
log.Errorf("Could not chunk chunks: %v", err)

func WithSourceIPs

func WithSourceIPs(sourceIPs string, l log.Logger) log.Logger

WithSourceIPs returns a Logger that has information about the source IPs in its details.

func WithTraceID

func WithTraceID(traceID string, l log.Logger) log.Logger

WithTraceID returns a Logger that has information about the traceID in its details.

func WithUserID

func WithUserID(userID string, l log.Logger) log.Logger

WithUserID returns a Logger that has information about the current user in its details.

func WriteJSONResponse

func WriteJSONResponse(w http.ResponseWriter, v interface{})

WriteJSONResponse writes some JSON as a HTTP response.

func YoloBuf

func YoloBuf(s string) []byte

Types

type Backoff

type Backoff struct {
	// contains filtered or unexported fields
}

Backoff implements exponential backoff with randomized wait times

func NewBackoff

func NewBackoff(ctx context.Context, cfg BackoffConfig) *Backoff

NewBackoff creates a Backoff object. Pass a Context that can also terminate the operation.

func (*Backoff) Err

func (b *Backoff) Err() error

Err returns the reason for terminating the backoff, or nil if it didn't terminate

func (*Backoff) NextDelay

func (b *Backoff) NextDelay() time.Duration

func (*Backoff) NumRetries

func (b *Backoff) NumRetries() int

NumRetries returns the number of retries so far

func (*Backoff) Ongoing

func (b *Backoff) Ongoing() bool

Ongoing returns true if caller should keep going

func (*Backoff) Reset

func (b *Backoff) Reset()

Reset the Backoff back to its initial condition

func (*Backoff) Wait

func (b *Backoff) Wait()

Wait sleeps for the backoff time then increases the retry count and backoff time Returns immediately if Context is terminated

type BackoffConfig

type BackoffConfig struct {
	MinBackoff time.Duration `yaml:"min_period"`  // start backoff at this level
	MaxBackoff time.Duration `yaml:"max_period"`  // increase exponentially to this level
	MaxRetries int           `yaml:"max_retries"` // give up after this many; zero means infinite retries
}

BackoffConfig configures a Backoff

func (*BackoffConfig) RegisterFlags

func (cfg *BackoffConfig) RegisterFlags(prefix string, f *flag.FlagSet)

RegisterFlags for BackoffConfig.

type CompressionType

type CompressionType int

CompressionType for encoding and decoding requests and responses.

const (
	NoCompression CompressionType = iota
	FramedSnappy
	RawSnappy
)

Values for CompressionType

func CompressionTypeFor

func CompressionTypeFor(version string) CompressionType

CompressionTypeFor a given version of the Prometheus remote storage protocol. See https://github.com/prometheus/prometheus/issues/2692.

type HistogramData

type HistogramData struct {
	// contains filtered or unexported fields
}

HistogramData keeps data required to build histogram Metric

func (*HistogramData) AddHistogram

func (d *HistogramData) AddHistogram(histo *dto.Histogram)

Adds histogram from gathered metrics to this histogram data.

func (*HistogramData) AddHistogramData

func (d *HistogramData) AddHistogramData(histo HistogramData)

Merged another histogram data into this one.

func (*HistogramData) Metric

func (d *HistogramData) Metric(desc *prometheus.Desc, labelValues ...string) prometheus.Metric

Return prometheus metric from this histogram data.

type HistogramDataCollector

type HistogramDataCollector struct {
	// contains filtered or unexported fields
}

HistogramDataCollector combines histogram data, with prometheus descriptor. It can be registered into prometheus to report histogram with stored data. Data can be updated via Add method.

func NewHistogramDataCollector

func NewHistogramDataCollector(desc *prometheus.Desc) *HistogramDataCollector

Creates new histogram data collector.

func (*HistogramDataCollector) Add

func (*HistogramDataCollector) Collect

func (h *HistogramDataCollector) Collect(out chan<- prometheus.Metric)

func (*HistogramDataCollector) Describe

func (h *HistogramDataCollector) Describe(out chan<- *prometheus.Desc)

type MetricFamiliesPerUser

type MetricFamiliesPerUser map[string]MetricFamilyMap

MetricFamiliesPerUser is a collection of metrics gathered via calling Gatherer.Gather() method on different gatherers, one per user.

func BuildMetricFamiliesPerUserFromUserRegistries

func BuildMetricFamiliesPerUserFromUserRegistries(regs map[string]*prometheus.Registry) MetricFamiliesPerUser

func (MetricFamiliesPerUser) GetSumOfCounters

func (d MetricFamiliesPerUser) GetSumOfCounters(counter string) float64

func (MetricFamiliesPerUser) GetSumOfGauges

func (d MetricFamiliesPerUser) GetSumOfGauges(gauge string) float64

func (MetricFamiliesPerUser) SendMaxOfGauges

func (d MetricFamiliesPerUser) SendMaxOfGauges(out chan<- prometheus.Metric, desc *prometheus.Desc, gauge string)

func (MetricFamiliesPerUser) SendSumOfCounters

func (d MetricFamiliesPerUser) SendSumOfCounters(out chan<- prometheus.Metric, desc *prometheus.Desc, counter string)

func (MetricFamiliesPerUser) SendSumOfCountersPerUser

func (d MetricFamiliesPerUser) SendSumOfCountersPerUser(out chan<- prometheus.Metric, desc *prometheus.Desc, counter string)

func (MetricFamiliesPerUser) SendSumOfCountersPerUserWithLabels

func (d MetricFamiliesPerUser) SendSumOfCountersPerUserWithLabels(out chan<- prometheus.Metric, desc *prometheus.Desc, metric string, labelNames ...string)

SendSumOfCountersPerUserWithLabels provides metrics with the provided label names on a per-user basis. This function assumes that `user` is the first label on the provided metric Desc

func (MetricFamiliesPerUser) SendSumOfCountersWithLabels

func (d MetricFamiliesPerUser) SendSumOfCountersWithLabels(out chan<- prometheus.Metric, desc *prometheus.Desc, counter string, labelNames ...string)

func (MetricFamiliesPerUser) SendSumOfGauges

func (d MetricFamiliesPerUser) SendSumOfGauges(out chan<- prometheus.Metric, desc *prometheus.Desc, gauge string)

func (MetricFamiliesPerUser) SendSumOfGaugesPerUserWithLabels

func (d MetricFamiliesPerUser) SendSumOfGaugesPerUserWithLabels(out chan<- prometheus.Metric, desc *prometheus.Desc, metric string, labelNames ...string)

SendSumOfGaugesPerUserWithLabels provides metrics with the provided label names on a per-user basis. This function assumes that `user` is the first label on the provided metric Desc

func (MetricFamiliesPerUser) SendSumOfGaugesWithLabels

func (d MetricFamiliesPerUser) SendSumOfGaugesWithLabels(out chan<- prometheus.Metric, desc *prometheus.Desc, gauge string, labelNames ...string)

func (MetricFamiliesPerUser) SendSumOfHistograms

func (d MetricFamiliesPerUser) SendSumOfHistograms(out chan<- prometheus.Metric, desc *prometheus.Desc, histogramName string)

func (MetricFamiliesPerUser) SendSumOfHistogramsWithLabels

func (d MetricFamiliesPerUser) SendSumOfHistogramsWithLabels(out chan<- prometheus.Metric, desc *prometheus.Desc, histogramName string, labelNames ...string)

func (MetricFamiliesPerUser) SendSumOfSummaries

func (d MetricFamiliesPerUser) SendSumOfSummaries(out chan<- prometheus.Metric, desc *prometheus.Desc, summaryName string)

func (MetricFamiliesPerUser) SendSumOfSummariesPerUser

func (d MetricFamiliesPerUser) SendSumOfSummariesPerUser(out chan<- prometheus.Metric, desc *prometheus.Desc, summaryName string)

func (MetricFamiliesPerUser) SendSumOfSummariesWithLabels

func (d MetricFamiliesPerUser) SendSumOfSummariesWithLabels(out chan<- prometheus.Metric, desc *prometheus.Desc, summaryName string, labelNames ...string)

type MetricFamilyMap

type MetricFamilyMap map[string]*dto.MetricFamily

MetricFamilyMap is a map of metric names to their family (metrics with same name, but different labels) Keeping map of metric name to its family makes it easier to do searches later.

func NewMetricFamilyMap

func NewMetricFamilyMap(metrics []*dto.MetricFamily) (MetricFamilyMap, error)

NewMetricFamilyMap sorts output from Gatherer.Gather method into a map. Gatherer.Gather specifies that there metric families are uniquely named, and we use that fact here. If they are not, this method returns error.

func (MetricFamilyMap) MaxGauges

func (mfm MetricFamilyMap) MaxGauges(name string) float64

func (MetricFamilyMap) SumCounters

func (mfm MetricFamilyMap) SumCounters(name string) float64

func (MetricFamilyMap) SumGauges

func (mfm MetricFamilyMap) SumGauges(name string) float64

func (MetricFamilyMap) SumHistograms

func (mfm MetricFamilyMap) SumHistograms(name string) HistogramData

func (MetricFamilyMap) SumHistogramsTo

func (mfm MetricFamilyMap) SumHistogramsTo(name string, output *HistogramData)

func (MetricFamilyMap) SumSummaries

func (mfm MetricFamilyMap) SumSummaries(name string) SummaryData

func (MetricFamilyMap) SumSummariesTo

func (mfm MetricFamilyMap) SumSummariesTo(name string, output *SummaryData)

type Op

type Op interface {
	Key() string
	Priority() int64 // The larger the number the higher the priority.
}

Op is an operation on the priority queue.

type PriorityQueue

type PriorityQueue struct {
	// contains filtered or unexported fields
}

PriorityQueue is a priority queue.

func NewPriorityQueue

func NewPriorityQueue(lengthGauge prometheus.Gauge) *PriorityQueue

NewPriorityQueue makes a new priority queue.

func (*PriorityQueue) Close

func (pq *PriorityQueue) Close()

Close signals that the queue should be closed when it is empty. A closed queue will not accept new items.

func (*PriorityQueue) Dequeue

func (pq *PriorityQueue) Dequeue() Op

Dequeue will return the op with the highest priority; block if queue is empty; returns nil if queue is closed.

func (*PriorityQueue) DiscardAndClose

func (pq *PriorityQueue) DiscardAndClose()

DiscardAndClose closes the queue and removes all the items from it.

func (*PriorityQueue) Enqueue

func (pq *PriorityQueue) Enqueue(op Op) bool

Enqueue adds an operation to the queue in priority order. Returns true if added; false if the operation was already on the queue.

func (*PriorityQueue) Length

func (pq *PriorityQueue) Length() int

Length returns the length of the queue.

type PrometheusLogger

type PrometheusLogger struct {
	// contains filtered or unexported fields
}

PrometheusLogger exposes Prometheus counters for each of go-kit's log levels.

func (*PrometheusLogger) Log

func (pl *PrometheusLogger) Log(kv ...interface{}) error

Log increments the appropriate Prometheus counter depending on the log level.

type SampleStreamIterator

type SampleStreamIterator struct {
	// contains filtered or unexported fields
}

SampleStreamIterator is a struct and not just a renamed type because otherwise the Metric field and Metric() methods would clash.

func NewSampleStreamIterator

func NewSampleStreamIterator(ss *model.SampleStream) SampleStreamIterator

NewSampleStreamIterator creates a SampleStreamIterator

func (SampleStreamIterator) Close

func (it SampleStreamIterator) Close()

Close implements the SeriesIterator interface.

func (SampleStreamIterator) Metric

func (it SampleStreamIterator) Metric() metric.Metric

Metric implements the SeriesIterator interface.

func (SampleStreamIterator) RangeValues

func (it SampleStreamIterator) RangeValues(in metric.Interval) []model.SamplePair

RangeValues implements the SeriesIterator interface.

func (SampleStreamIterator) ValueAtOrBeforeTime

func (it SampleStreamIterator) ValueAtOrBeforeTime(ts model.Time) model.SamplePair

ValueAtOrBeforeTime implements the SeriesIterator interface.

type SummaryData

type SummaryData struct {
	// contains filtered or unexported fields
}

SummaryData keeps all data needed to create summary metric

func (*SummaryData) AddSummary

func (s *SummaryData) AddSummary(sum *dto.Summary)

func (*SummaryData) Metric

func (s *SummaryData) Metric(desc *prometheus.Desc, labelValues ...string) prometheus.Metric

Directories

Path Synopsis
Package fakeauth provides middlewares thats injects a fake userID, so the rest of the code can continue to be multitenant.
Package fakeauth provides middlewares thats injects a fake userID, so the rest of the code can continue to be multitenant.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL