Documentation
¶
Index ¶
- Constants
- Variables
- func NewAltTokenSource(tokenURL, tokenBody string) oauth2.TokenSource
- func Testing() bool
- func Version() (string, error)
- func WithMetadataFunc(ctx context.Context, mf MetadataFunc) context.Context
- type AltTokenSource
- type EfficiencyOpts
- type Exporter
- type ExporterOpts
- type Lease
- type Matchers
- type MetadataFunc
- type MetricMetadata
- type Storage
Constants ¶
const ( // DefaultShardCount represents number of shards by which series are bucketed. DefaultShardCount = 1024 // DefaultShardBufferSize represents the buffer size for each individual shard. // Each element in buffer (queue) consists of sample and hash. DefaultShardBufferSize = 2048 // BatchSizeMax represents maximum number of samples to pack into a batch sent to GCM. BatchSizeMax = 200 // Prefix for GCM metric. MetricTypePrefix = "prometheus.googleapis.com" )
const ( CompressionNone = "none" CompressionGZIP = "gzip" )
Supported gRPC compression formats.
const ( KeyProjectID = "project_id" KeyLocation = "location" KeyCluster = "cluster" KeyNamespace = "namespace" KeyJob = "job" KeyInstance = "instance" )
The target label keys used for the Prometheus monitored resource.
const (
// ClientName is used to identify the User Agent.
ClientName = "prometheus-engine-export"
)
Variables ¶
var (
ErrLocationGlobal = errors.New("Location must be set to a named Google Cloud " +
"region and cannot be set to \"global\". Please choose the " +
"Google Cloud region that is physically nearest to your cluster. " +
"See https://www.cloudinfrastructuremap.com/")
)
Functions ¶
func NewAltTokenSource ¶ added in v0.3.2
func NewAltTokenSource(tokenURL, tokenBody string) oauth2.TokenSource
NewAltTokenSource constructs a new alternate token source for generating tokens.
func Testing ¶ added in v0.7.1
func Testing() bool
Testing returns true if running within a unit test. TODO(TheSpiritXIII): Replace with https://github.com/golang/go/issues/52600
func Version ¶
Version is used in the User Agent. This version is automatically detected if this function is imported as a library. However, the version is statically set if this function is used in a binary in prometheus-engine due to Golang restrictions. While testing, the static version is validated for correctness.
func WithMetadataFunc ¶ added in v0.0.1
func WithMetadataFunc(ctx context.Context, mf MetadataFunc) context.Context
WithMetadataFunc stores mf in the context.
Types ¶
type AltTokenSource ¶ added in v0.3.2
type AltTokenSource struct {
// contains filtered or unexported fields
}
AltTokenSource is the structure holding the data for the functionality needed to generates tokens
type EfficiencyOpts ¶ added in v0.7.4
type EfficiencyOpts struct {
// BatchSize controls a maximum batch size to use when sending data to the GCM
// API. Defaults to BatchSizeMax when 0. The BatchSizeMax is also
// the maximum number this field can have due to GCM quota for write requests
// size. See https://cloud.google.com/monitoring/quotas?hl=en#custom_metrics_quotas.
BatchSize uint
// ShardCount controls number of shards. Refer to Exporter.Run documentation
// to learn more about algorithm. Defaults to DefaultShardCount when 0.
ShardCount uint
// ShardBufferSize controls the size for each individual shard. Each element
// in buffer (queue) consists of sample and hash. Refer to Exporter.Run
// documentation to learn more about algorithm. Defaults to
// DefaultShardBufferSize when 0.
ShardBufferSize uint
}
EfficiencyOpts represents exporter options that allows fine-tuning of internal data structure sizes. Only for advance users. No compatibility guarantee (might change in future).
type Exporter ¶
type Exporter struct {
// contains filtered or unexported fields
}
Exporter converts Prometheus samples into Cloud Monitoring samples and exports them.
func New ¶
func New(logger log.Logger, reg prometheus.Registerer, opts ExporterOpts) (*Exporter, error)
New returns a new Cloud Monitoring Exporter.
func NopExporter ¶ added in v0.2.2
func NopExporter() *Exporter
NopExporter returns an inactive exporter.
func (*Exporter) ApplyConfig ¶
ApplyConfig updates the exporter state to the given configuration. Must be called at least once before Export() can be used.
func (*Exporter) Export ¶
func (e *Exporter) Export(metadata MetadataFunc, batch []record.RefSample, exemplarMap map[storage.SeriesRef]record.RefExemplar)
Export enqueues the samples and exemplars to be written to Cloud Monitoring.
func (*Exporter) Run ¶
Run sends exported samples to Google Cloud Monitoring. Must be called at most once. ApplyConfig must be called once prior to calling Run.
Run starts a loop that gathers samples and sends them to GCM.
Samples must not arrive at the GCM API out of order. To ensure that, there must be at most one in-flight request per series. Tracking every series individually would also require separate queue per series. This would come with a lot of overhead and implementation complexity. Instead, we shard the series space and maintain one queue per shard. For every shard we ensure that there is at most one in-flight request.
One solution would be to have a separate send loop per shard that reads from the queue, accumulates a batch, and sends it to the GCM API. The drawback is that one has to get the number of shards right. Too low, and samples per shard cannot be sent fast enough. Too high, and batches do not fill up, potentially sending new requests for every sample. As a result, fine-tuning at startup but also runtime is necessary to respond to changing load patterns and latency of the API.
We largely avoid this issue by filling up batches from multiple shards. Under high load, a batch contains samples from fewer shards, under low load from more. The per-shard overhead is minimal and thus a high number can be picked, which allows us to cover a large range of potential throughput and latency combinations without requiring user configuration or, even worse, runtime changes to the shard number.
func (*Exporter) SetLabelsByIDFunc ¶
SetLabelsByIDFunc injects a function that can be used to retrieve a label set based on a series ID we got through exported sample records. Must be called before any call to Export is made.
type ExporterOpts ¶
type ExporterOpts struct {
// Whether to disable exporting of metrics.
Disable bool
// GCM API endpoint to send metric data to.
Endpoint string
// Compression format to use for gRPC requests.
Compression string
// Credentials file for authentication with the GCM API.
CredentialsFile string
// CredentialsFromJSON represents content of credentials file for
// authentication with the GCM API. CredentialsFile has priority over this.
CredentialsFromJSON []byte
// Disable authentication (for debugging purposes).
DisableAuth bool
// A user agent product string added to the regular user agent.
// See: https://www.rfc-editor.org/rfc/rfc7231#section-5.5.3
UserAgentProduct string
// A string added as a suffix to the regular user agent.
UserAgentMode string
// UserAgentEnv where calls to GCM API are made.
UserAgentEnv string
// Default monitored resource fields set on exported data.
ProjectID string
Location string
Cluster string
// A list of metric matchers. Only Prometheus time series satisfying at
// least one of the matchers are exported.
// This option matches the semantics of the Prometheus federation match[]
// parameter.
Matchers Matchers
// Prefix under which metrics are written to GCM.
MetricTypePrefix string
// A lease on a time range for which the exporter send sample data.
// It is checked for on each batch provided to the Export method.
// If unset, data is always sent.
Lease Lease
// Request URL and body for generating an alternative GCE token source.
// This allows metrics to be exported to an alternative project.
TokenURL string
TokenBody string
// The project ID of an alternative project for quota attribution.
QuotaProject string
// Efficiency represents exporter options that allows fine-tuning of
// internal data structure sizes. Only for advance users. No compatibility
// guarantee (might change in future).
Efficiency EfficiencyOpts
}
ExporterOpts holds options for an exporter.
type Lease ¶ added in v0.2.2
type Lease interface {
// Range informs whether the caller currently holds the lease and for what time range.
// The range is inclusive.
Range() (start, end time.Time, ok bool)
// Run background processing until context is cancelled.
Run(context.Context)
// OnLeaderChange sets a callback that is invoked when the lease leader changes.
// Must be called before Run.
OnLeaderChange(func())
}
Lease determines a currently owned time range.
type Matchers ¶ added in v0.0.4
Matchers holds a list of metric selectors that can be set as a flag.
func (*Matchers) IsCumulative ¶ added in v0.0.4
type MetadataFunc ¶ added in v0.0.1
type MetadataFunc func(metric string) (MetricMetadata, bool)
MetadataFunc gets metadata for a specific metric name.
func MetadataFuncFromContext ¶ added in v0.0.1
func MetadataFuncFromContext(ctx context.Context) (MetadataFunc, bool)
MetadataFuncFromContext extracts a MetataFunc from ctx.
type MetricMetadata ¶ added in v0.0.1
type MetricMetadata struct {
Metric string
Type textparse.MetricType
Help string
Unit string
}
MetricMetadata is a copy of MetricMetadata in Prometheus's scrape package. It is copied to break a dependency cycle.
type Storage ¶
type Storage struct {
// contains filtered or unexported fields
}
Storage provides a stateful wrapper around an Exporter that implements Prometheus's storage interface (Appendable).
For performance reasons Exporter is optimized to be tightly integrate with Prometheus's storage. This makes it rely on external state (series ID to label mapping). For use cases where a full Prometheus storage engine is not present (e.g. rule evaluation service), Storage acts as a simple drop-in replacement that directly manages the state required by Exporter.
func NewStorage ¶
NewStorage returns a new Prometheus storage that's exporting data via the exporter.
func (*Storage) ApplyConfig ¶ added in v0.1.0
ApplyConfig applies the new configuration to the storage.
Source Files
Directories