Documentation
¶
Index ¶
- type AggregateMetricsCollector
- type CapacityCollectionMetricsCollector
- type CapacityCollectionMetricsInstance
- type Collector
- func (c *Collector) ApplyQuotaOverridesJob(registerer prometheus.Registerer) jobloop.Job
- func (c *Collector) CapacityScrapeJob(registerer prometheus.Registerer) jobloop.Job
- func (c *Collector) CheckConsistencyJob(registerer prometheus.Registerer) jobloop.Job
- func (c *Collector) CleanupOldCommitmentsJob(registerer prometheus.Registerer) jobloop.Job
- func (c *Collector) ExpiringCommitmentNotificationJob(registerer prometheus.Registerer) jobloop.Job
- func (c *Collector) MailDeliveryJob(registerer prometheus.Registerer, client MailClient) jobloop.Job
- func (c *Collector) ScanDomains(ctx context.Context, opts ScanDomainsOpts) (result []string, resultErr error)
- func (c *Collector) ScanDomainsAndProjectsJob(registerer prometheus.Registerer) jobloop.Job
- func (c *Collector) ScanProjects(ctx context.Context, domain *db.Domain) (result []string, resultErr error)
- func (c *Collector) ScrapeJob(registerer prometheus.Registerer) jobloop.Job
- func (c *Collector) SyncQuotaToBackendJob(registerer prometheus.Registerer) jobloop.Job
- type DataMetricsReporter
- type MailClient
- type MailRequest
- type QuotaCollectionMetricsInstance
- type ScanDomainsOpts
- type TaskTiming
- type UndeliverableMailError
- type UsageCollectionMetricsCollector
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AggregateMetricsCollector ¶
AggregateMetricsCollector is a prometheus.Collector that submits dynamically-calculated aggregate metrics about scraping progress.
func (*AggregateMetricsCollector) Collect ¶
func (c *AggregateMetricsCollector) Collect(ch chan<- prometheus.Metric)
Collect implements the prometheus.Collector interface.
func (*AggregateMetricsCollector) Describe ¶
func (c *AggregateMetricsCollector) Describe(ch chan<- *prometheus.Desc)
Describe implements the prometheus.Collector interface.
type CapacityCollectionMetricsCollector ¶
type CapacityCollectionMetricsCollector struct { Cluster *core.Cluster DB *gorp.DbMap // When .Override is set, the DB is bypassed and only the given // CapacityCollectionMetricsInstances are considered. This is used for testing only. Override []CapacityCollectionMetricsInstance }
CapacityCollectionMetricsCollector is a prometheus.Collector that submits metrics
func (*CapacityCollectionMetricsCollector) Collect ¶
func (c *CapacityCollectionMetricsCollector) Collect(ch chan<- prometheus.Metric)
Collect implements the prometheus.Collector interface.
func (*CapacityCollectionMetricsCollector) Describe ¶
func (c *CapacityCollectionMetricsCollector) Describe(ch chan<- *prometheus.Desc)
Describe implements the prometheus.Collector interface.
type CapacityCollectionMetricsInstance ¶
type CapacityCollectionMetricsInstance struct { ServiceType db.ServiceType SerializedMetrics string }
CapacityCollectionMetricsInstance describes a single project service for which collection metrics are submitted. It appears in type CapacityCollectionMetricsCollector.
type Collector ¶
type Collector struct { Cluster *core.Cluster DB *gorp.DbMap // Usually logg.Error, but can be changed inside unit tests. LogError func(msg string, args ...any) // Usually time.Now, but can be changed inside unit tests. // MeasureTimeAtEnd behaves slightly differently in unit tests: It will advance // the mock.Clock before reading it to simulate time passing during the previous task. MeasureTime func() time.Time MeasureTimeAtEnd func() time.Time // Usually addJitter, but can be changed inside unit tests. AddJitter func(time.Duration) time.Duration }
Collector provides methods that implement the collection jobs performed by limes-collect. The struct contains a reference to the Cluster configuration, which holds references to the keystone DiscoveryPlugin and LiquidConnections (which define the service type to be targeted), and a few other things; basically everything that needs to be replaced by a mock implementation for the collector's unit tests.
func NewCollector ¶
NewCollector creates a Collector instance.
func (*Collector) ApplyQuotaOverridesJob ¶
func (c *Collector) ApplyQuotaOverridesJob(registerer prometheus.Registerer) jobloop.Job
ApplyQuotaOverridesJob is a jobloop.CronJob.
It loads quota overrides from the respective config file and updates the `project_resources.override_quota_from_config` column to match the configured values.
func (*Collector) CapacityScrapeJob ¶
func (c *Collector) CapacityScrapeJob(registerer prometheus.Registerer) jobloop.Job
CapacityScrapeJob is a jobloop.Job. Each task scrapes one Liquid, equal to one ClusterService entry. ClusterResources and ClusterAZResources are managed indirectly by this job, because a bump of the InfoVersion on Liquid side causes a reconciliation against the DB. Extraneous ClusterServices are only deleted on startup of the Collector, by Cluster.Connect.
func (*Collector) CheckConsistencyJob ¶
func (c *Collector) CheckConsistencyJob(registerer prometheus.Registerer) jobloop.Job
func (*Collector) CleanupOldCommitmentsJob ¶
func (c *Collector) CleanupOldCommitmentsJob(registerer prometheus.Registerer) jobloop.Job
CleanupOldCommitmentsJob is a jobloop.CronJob.
It moves expired commitments to state "expired" and cleans up old expired commitments that do not have any non-expired predecessors.
func (*Collector) ExpiringCommitmentNotificationJob ¶
func (c *Collector) ExpiringCommitmentNotificationJob(registerer prometheus.Registerer) jobloop.Job
ExpiringCommitmentNotificationJob is a jobloop.Job. A task scrapes commitments that are or are about to expire. For all applicable commitments within a project the mail content to inform customers will be prepared and added to a queue. Long-term commitments will be queued while short-term commitments will only be marked as notified.
func (*Collector) MailDeliveryJob ¶
func (c *Collector) MailDeliveryJob(registerer prometheus.Registerer, client MailClient) jobloop.Job
MailDeliveryJob is a jobloop.CronJob. A task searches for a queued mail notification. If any is found, it builds the mail request body and posts the mail to the mail API. Unsuccessful mail deliveries will increase the fail counter and will be requeued with an updated submission timestamp.
func (*Collector) ScanDomains ¶
func (c *Collector) ScanDomains(ctx context.Context, opts ScanDomainsOpts) (result []string, resultErr error)
ScanDomains queries Keystone to discover new domains, and returns a list of UUIDs for the newly discovered domains.
func (*Collector) ScanDomainsAndProjectsJob ¶
func (c *Collector) ScanDomainsAndProjectsJob(registerer prometheus.Registerer) jobloop.Job
ScanDomainsAndProjectsJob is a jobloop.CronJob. It syncs domains and projects from Keystone into the Limes database.
func (*Collector) ScanProjects ¶
func (c *Collector) ScanProjects(ctx context.Context, domain *db.Domain) (result []string, resultErr error)
ScanProjects queries Keystone to discover new projects in the given domain.
func (*Collector) ScrapeJob ¶
func (c *Collector) ScrapeJob(registerer prometheus.Registerer) jobloop.Job
ScrapeJob looks at one specific project service per task, collects quota and usage information from the backend service as well as checks the database for outdated or missing rate records for the given service. The backend quota is adjusted if it differs from the desired values and rate records are updated by querying the backend service.
This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.
func (*Collector) SyncQuotaToBackendJob ¶
func (c *Collector) SyncQuotaToBackendJob(registerer prometheus.Registerer) jobloop.Job
SyncQuotaToBackendJob looks for project services that need to have their quota applied to the backend, and runs SetQuota for those services.
This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.
type DataMetricsReporter ¶
DataMetricsReporter renders Prometheus metrics for data attributes (quota, usage, etc.) for all projects known to Limes.
It is an http.Handler, instead of implementing the prometheus.Collector interface (like all the other Collector types in this package) and going through the normal promhttp facility.
We are not going through promhttp here because promhttp insists on holding all metrics in memory before rendering them out (in order to sort them). Given the extremely high cardinality of these metrics, this results in unreasonably high memory usage spikes.
This implementation also holds all the metrics in memory (because ORDER BY on database level turned out to be prohibitively expensive), but we hold their rendered forms (i.e. something like `{bar="bar",foo="foo"} 42` instead of a dozen allocations for each label name, label value, label pair, a map of label pairs, and so on) in order to save memory.
This exporter cannot use Cluster.LiquidConnections, because it runs outside of the collect task. Therefore, it uses the convenience methods of the Cluster to get the necessary liquid.ResourceInfo data.
func (*DataMetricsReporter) ServeHTTP ¶
func (d *DataMetricsReporter) ServeHTTP(w http.ResponseWriter, r *http.Request)
ServeHTTP implements the http.Handler interface.
type MailClient ¶
type MailClient interface { // Builds the request to sent the mail content to a mail API. PostMail(ctx context.Context, req MailRequest) error }
MailClient is an interface that provides the methods to communicate with a mail backend service.
func NewMailClient ¶
func NewMailClient(provider *gophercloud.ProviderClient, endpoint string) (MailClient, error)
NewMailClient returns a service client to communicate with a mail API.
type MailRequest ¶
type QuotaCollectionMetricsInstance ¶
type QuotaCollectionMetricsInstance struct { Project core.KeystoneProject ServiceType db.ServiceType SerializedMetrics string }
QuotaCollectionMetricsInstance describes a single project service for which collection metrics are submitted. It appears in type UsageCollectionMetricsCollector.
type ScanDomainsOpts ¶
type ScanDomainsOpts struct { // Recurse into ScanProjects for all domains in the selected cluster, // rather than just for new domains. ScanAllProjects bool }
ScanDomainsOpts contains additional options for ScanDomains().
type TaskTiming ¶
type TaskTiming struct { StartedAt time.Time // filled during DiscoverTask FinishedAt time.Time // filled during ProcessTask }
TaskTiming appears in the task types of our ProducerConsumerJobs.
func (TaskTiming) Duration ¶
func (t TaskTiming) Duration() time.Duration
Duration measures the duration of the main portion of a task.
type UndeliverableMailError ¶
type UndeliverableMailError struct {
Inner error
}
UndeliverableMailError is a custom error type to define udeliverable mails. Used in the MailClient interface implementations.
func (UndeliverableMailError) Error ¶
func (e UndeliverableMailError) Error() string
implements https://pkg.go.dev/builtin#error
func (UndeliverableMailError) Unwrap ¶
func (e UndeliverableMailError) Unwrap() error
implements the interface implied by https://pkg.go.dev/errors
type UsageCollectionMetricsCollector ¶
type UsageCollectionMetricsCollector struct { Cluster *core.Cluster DB *gorp.DbMap // When .Override is set, the DB is bypassed and only the given // QuotaCollectionMetricsInstances are considered. This is used for testing only. Override []QuotaCollectionMetricsInstance }
UsageCollectionMetricsCollector is a prometheus.Collector that submits metrics
func (*UsageCollectionMetricsCollector) Collect ¶
func (c *UsageCollectionMetricsCollector) Collect(ch chan<- prometheus.Metric)
Collect implements the prometheus.Collector interface.
func (*UsageCollectionMetricsCollector) Describe ¶
func (c *UsageCollectionMetricsCollector) Describe(ch chan<- *prometheus.Desc)
Describe implements the prometheus.Collector interface.