collector

package
v0.0.0-...-acbf869 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 27, 2025 License: Apache-2.0 Imports: 30 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AggregateMetricsCollector 

type AggregateMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
}

AggregateMetricsCollector is a prometheus.Collector that submits dynamically-calculated aggregate metrics about scraping progress.

func (*AggregateMetricsCollector) Collect 

func (c *AggregateMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*AggregateMetricsCollector) Describe 

func (c *AggregateMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type CapacityCollectionMetricsCollector 

type CapacityCollectionMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// When .Override is set, the DB is bypassed and only the given
	// CapacityCollectionMetricsInstances are considered. This is used for testing only.
	Override []CapacityCollectionMetricsInstance
}

CapacityCollectionMetricsCollector is a prometheus.Collector that submits metrics

func (*CapacityCollectionMetricsCollector) Collect 

func (c *CapacityCollectionMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*CapacityCollectionMetricsCollector) Describe 

func (c *CapacityCollectionMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type CapacityCollectionMetricsInstance 

type CapacityCollectionMetricsInstance struct {
	ServiceType       db.ServiceType
	SerializedMetrics string
}

CapacityCollectionMetricsInstance describes a single project service for which collection metrics are submitted. It appears in type CapacityCollectionMetricsCollector.

type Collector 

type Collector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// Usually logg.Error, but can be changed inside unit tests.
	LogError func(msg string, args ...any)
	// Usually time.Now, but can be changed inside unit tests.
	// MeasureTimeAtEnd behaves slightly differently in unit tests: It will advance
	// the mock.Clock before reading it to simulate time passing during the previous task.
	MeasureTime      func() time.Time
	MeasureTimeAtEnd func() time.Time
	// Usually addJitter, but can be changed inside unit tests.
	AddJitter func(time.Duration) time.Duration
}

Collector provides methods that implement the collection jobs performed by limes-collect. The struct contains a reference to the Cluster configuration, which holds references to the keystone DiscoveryPlugin and LiquidConnections (which define the service type to be targeted), and a few other things; basically everything that needs to be replaced by a mock implementation for the collector's unit tests.

func NewCollector 

func NewCollector(cluster *core.Cluster) *Collector

NewCollector creates a Collector instance.

func (*Collector) ApplyQuotaOverridesJob 

func (c *Collector) ApplyQuotaOverridesJob(registerer prometheus.Registerer) jobloop.Job

ApplyQuotaOverridesJob is a jobloop.CronJob.

It loads quota overrides from the respective config file and updates the `project_resources.override_quota_from_config` column to match the configured values.

func (*Collector) CapacityScrapeJob 

func (c *Collector) CapacityScrapeJob(registerer prometheus.Registerer) jobloop.Job

CapacityScrapeJob is a jobloop.Job. Each task scrapes one Liquid, equal to one ClusterService entry. ClusterResources and ClusterAZResources are managed indirectly by this job, because a bump of the InfoVersion on Liquid side causes a reconciliation against the DB. Extraneous ClusterServices are only deleted on startup of the Collector, by Cluster.Connect.

func (*Collector) CheckConsistencyJob 

func (c *Collector) CheckConsistencyJob(registerer prometheus.Registerer) jobloop.Job

func (*Collector) CleanupOldCommitmentsJob 

func (c *Collector) CleanupOldCommitmentsJob(registerer prometheus.Registerer) jobloop.Job

CleanupOldCommitmentsJob is a jobloop.CronJob.

It moves expired commitments to state "expired" and cleans up old expired commitments that do not have any non-expired predecessors.

func (*Collector) ExpiringCommitmentNotificationJob 

func (c *Collector) ExpiringCommitmentNotificationJob(registerer prometheus.Registerer) jobloop.Job

ExpiringCommitmentNotificationJob is a jobloop.Job. A task scrapes commitments that are or are about to expire. For all applicable commitments within a project the mail content to inform customers will be prepared and added to a queue. Long-term commitments will be queued while short-term commitments will only be marked as notified.

func (*Collector) MailDeliveryJob 

func (c *Collector) MailDeliveryJob(registerer prometheus.Registerer, client MailClient) jobloop.Job

MailDeliveryJob is a jobloop.CronJob. A task searches for a queued mail notification. If any is found, it builds the mail request body and posts the mail to the mail API. Unsuccessful mail deliveries will increase the fail counter and will be requeued with an updated submission timestamp.

func (*Collector) ScanDomains 

func (c *Collector) ScanDomains(ctx context.Context, opts ScanDomainsOpts) (result []string, resultErr error)

ScanDomains queries Keystone to discover new domains, and returns a list of UUIDs for the newly discovered domains.

func (*Collector) ScanDomainsAndProjectsJob 

func (c *Collector) ScanDomainsAndProjectsJob(registerer prometheus.Registerer) jobloop.Job

ScanDomainsAndProjectsJob is a jobloop.CronJob. It syncs domains and projects from Keystone into the Limes database.

func (*Collector) ScanProjects 

func (c *Collector) ScanProjects(ctx context.Context, domain *db.Domain) (result []string, resultErr error)

ScanProjects queries Keystone to discover new projects in the given domain.

func (*Collector) ScrapeJob 

func (c *Collector) ScrapeJob(registerer prometheus.Registerer) jobloop.Job

ScrapeJob looks at one specific project service per task, collects quota and usage information from the backend service as well as checks the database for outdated or missing rate records for the given service. The backend quota is adjusted if it differs from the desired values and rate records are updated by querying the backend service.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

func (*Collector) SyncQuotaToBackendJob 

func (c *Collector) SyncQuotaToBackendJob(registerer prometheus.Registerer) jobloop.Job

SyncQuotaToBackendJob looks for project services that need to have their quota applied to the backend, and runs SetQuota for those services.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

type DataMetricsReporter 

type DataMetricsReporter struct {
	Cluster      *core.Cluster
	DB           *gorp.DbMap
	ReportZeroes bool
}

DataMetricsReporter renders Prometheus metrics for data attributes (quota, usage, etc.) for all projects known to Limes.

It is an http.Handler, instead of implementing the prometheus.Collector interface (like all the other Collector types in this package) and going through the normal promhttp facility.

We are not going through promhttp here because promhttp insists on holding all metrics in memory before rendering them out (in order to sort them). Given the extremely high cardinality of these metrics, this results in unreasonably high memory usage spikes.

This implementation also holds all the metrics in memory (because ORDER BY on database level turned out to be prohibitively expensive), but we hold their rendered forms (i.e. something like `{bar="bar",foo="foo"} 42` instead of a dozen allocations for each label name, label value, label pair, a map of label pairs, and so on) in order to save memory.

This exporter cannot use Cluster.LiquidConnections, because it runs outside of the collect task. Therefore, it uses the convenience methods of the Cluster to get the necessary liquid.ResourceInfo data.

func (*DataMetricsReporter) ServeHTTP 

func (d *DataMetricsReporter) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface.

type MailClient 

type MailClient interface {
	// Builds the request to sent the mail content to a mail API.
	PostMail(ctx context.Context, req MailRequest) error
}

MailClient is an interface that provides the methods to communicate with a mail backend service.

func NewMailClient 

func NewMailClient(provider *gophercloud.ProviderClient, endpoint string) (MailClient, error)

NewMailClient returns a service client to communicate with a mail API.

type MailRequest 

type MailRequest struct {
	ProjectID string `json:"project_id"`
	Subject   string `json:"subject"`
	MimeType  string `json:"mime_type"`
	MailText  string `json:"mail_text"`
}

type QuotaCollectionMetricsInstance 

type QuotaCollectionMetricsInstance struct {
	Project           core.KeystoneProject
	ServiceType       db.ServiceType
	SerializedMetrics string
}

QuotaCollectionMetricsInstance describes a single project service for which collection metrics are submitted. It appears in type UsageCollectionMetricsCollector.

type ScanDomainsOpts 

type ScanDomainsOpts struct {
	// Recurse into ScanProjects for all domains in the selected cluster,
	// rather than just for new domains.
	ScanAllProjects bool
}

ScanDomainsOpts contains additional options for ScanDomains().

type TaskTiming 

type TaskTiming struct {
	StartedAt  time.Time // filled during DiscoverTask
	FinishedAt time.Time // filled during ProcessTask
}

TaskTiming appears in the task types of our ProducerConsumerJobs.

func (TaskTiming) Duration 

func (t TaskTiming) Duration() time.Duration

Duration measures the duration of the main portion of a task.

type UndeliverableMailError 

type UndeliverableMailError struct {
	Inner error
}

UndeliverableMailError is a custom error type to define udeliverable mails. Used in the MailClient interface implementations.

func (UndeliverableMailError) Error 

func (e UndeliverableMailError) Error() string

implements https://pkg.go.dev/builtin#error

func (UndeliverableMailError) Unwrap 

func (e UndeliverableMailError) Unwrap() error

implements the interface implied by https://pkg.go.dev/errors

type UsageCollectionMetricsCollector 

type UsageCollectionMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// When .Override is set, the DB is bypassed and only the given
	// QuotaCollectionMetricsInstances are considered. This is used for testing only.
	Override []QuotaCollectionMetricsInstance
}

UsageCollectionMetricsCollector is a prometheus.Collector that submits metrics

func (*UsageCollectionMetricsCollector) Collect 

func (c *UsageCollectionMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*UsageCollectionMetricsCollector) Describe 

func (c *UsageCollectionMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.