controller

package
v0.1.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 8, 2025 License: Apache-2.0 Imports: 39 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewCollectorCoverage 

func NewCollectorCoverage() prometheus.Collector

NewCollectorCoverage returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_coverage" which represents the test coverage percentage of the most recent GitLab CI pipeline.

This metric uses a set of default labels (such as project, ref, kind, etc.) to provide detailed context about the pipeline.

It helps monitor code quality via pipeline coverage over time.

func NewCollectorDurationSeconds 

func NewCollectorDurationSeconds() prometheus.Collector

NewCollectorDurationSeconds returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_duration_seconds". This metric measures the duration (in seconds) of the most recent GitLab CI pipeline run.

The metric includes default labels to provide contextual information about the pipeline, such as project, ref, kind, etc.

func NewCollectorEnvironmentBehindCommitsCount 

func NewCollectorEnvironmentBehindCommitsCount() prometheus.Collector

NewCollectorEnvironmentBehindCommitsCount returns a new Prometheus gauge collector for the metric "gitlab_ci_environment_behind_commits_count". This metric measures the number of commits that the environment is behind based on its last deployment.

This metric uses environment-specific labels (such as project and environment) to properly attribute the measurement.

func NewCollectorEnvironmentBehindDurationSeconds 

func NewCollectorEnvironmentBehindDurationSeconds() prometheus.Collector

NewCollectorEnvironmentBehindDurationSeconds returns a new Prometheus gauge collector for the metric "gitlab_ci_environment_behind_duration_seconds". This metric measures the duration in seconds that an environment is behind the most recent commit, based on its last deployment.

The collector uses environment-related labels such as project and environment to provide context.

func NewCollectorEnvironmentDeploymentCount 

func NewCollectorEnvironmentDeploymentCount() prometheus.Collector

NewCollectorEnvironmentDeploymentCount returns a new Prometheus counter collector for the metric "gitlab_ci_environment_deployment_count". This metric counts the total number of deployments for a given environment.

It uses environment labels to associate deployment counts with specific projects/environments.

func NewCollectorEnvironmentDeploymentDurationSeconds 

func NewCollectorEnvironmentDeploymentDurationSeconds() prometheus.Collector

NewCollectorEnvironmentDeploymentDurationSeconds returns a new Prometheus gauge collector for the metric "gitlab_ci_environment_deployment_duration_seconds". This metric tracks the duration (in seconds) of the most recent deployment of a specific environment.

This collector also uses environment labels to provide detailed context.

func NewCollectorEnvironmentDeploymentJobID 

func NewCollectorEnvironmentDeploymentJobID() prometheus.Collector

NewCollectorEnvironmentDeploymentJobID returns a new Prometheus gauge collector for the metric "gitlab_ci_environment_deployment_job_id". This metric stores the ID of the most recent deployment job associated with an environment.

It uses environment labels to contextualize the job ID per environment.

func NewCollectorEnvironmentDeploymentStatus 

func NewCollectorEnvironmentDeploymentStatus() prometheus.Collector

NewCollectorEnvironmentDeploymentStatus returns a new Prometheus gauge collector for the metric "gitlab_ci_environment_deployment_status". This metric represents the status of the most recent deployment for a given environment.

The collector adds an extra label "status" to the standard environment labels, enabling tracking the deployment status alongside environment details.

func NewCollectorEnvironmentDeploymentTimestamp 

func NewCollectorEnvironmentDeploymentTimestamp() prometheus.Collector

NewCollectorEnvironmentDeploymentTimestamp returns a new Prometheus gauge collector for the metric "gitlab_ci_environment_deployment_timestamp". This metric stores the creation timestamp of the most recent deployment of the environment.

It uses environment labels to provide the context of when each environment was last deployed.

func NewCollectorEnvironmentInformation 

func NewCollectorEnvironmentInformation() prometheus.Collector

NewCollectorEnvironmentInformation returns a new Prometheus gauge collector for the metric "gitlab_ci_environment_information". This metric provides detailed static information about the environment.

The collector uses a combination of basic environment labels and additional environment information labels to give context such as environment ID, external URL, kind, ref, commit IDs, availability, and username.

func NewCollectorID 

func NewCollectorID() prometheus.Collector

NewCollectorID returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_id". This metric tracks the ID of the most recent GitLab pipeline.

It uses default labels to provide contextual metadata such as project, topics, kind, ref, source, and variables.

func NewCollectorJobArtifactSizeBytes 

func NewCollectorJobArtifactSizeBytes() prometheus.Collector

NewCollectorJobArtifactSizeBytes returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_job_artifact_size_bytes". This metric measures the total artifact size in bytes for the most recent job.

It uses both the default labels and job-specific labels to provide detailed context such as stage, job name, runner description, tags, and failure reasons.

func NewCollectorJobDurationSeconds 

func NewCollectorJobDurationSeconds() prometheus.Collector

NewCollectorJobDurationSeconds returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_job_duration_seconds". This metric tracks the duration in seconds of the most recent GitLab CI job.

The collector uses a combination of default labels (project, ref, etc.) and job-specific labels (stage, job name, runner, etc.) to provide detailed context for each job's duration.

func NewCollectorJobID 

func NewCollectorJobID() prometheus.Collector

NewCollectorJobID returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_job_id". This metric tracks the ID of the most recent job executed in GitLab CI pipelines.

It uses both the default and job-specific labels to provide detailed metadata for each job, helping uniquely identify jobs within projects and stages.

func NewCollectorJobQueuedDurationSeconds 

func NewCollectorJobQueuedDurationSeconds() prometheus.Collector

NewCollectorJobQueuedDurationSeconds returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_job_queued_duration_seconds". This metric measures the time in seconds that the most recent job was queued before it started running.

This collector also uses default and job-specific labels to add context on project, job stage, runner, etc., allowing fine-grained monitoring of job queue delays.

func NewCollectorJobRunCount 

func NewCollectorJobRunCount() prometheus.Collector

NewCollectorJobRunCount returns a new Prometheus counter collector for the metric "gitlab_ci_pipeline_job_run_count". This metric counts the total number of times a specific job has been executed.

It uses a combination of default labels (e.g., project, ref) and job-specific labels (e.g., stage, job_name, runner_description) to provide detailed context for counting job runs.

func NewCollectorJobStatus 

func NewCollectorJobStatus() prometheus.Collector

NewCollectorJobStatus returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_job_status". This metric reports the status of the most recent job execution.

It uses default labels combined with job-specific labels and also includes status labels (such as success, failed, running) to reflect the current state of each job.

func NewCollectorJobTimestamp 

func NewCollectorJobTimestamp() prometheus.Collector

NewCollectorJobTimestamp returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_job_timestamp". This metric records the creation timestamp of the most recent job.

Like others, it uses both default and job-specific labels to provide detailed metadata for each job's timestamp.

func NewCollectorQueuedDurationSeconds 

func NewCollectorQueuedDurationSeconds() prometheus.Collector

NewCollectorQueuedDurationSeconds returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_queued_duration_seconds". This metric tracks how long (in seconds) the most recent pipeline was queued before starting execution.

It uses the same default labels as other pipeline metrics for consistent context.

func NewCollectorRunCount 

func NewCollectorRunCount() prometheus.Collector

NewCollectorRunCount returns a new Prometheus counter collector for the metric "gitlab_ci_pipeline_run_count". This metric counts how many times a particular pipeline has been executed.

It uses the default labels to differentiate counts per pipeline.

func NewCollectorRunners added in v0.1.6 

func NewCollectorRunners() prometheus.Collector

NewCollectorRunners returns a new Prometheus gauge collector for the metric "gitlab_ci_runners". This metric reports information about your runners.

The labels include the default set of labels plus an additional "runnerLabels" label and "runnnerInformationLabels" that describes the runners state.

func NewCollectorStatus 

func NewCollectorStatus() prometheus.Collector

NewCollectorStatus returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_status". This metric reports the status of the most recent pipeline.

The labels include the default set of labels plus an additional "status" label that describes the pipeline status (e.g., success, failed, running).

func NewCollectorTestCaseExecutionTime 

func NewCollectorTestCaseExecutionTime() prometheus.Collector

NewCollectorTestCaseExecutionTime returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_case_execution_time". This metric measures the execution duration in seconds for each individual test case.

The collector is labeled with defaultLabels, testSuiteLabels, and testCaseLabels to provide full context about the pipeline, test suite, and specific test case.

func NewCollectorTestCaseStatus 

func NewCollectorTestCaseStatus() prometheus.Collector

NewCollectorTestCaseStatus returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_case_status". This metric tracks the status of each test case in the most recent job.

Labels include defaultLabels, testSuiteLabels, testCaseLabels, and statusLabels, to provide detailed information about pipeline, suite, test case, and test result status.

func NewCollectorTestReportErrorCount 

func NewCollectorTestReportErrorCount() prometheus.Collector

NewCollectorTestReportErrorCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_report_error_count". This metric tracks the number of tests that errored (i.e., had runtime errors) in the most recently finished pipeline.

It uses the defaultLabels to provide detailed context about the pipeline.

func NewCollectorTestReportFailedCount 

func NewCollectorTestReportFailedCount() prometheus.Collector

NewCollectorTestReportFailedCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_report_failed_count". This metric counts the number of failed tests in the most recently finished pipeline.

It uses the defaultLabels to associate the metric with relevant pipeline context, such as project, ref, and job details.

func NewCollectorTestReportSkippedCount 

func NewCollectorTestReportSkippedCount() prometheus.Collector

NewCollectorTestReportSkippedCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_report_skipped_count". This metric records the number of tests that were skipped in the most recently finished pipeline.

The defaultLabels provide pipeline metadata for labeling this metric.

func NewCollectorTestReportSuccessCount 

func NewCollectorTestReportSuccessCount() prometheus.Collector

NewCollectorTestReportSuccessCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_report_success_count". This metric tracks the number of successful tests in the most recently finished pipeline.

The defaultLabels provide pipeline-related metadata to label the metric.

func NewCollectorTestReportTotalCount 

func NewCollectorTestReportTotalCount() prometheus.Collector

NewCollectorTestReportTotalCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_report_total_count". This metric records the total number of tests run in the most recently finished pipeline.

It also uses the defaultLabels to provide detailed context for the metric.

func NewCollectorTestReportTotalTime 

func NewCollectorTestReportTotalTime() prometheus.Collector

NewCollectorTestReportTotalTime returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_report_total_time". This metric tracks the total duration, in seconds, of all tests executed in the most recently finished pipeline.

It uses the defaultLabels to associate the metric with pipeline-specific metadata, such as project, ref, and job information.

func NewCollectorTestSuiteErrorCount 

func NewCollectorTestSuiteErrorCount() prometheus.Collector

NewCollectorTestSuiteErrorCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_suite_error_count". This metric tracks the number of errored tests within a test suite.

The collector uses defaultLabels and testSuiteLabels to provide necessary context.

func NewCollectorTestSuiteFailedCount 

func NewCollectorTestSuiteFailedCount() prometheus.Collector

NewCollectorTestSuiteFailedCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_suite_failed_count". This metric tracks the number of failed tests within a test suite.

The collector is labeled with defaultLabels and testSuiteLabels to provide pipeline and test suite context.

func NewCollectorTestSuiteSkippedCount 

func NewCollectorTestSuiteSkippedCount() prometheus.Collector

NewCollectorTestSuiteSkippedCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_suite_skipped_count". This metric tracks the number of skipped tests within a test suite.

Labels include defaultLabels and testSuiteLabels to capture pipeline and test suite details.

func NewCollectorTestSuiteSuccessCount 

func NewCollectorTestSuiteSuccessCount() prometheus.Collector

NewCollectorTestSuiteSuccessCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_suite_success_count". This metric counts the number of successful tests within the test suite.

Labels include defaultLabels and testSuiteLabels for pipeline and test suite context.

func NewCollectorTestSuiteTotalCount 

func NewCollectorTestSuiteTotalCount() prometheus.Collector

NewCollectorTestSuiteTotalCount returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_suite_total_count". This metric tracks the total number of tests within a test suite.

The metric is labeled using defaultLabels and testSuiteLabels to capture pipeline and test suite information.

func NewCollectorTestSuiteTotalTime 

func NewCollectorTestSuiteTotalTime() prometheus.Collector

NewCollectorTestSuiteTotalTime returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_test_suite_total_time". This metric records the total duration in seconds of the entire test suite execution.

It uses defaultLabels combined with testSuiteLabels to provide detailed context about the pipeline and specific test suite.

func NewCollectorTimestamp 

func NewCollectorTimestamp() prometheus.Collector

NewCollectorTimestamp returns a new Prometheus gauge collector for the metric "gitlab_ci_pipeline_timestamp". This metric stores the timestamp of the last update (e.g., last run or modification) of the most recent pipeline.

It uses the default labels to provide context on which pipeline this timestamp belongs to.

func NewInternalCollectorCurrentlyQueuedTasksCount 

func NewInternalCollectorCurrentlyQueuedTasksCount() prometheus.Collector

NewInternalCollectorCurrentlyQueuedTasksCount creates and returns a new Prometheus GaugeVec metric collector for tracking the number of currently queued tasks in the system.

This metric has no labels (empty label slice), representing a simple gauge value. It can be used internally to monitor the task queue size.

func NewInternalCollectorEnvironmentsCount 

func NewInternalCollectorEnvironmentsCount() prometheus.Collector

NewInternalCollectorEnvironmentsCount returns a new Prometheus gauge collector for the metric "gcpe_environments_count" which tracks the total number of GitLab environments currently being exported by the application.

This metric has no labels, representing a simple numeric gauge.

func NewInternalCollectorExecutedTasksCount 

func NewInternalCollectorExecutedTasksCount() prometheus.Collector

NewInternalCollectorExecutedTasksCount returns a new Prometheus gauge collector for the metric "gcpe_executed_tasks_count" which tracks the total number of tasks that have been executed by the system.

This metric is a gauge without labels.

func NewInternalCollectorGitLabAPIRequestsCount 

func NewInternalCollectorGitLabAPIRequestsCount() prometheus.Collector

NewInternalCollectorGitLabAPIRequestsCount returns a new Prometheus gauge collector for the metric "gcpe_gitlab_api_requests_count" which monitors the total number of GitLab API requests made by the application.

This metric is useful to observe API usage and detect potential throttling issues.

This gauge has no labels.

func NewInternalCollectorGitLabAPIRequestsLimit 

func NewInternalCollectorGitLabAPIRequestsLimit() prometheus.Collector

NewInternalCollectorGitLabAPIRequestsLimit returns a new Prometheus gauge collector for the metric "gcpe_gitlab_api_requests_limit" which tracks the total number of GitLab API requests allowed per rate limit window.

This metric provides context to the remaining requests metric and helps understand the API quota.

The metric has no labels.

func NewInternalCollectorGitLabAPIRequestsRemaining 

func NewInternalCollectorGitLabAPIRequestsRemaining() prometheus.Collector

NewInternalCollectorGitLabAPIRequestsRemaining returns a new Prometheus gauge collector for the metric "gcpe_gitlab_api_requests_remaining" which tracks the number of remaining GitLab API requests allowed within the current rate limit window.

This metric helps monitor how close the application is to hitting the GitLab API rate limit.

The metric has no labels.

func NewInternalCollectorMetricsCount 

func NewInternalCollectorMetricsCount() prometheus.Collector

NewInternalCollectorMetricsCount returns a new Prometheus gauge collector for the metric "gcpe_metrics_count" which counts the number of GitLab pipeline metrics currently being exported by the application.

This metric can be used to monitor the volume of pipeline metrics managed and exposed.

The metric has no labels.

func NewInternalCollectorProjectsCount 

func NewInternalCollectorProjectsCount() prometheus.Collector

NewInternalCollectorProjectsCount returns a new Prometheus gauge collector for the metric "gcpe_projects_count" which tracks the number of GitLab projects currently being exported by the application.

This metric helps monitor how many projects the exporter is handling.

The metric has no labels.

func NewInternalCollectorRefsCount 

func NewInternalCollectorRefsCount() prometheus.Collector

NewInternalCollectorRefsCount returns a new Prometheus gauge collector for the metric "gcpe_refs_count" which tracks the number of GitLab refs (branches, tags, etc.) currently being exported.

This metric is useful to understand the scope of ref data being monitored.

The metric has no labels.

func NewInternalCollectorRunnersCount added in v0.1.6 

func NewInternalCollectorRunnersCount() prometheus.Collector

NewInternalCollectorRunnersCount returns a new Prometheus gauge collector for the metric "gcpe_runners_count" which tracks the total number of GitLab runners currently being exported by the application.

This metric has no labels, representing a simple numeric gauge.

Types

type Controller 

type Controller struct {
	Config         config.Config  // Application configuration settings
	Redis          *redis.Client  // Redis client for caching and coordination
	Gitlab         *gitlab.Client // GitLab API client
	Store          store.Store    // Storage interface to persist data (backed by Redis)
	TaskController TaskController // Manages background tasks and job queues

	// UUID uniquely identifies this controller instance among others when running
	// in clustered mode, facilitating coordination via Redis.
	UUID uuid.UUID
}

Controller holds the necessary clients and components to run the application and handle its operations. It includes configuration, connections to Redis, GitLab client, storage interface, and task management. The UUID field uniquely identifies this controller instance, especially useful in clustered deployments where multiple exporter instances share Redis.

func New 

func New(ctx context.Context, cfg config.Config, version string) (c Controller, err error)

New creates and initializes a new Controller instance. It sets up tracing, Redis connection, task controller, storage, GitLab client, and starts the scheduler.

Parameters: - ctx: Context for cancellation and deadlines. - cfg: Configuration object with all needed parameters. - version: Version string of the running application (used in GitLab client configuration).

Returns: - c: Initialized Controller instance. - err: Any error encountered during setup.

func (*Controller) GarbageCollectEnvironments 

func (c *Controller) GarbageCollectEnvironments(ctx context.Context) error

GarbageCollectEnvironments removes environments from the store that are no longer valid or configured.

func (*Controller) GarbageCollectMetrics 

func (c *Controller) GarbageCollectMetrics(ctx context.Context) error

GarbageCollectMetrics performs cleanup of obsolete or invalid metrics in the store. It ensures metrics linked to non-existent projects, refs, or environments are removed, and also respects project-level settings that may disable certain metrics.

The function fetches all stored environments, refs, and metrics, then iterates over each metric: - It first verifies that required labels ("project" plus either "ref" or "environment") exist; otherwise, it deletes the metric. - If the metric is related to a Ref (has "ref" label), it checks:

  • Whether the ref still exists; if not, deletes the metric.
  • If job-related metrics are disabled for the ref's project, deletes the metric.
  • If sparse output mode for status metrics is enabled on the ref's project and the metric value isn't 1, deletes the metric.

- If the metric is related to an Environment (has "environment" label), it checks:

  • Whether the environment still exists; if not, deletes the metric.
  • If sparse output mode for environment deployment status metrics is enabled and the metric value isn't 1, deletes the metric.

This keeps the metrics store consistent, up-to-date, and free from stale data, according to project configurations.

func (*Controller) GarbageCollectProjects 

func (c *Controller) GarbageCollectProjects(ctx context.Context) error

GarbageCollectProjects removes projects from the store that are no longer configured or discovered.

func (*Controller) GarbageCollectRefs 

func (c *Controller) GarbageCollectRefs(ctx context.Context) error

GarbageCollectRefs cleans up stored refs that are no longer valid or configured.

func (*Controller) GarbageCollectRunners added in v0.1.6 

func (c *Controller) GarbageCollectRunners(ctx context.Context) error

GarbageCollectRunners cleans up stored runners that are no longer valid or configured.

func (*Controller) GetGitLabMetadata 

func (c *Controller) GetGitLabMetadata(ctx context.Context) error

GetGitLabMetadata retrieves metadata from the GitLab API and updates the local GitLab version if available.

func (*Controller) GetRefs 

func (c *Controller) GetRefs(ctx context.Context, p schemas.Project) (refs schemas.Refs, err error)

GetRefs retrieves the references (branches, tags, merge requests) for a given project `p`, based on the project's pull configuration settings.

The method returns a map of references keyed by their identifiers and any error encountered during retrieval.

Behavior details: - Initializes an empty collection of refs to accumulate results.

- If branch pulling is enabled:

  • If any of the following branch filters are active:
  • Include deleted branches (ExcludeDeleted is false),
  • Limit to most recent branches (MostRecent > 0),
  • Limit by age of branches (MaxAgeSeconds > 0), then branches are fetched via the pipelines API (GetRefsFromPipelines) for branches.
  • Otherwise, branches are fetched using the direct project branches API (GetProjectBranches).
  • Merges the pulled branches into the refs map.

- If tag pulling is enabled:

  • Similar to branches, if any tag filters are active (ExcludeDeleted, MostRecent, MaxAgeSeconds), tags are fetched from pipelines API (GetRefsFromPipelines) for tags,
  • Otherwise, tags are fetched via the project tags API (GetProjectTags).
  • Merges the pulled tags into the refs map.

- If merge requests pulling is enabled:

  • Merge requests are always fetched from the pipelines API (GetRefsFromPipelines).
  • Merges the pulled merge requests into the refs map.

- Returns the combined refs map and any error encountered.

This method ensures that refs are fetched efficiently according to filtering options, using the pipelines API when advanced filtering is required, otherwise falling back to simpler APIs

func (*Controller) HealthCheckHandler 

func (c *Controller) HealthCheckHandler(ctx context.Context) (h healthcheck.Handler)

HealthCheckHandler creates and returns a health check handler for the controller.

func (*Controller) MetricsHandler 

func (c *Controller) MetricsHandler(w http.ResponseWriter, r *http.Request)

MetricsHandler serves the /metrics HTTP endpoint to expose Prometheus metrics.

func (*Controller) ProcessJobMetrics 

func (c *Controller) ProcessJobMetrics(ctx context.Context, ref schemas.Ref, job schemas.Job)

ProcessJobMetrics processes metrics for a given pipeline job and updates the store accordingly.

func (*Controller) ProcessPipelinesMetrics added in v0.1.4 

func (c *Controller) ProcessPipelinesMetrics(ctx context.Context, ref schemas.Ref, apiPipeline *goGitlab.PipelineInfo) error

func (*Controller) ProcessRunnerMetrics added in v0.1.6 

func (c *Controller) ProcessRunnerMetrics(ctx context.Context, runner schemas.Runner) (err error)

ProcessRunnerMetrics processes metrics for a given runner and updates the store accordingly.

func (*Controller) ProcessTestCaseMetrics 

func (c *Controller) ProcessTestCaseMetrics(ctx context.Context, ref schemas.Ref, ts schemas.TestSuite, tc schemas.TestCase)

ProcessTestCaseMetrics processes and stores metrics for a single test case within a given test suite and GitLab ref (branch, tag, or merge request). It updates the metrics store with execution time and the test case status.

func (*Controller) ProcessTestReportMetrics 

func (c *Controller) ProcessTestReportMetrics(ctx context.Context, ref schemas.Ref, tr schemas.TestReport)

ProcessTestReportMetrics ..

func (*Controller) ProcessTestSuiteMetrics 

func (c *Controller) ProcessTestSuiteMetrics(ctx context.Context, ref schemas.Ref, ts schemas.TestSuite)

ProcessTestSuiteMetrics ..

func (*Controller) PullEnvironmentMetrics 

func (c *Controller) PullEnvironmentMetrics(ctx context.Context, env schemas.Environment) (err error)

PullEnvironmentMetrics retrieves and updates metrics related to a specific environment. It ensures the environment is up to date before collecting metrics, compares commits to calculate how far behind the environment is, updates deployment counts and durations, and sets various metrics in the store.

The function workflow is as follows: 1. Refresh the environment from the store to avoid working on stale data. 2. Store the current deployment JobID for comparison later. 3. Update the environment from the GitLab API. 4. Depending on whether the latest deployment's ref is a branch or tag, fetch the latest commit ID and timestamp. 5. Calculate how many commits behind the environment is compared to the latest commit. 6. Update or reuse stored metrics for environment commit counts and behind durations. 7. Update deployment count if a new deployment job has started. 8. Store various deployment-related metrics including durations, job IDs, timestamps, and status. 9. Emit status metrics considering if sparse output is enabled.

Returns an error if any step fails.

func (*Controller) PullEnvironmentsFromProject 

func (c *Controller) PullEnvironmentsFromProject(ctx context.Context, p schemas.Project) (err error)

PullEnvironmentsFromProject fetches the list of environments for a given project from the GitLab API, then checks if each environment already exists in the local store. For any environment not found in the store, it updates the store with the new environment, logs the discovery, and schedules a task to pull metrics for that environment.

func (*Controller) PullProject 

func (c *Controller) PullProject(ctx context.Context, name string, pull config.ProjectPull) error

PullProject fetches a GitLab project by its name, updates local store and schedules follow-up tasks. It takes a context, the project name, and a pull configuration as parameters.

func (*Controller) PullProjectsFromWildcard 

func (c *Controller) PullProjectsFromWildcard(ctx context.Context, w config.Wildcard) error

PullProjectsFromWildcard searches for projects matching the given wildcard criteria, adds new projects to the store, and schedules related tasks.

func (*Controller) PullRefMetrics 

func (c *Controller) PullRefMetrics(ctx context.Context, ref schemas.Ref) error

PullRefMetrics fetches and updates metrics related to a specific GitLab ref (branch, tag, or merge request). It retrieves the latest pipeline information, updates stored metrics, and optionally pulls job and test report metrics.

func (*Controller) PullRefMostRecentJobsMetrics 

func (c *Controller) PullRefMostRecentJobsMetrics(ctx context.Context, ref schemas.Ref) error

PullRefMostRecentJobsMetrics collects and processes metrics for the most recent jobs associated with a specific Git reference (e.g., a branch or tag).

func (*Controller) PullRefPipelineJobsMetrics 

func (c *Controller) PullRefPipelineJobsMetrics(ctx context.Context, ref schemas.Ref) error

PullRefPipelineJobsMetrics fetches the pipeline jobs for the given reference and processes their metrics.

func (*Controller) PullRefsFromProject 

func (c *Controller) PullRefsFromProject(ctx context.Context, p schemas.Project) error

PullRefsFromProject retrieves all refs (branches, tags, merge requests) for the given project `p` by calling GetRefs. For each ref retrieved, it checks if the ref already exists in the store. If the ref does not exist, it logs the discovery of a new ref, saves it to the store, and schedules a task to pull metrics related to that ref.

This function returns an error if any operation fails (fetching refs, checking existence, storing the ref). Otherwise, it returns nil.

The process helps keep the store updated with the latest refs and triggers metrics collection for newly discovered refs.

func (*Controller) PullRunnersFromProject added in v0.1.6 

func (c *Controller) PullRunnersFromProject(ctx context.Context, p schemas.Project) (err error)

PullRunnersFromProject fetches the list of runners for a given project from the GitLab API, then checks if each runner already exists in the local store. For any runner not found in the store, it updates the store with the new runner, logs the discovery, and schedules a task to pull metrics for that runner.

func (*Controller) Schedule 

func (c *Controller) Schedule(ctx context.Context, pull config.Pull, gc config.GarbageCollect)

Schedule initializes and schedules various periodic tasks based on configuration. It starts an OpenTelemetry span to trace scheduling operations.

It launches a background goroutine to refresh GitLab metadata asynchronously.

Then, it iterates over a map of task types and their scheduler configurations derived from the Pull and GarbageCollect configs.

For each task type: - If OnInit is true, the task is scheduled immediately once. - If Scheduled is true, the task is scheduled repeatedly at the configured interval.

If a Redis client is configured, it also schedules a keepalive task for Redis.

Note: The Redis keepalive scheduling currently happens inside the loop for each task, 

which might be more efficient to call just once outside the loop.

func (*Controller) ScheduleRedisSetKeepalive 

func (c *Controller) ScheduleRedisSetKeepalive(ctx context.Context)

ScheduleRedisSetKeepalive periodically updates a Redis key to signal that this instance of the process is alive and actively processing tasks.

It starts a new goroutine that:

  • Creates a ticker firing every 1 seconds.
  • On each tick, it calls SetKeepalive on the Redis store to update the key with a 10-second expiration, effectively refreshing the liveness indicator.
  • If the context is canceled, the goroutine logs and exits cleanly.

If updating the keepalive key fails, it logs a fatal error and terminates the process, since keepalive failures indicate a critical problem with Redis connectivity or availability.

func (*Controller) ScheduleTask 

func (c *Controller) ScheduleTask(ctx context.Context, tt schemas.TaskType, uniqueID string, args ...interface{})

ScheduleTask schedules a new task of type `tt` with a unique identifier `uniqueID` and optional arguments.

It performs the following steps:

  1. Starts an OpenTelemetry span for tracing the scheduling operation, annotating it with the task type and unique ID.
  2. Retrieves the task constructor from the TaskMap and creates a new job instance with the provided arguments.
  3. Checks the current length of the task queue to avoid overfilling it beyond its buffer size. If the queue is full, the task is not scheduled.
  4. Attempts to declare the task in the persistent store queue to ensure idempotency and track the task state. If the task is already queued, it skips scheduling to avoid duplicates.
  5. If the task is successfully registered and the queue has capacity, it asynchronously adds the job to the task queue.
  6. Logs warnings or debug messages at each failure or skip point to aid in diagnostics.

This function helps ensure tasks are only scheduled when the queue has capacity and the task is not already enqueued, preventing duplicate work and managing system load effectively.

func (*Controller) ScheduleTaskWithTicker 

func (c *Controller) ScheduleTaskWithTicker(ctx context.Context, tt schemas.TaskType, intervalSeconds int)

ScheduleTaskWithTicker repeatedly schedules a task of the specified type `tt` at fixed intervals defined by `intervalSeconds`.

It performs the following: 1. Starts an OpenTelemetry span for tracing, recording the task type and interval. 2. Validates the interval; if it is zero or negative, logs a warning and disables scheduling for the task. 3. Logs a debug message confirming the task has been scheduled with the given interval. 4. Updates monitoring metadata to indicate when the next scheduling is expected. 5. Launches a goroutine that ticks every `intervalSeconds` seconds:

  • On each tick, it schedules the task using `ScheduleTask` with a fixed unique ID "_".
  • Updates monitoring to track the next scheduled time.
  • Listens for context cancellation to cleanly stop the ticker and log shutdown.

This function ensures that periodic tasks are automatically scheduled at consistent intervals, allowing the system to regularly perform recurring work without manual intervention.

func (*Controller) TaskHandlerGarbageCollectEnvironments 

func (c *Controller) TaskHandlerGarbageCollectEnvironments(ctx context.Context) error

TaskHandlerGarbageCollectEnvironments handles the task of garbage collecting unused or obsolete environments. It ensures the task is properly unqueued and updates task scheduling monitoring. Returns an error if the garbage collection fails.

func (*Controller) TaskHandlerGarbageCollectMetrics 

func (c *Controller) TaskHandlerGarbageCollectMetrics(ctx context.Context) error

TaskHandlerGarbageCollectMetrics handles the garbage collection of metrics data. It ensures that the task is properly unqueued once done, and updates monitoring info about the last time this type of task was scheduled. Any error from the actual garbage collection is returned.

func (*Controller) TaskHandlerGarbageCollectProjects 

func (c *Controller) TaskHandlerGarbageCollectProjects(ctx context.Context) error

TaskHandlerGarbageCollectProjects handles the task of garbage collecting unused or obsolete projects. It ensures the task is properly unqueued and updates task scheduling monitoring. Returns an error if the garbage collection fails.

func (*Controller) TaskHandlerGarbageCollectRefs 

func (c *Controller) TaskHandlerGarbageCollectRefs(ctx context.Context) error

TaskHandlerGarbageCollectRefs handles the task of garbage collecting unused or obsolete refs. It ensures the task is properly unqueued and updates task scheduling monitoring. Returns an error if the garbage collection fails.

func (*Controller) TaskHandlerGarbageCollectRunners added in v0.1.6 

func (c *Controller) TaskHandlerGarbageCollectRunners(ctx context.Context) error

TaskHandlerGarbageCollectRunners TODO

func (*Controller) TaskHandlerPullEnvironmentMetrics 

func (c *Controller) TaskHandlerPullEnvironmentMetrics(ctx context.Context, env schemas.Environment)

TaskHandlerPullEnvironmentMetrics handles the task of pulling metrics for a specific environment. It ensures the task is unqueued after processing, and logs any errors encountered without retrying the task.

func (*Controller) TaskHandlerPullEnvironmentsFromProject 

func (c *Controller) TaskHandlerPullEnvironmentsFromProject(ctx context.Context, p schemas.Project)

TaskHandlerPullEnvironmentsFromProject handles the task of pulling environments from a given project. It unqueues the task once done and logs any errors without retrying.

func (*Controller) TaskHandlerPullEnvironmentsFromProjects 

func (c *Controller) TaskHandlerPullEnvironmentsFromProjects(ctx context.Context)

TaskHandlerPullEnvironmentsFromProjects schedules tasks to pull environments from all stored projects. It dequeues the current task after execution and updates task scheduling monitoring.

For each project in the store, it schedules a task to pull that project's environments.

func (*Controller) TaskHandlerPullMetrics 

func (c *Controller) TaskHandlerPullMetrics(ctx context.Context)

TaskHandlerPullMetrics schedules metric pull tasks for all environments and refs stored in the system. After running, it removes itself from the queue and updates task scheduling monitoring.

It fetches all environments and refs from the store, then schedules separate tasks to pull metrics for each environment and each ref.

func (*Controller) TaskHandlerPullProject 

func (c *Controller) TaskHandlerPullProject(ctx context.Context, name string, pull config.ProjectPull) error

TaskHandlerPullProject handles a task to pull metrics or data for a specific project. It ensures that the task is unqueued after processing regardless of success or failure.

func (*Controller) TaskHandlerPullProjectsFromWildcard 

func (c *Controller) TaskHandlerPullProjectsFromWildcard(ctx context.Context, id string, w config.Wildcard) error

TaskHandlerPullProjectsFromWildcard handles a task to pull multiple projects based on a wildcard configuration. It unqueues the task when done.

func (*Controller) TaskHandlerPullProjectsFromWildcards 

func (c *Controller) TaskHandlerPullProjectsFromWildcards(ctx context.Context)

TaskHandlerPullProjectsFromWildcards schedules tasks to pull projects matching configured wildcards. It dequeues the current task when done and updates monitoring for task scheduling. For each wildcard configured, it schedules a task to pull projects that match that wildcard.

func (*Controller) TaskHandlerPullRefMetrics 

func (c *Controller) TaskHandlerPullRefMetrics(ctx context.Context, ref schemas.Ref)

TaskHandlerPullRefMetrics handles the task of pulling metrics for a specific ref (branch, tag, or merge request) within a project. It ensures the task is dequeued once completed. Errors during the pull are logged as warnings without retrying.

func (*Controller) TaskHandlerPullRefsFromProject 

func (c *Controller) TaskHandlerPullRefsFromProject(ctx context.Context, p schemas.Project)

TaskHandlerPullRefsFromProject handles the task of pulling refs (branches, tags, etc.) from a given project. It ensures the task is removed from the queue after execution. Any errors encountered during the pull are logged as warnings, and the task is not retried.

func (*Controller) TaskHandlerPullRefsFromProjects 

func (c *Controller) TaskHandlerPullRefsFromProjects(ctx context.Context)

TaskHandlerPullRefsFromProjects schedules tasks to pull refs (branches, tags, MRs) from all stored projects. It dequeues the current task after execution and updates task scheduling monitoring.

For each project in the store, it schedules a task to pull that project's refs.

func (*Controller) TaskHandlerPullRunnerMetrics added in v0.1.6 

func (c *Controller) TaskHandlerPullRunnerMetrics(ctx context.Context, runner schemas.Runner)

TaskHandlerPullRunnerMetrics TODO

func (*Controller) TaskHandlerPullRunnersFromProject added in v0.1.6 

func (c *Controller) TaskHandlerPullRunnersFromProject(ctx context.Context, p schemas.Project)

TaskHandlerPullRunnersFromProject TODO

func (*Controller) TaskHandlerPullRunnersFromProjects added in v0.1.6 

func (c *Controller) TaskHandlerPullRunnersFromProjects(ctx context.Context)

TaskHandlerPullRunnersFromProjects TODO

func (*Controller) UpdateEnvironment 

func (c *Controller) UpdateEnvironment(ctx context.Context, env *schemas.Environment) error

UpdateEnvironment fetches the latest state of a given environment from the GitLab API, updates the local environment object with the latest details (availability, external URL, and latest deployment), and then saves the updated environment back to the local store.

func (*Controller) UpdateRunner added in v0.1.6 

func (c *Controller) UpdateRunner(ctx context.Context, runner *schemas.Runner) error

UpdateRunner fetches the latest state of a given runner from the GitLab API, updates the local runner object with the latest details (Paused, Contacted At, and maintenance notes), and then saves the updated runner back to the local store.

func (*Controller) WebhookHandler 

func (c *Controller) WebhookHandler(w http.ResponseWriter, r *http.Request)

WebhookHandler handles incoming GitLab webhook HTTP requests.

type Registry 

type Registry struct {
	*prometheus.Registry // The main Prometheus registry.

	// InternalCollectors holds custom internal application metrics (not user-facing).
	InternalCollectors struct {
		CurrentlyQueuedTasksCount  prometheus.Collector // Number of tasks currently queued.
		EnvironmentsCount          prometheus.Collector // Total number of environments tracked.
		ExecutedTasksCount         prometheus.Collector // Total number of tasks that have been executed.
		GitLabAPIRequestsCount     prometheus.Collector // Total number of GitLab API requests made.
		GitlabAPIRequestsRemaining prometheus.Collector // Number of remaining GitLab API requests (rate limit).
		GitlabAPIRequestsLimit     prometheus.Collector // GitLab API request limit.
		MetricsCount               prometheus.Collector // Number of exported metrics.
		ProjectsCount              prometheus.Collector // Total number of GitLab projects tracked.
		RefsCount                  prometheus.Collector // Total number of Git refs (branches, tags, MRs).
		RunnersCount               prometheus.Collector // Total number of Runners exported.
	}

	// Collectors maps each MetricKind to its Prometheus collector.
	Collectors RegistryCollectors
}

Registry wraps a pointer to prometheus.Registry and manages metric collectors.

func NewRegistry 

func NewRegistry(ctx context.Context) *Registry

NewRegistry initializes and returns a new Registry instance with all the necessary collectors registered.

func (*Registry) ExportInternalMetrics 

func (r *Registry) ExportInternalMetrics(ctx context.Context, g *gitlab.Client, s store.Store) (err error)

ExportInternalMetrics gathers internal statistics from the store and GitLab client, then sets the values for the corresponding Prometheus internal collectors.

func (*Registry) ExportMetrics 

func (r *Registry) ExportMetrics(metrics schemas.Metrics)

ExportMetrics updates the corresponding Prometheus collectors with the provided metric data. It iterates over all metrics and dispatches their values to the appropriate registered collectors.

func (*Registry) GetCollector 

func (r *Registry) GetCollector(kind schemas.MetricKind) prometheus.Collector

GetCollector returns the Prometheus collector associated with the given metric kind. It retrieves the collector from the Registry.Collectors map using the provided metric kind as the key.

func (*Registry) RegisterCollectors 

func (r *Registry) RegisterCollectors() error

RegisterCollectors adds all defined custom metric collectors to the Prometheus registry. It iterates over the Registry.Collectors map and registers each collector. If a registration fails, it returns a formatted error.

func (*Registry) RegisterInternalCollectors 

func (r *Registry) RegisterInternalCollectors()

RegisterInternalCollectors declares and registers internal application metrics to the Prometheus registry.

type RegistryCollectors 

type RegistryCollectors map[schemas.MetricKind]prometheus.Collector

RegistryCollectors defines a mapping between metric kinds and their Prometheus collectors.

type TaskController 

type TaskController struct {
	Factory                  taskq.Factory                                      // Factory creates task queues and manages their lifecycle.
	Queue                    taskq.Queue                                        // Queue is the actual task queue instance where tasks are enqueued and consumed.
	TaskMap                  *taskq.TaskMap                                     // TaskMap holds the mapping of task types to their handlers for processing.
	TaskSchedulingMonitoring map[schemas.TaskType]*monitor.TaskSchedulingStatus // TaskSchedulingMonitoring holds monitoring status per task type to track scheduling health.
}

TaskController holds the components needed to manage task queues and scheduling.

func NewTaskController 

func NewTaskController(ctx context.Context, r *redis.Client, maximumJobsQueueSize int) (t TaskController)

NewTaskController initializes and returns a new TaskController. It sets up the task queue backed either by Redis (if provided) or an in-memory queue. maximumJobsQueueSize controls the queue buffer size. The function also starts consumers if Redis is used and purges the queue at startup.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.