cloudzero-agent

module
v1.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2025 License: Apache-2.0

README

CloudZero Agent

Contributor Covenant License GitHub release

deployment

This repository contains several applications to support Kubernetes integration with the CloudZero platform, including:

  • CloudZero Insights Controller - provides telemetry to the CloudZero platform to enabling complex cost allocation and analysis. This webhook application securely receives resource provisioning and deprovisioning requests from the Kubernetes API. It collects resource labels, annotations, and relationship metadata between resources, ultimately supporting the identification of CSP resources not directly connected to a Kubernetes node.

  • CloudZero Collector - The collector application which implements a prometheus compliant interface for metrics collection; which writes the metrics payloads to files to a shared location for consumption by the shipper. Today the collector classifies incoming metrics data, and will save the data into either cost telemetry files, or into observability files. These files are compressed on disk to save space.

  • CloudZero Shipper - The shipper application monitors shared locations for metrics file creation, allocates pre-signed S3 PUT URLs for customers (using the CloudZero upload API), and then uploads data to the AWS S3 bucket at set intervals. This approach protects against invalid API keys and enables end-to-end file tracking.

  • CloudZero Agent Validator - the validator application is part of the agent’s pod lifecycle hooks. It is responsible for performing basic validation checks, and notifying the CloudZero platform of installation status changes (initializing, started, stopping). This application runs during the lifecycle hook, then exits when complete.

Note the agent application which is responsible for executing metrics scrape jobs at various intervals. The agent will communicate with a kube-state-metrics exporter application, and cAdvisor exporter applications (one per machine instance). For large scale clusters, the agent runs in “federated mode” (aka daemonset mode), where each instance on each machine is responsible for metrics collection on that single machine.

⚡ Getting Started With CloudZero Insights Controller

The easiest way to get started with the CloudZero Insights Controller is by using the cloudzero-agent Helm chart from the cloudzero-charts repository.

Installation

See the Installation Guide for details.

Configuration

See the Configuration Guide for details.

Cleanup
make undeploy-admission-controller
make undeploy-test-app
Debugging

The applications are based on a scratch container, so no shell is available. The container images are less than 8MB.

To monitor the data directory, you must deploy a debug container as follows:

  1. Deploy a debug container

    kubectl apply  -f cluster/deployments/debug/deployment.yaml
    
  2. Attach to the shell of the debug container

    kubectl exec -it temp-shell -- /bin/sh
    

    To inspect the data directory, cd /cloudzero/data


Clean Up
eksctl delete cluster -f cluster/cluster.yaml --disable-nodegroup-eviction

Collector & Shipper Architecture

This project provides a collector application, written in golang, which provides two applications:

  • Collector - the collector application exposes a prometheus remote write API which can receive POST requests from prometheus in either v1 or v2 encoded format. It decodes the messages, then writes them to the data directory as Brotri-compressed JSON.
  • Shipper - the shipper application watches the data directory looking for completed parquet files on a regular interval (eg. 10 min), then will call the CloudZero upload API to allocate S3 Presigned PUT URLS. These URLs are used to upload the file. The application has the ability to compress the files before sending them to S3.

Message Format

The output of the CloudZero Insights Controller application is a JSON object that represents cloudzero metrics, which is POSTed to the CloudZero remote write API. The format of these objects is based on the Prometheus Timeseries protobuf message, defined here. Protobuf definitions for the cloudzero metrics are in the proto/ directory.

There are four kinds of objects that can be sent:

  1. Pod metrics
Metric Names
  • cloudzero_pod_labels
  • cloudzero_pod_annotations
Required Fields
  • __name__; will be one of the valid pod metric names
  • namespace; the namespace that the pod is launched in
  • resource_type; will always be pod for pod metrics
Example
{
  "labels": [
    {
      "name": "__name__",
      "value": "cloudzero_pod_labels"
    },
    {
      "name": "namespace",
      "value": "default"
    },
    {
      "name": "pod",
      "value": "hello-28889630-955wd"
    },
    {
      "name": "resource_type",
      "value": "pod"
    },
    {
      "name": "label_batch.kubernetes.io/controller-uid",
      "value": "cc52c38d-b461-40ab-a65d-2d5a68ac08e5"
    },
    {
      "name": "label_batch.kubernetes.io/job-name",
      "value": "hello-28889630"
    },
    {
      "name": "label_controller-uid",
      "value": "cc52c38d-b461-40ab-a65d-2d5a68ac08e5"
    },
    {
      "name": "label_job-name",
      "value": "hello-28889630"
    }
  ],
  "samples": [
    {
      "value": 1.0,
      "timestamp": "1733378003953"
    }
  ]
}
  1. Workload Metrics
Metric Names
  • cloudzero_deployment_labels
  • cloudzero_deployment_annotations
  • cloudzero_statefulset_labels
  • cloudzero_statefulset_annotations
  • cloudzero_daemonset_labels
  • cloudzero_daemonset_annotations
  • cloudzero_job_labels
  • cloudzero_job_annotations
  • cloudzero_cronjob_labels
  • cloudzero_cronjob_annotations
Required Fields
  • __name__; will be one of the valid workload metric names
  • namespace; the namespace that the workload is launched in
  • workload; the name of the workload
  • resource_type; will be one of deployment, statefulset, daemonset, job, or cronjob
Example
{
  "labels": [
    {
      "name": "__name__",
      "value": "cloudzero_deployment_labels"
    },
    {
      "name": "namespace",
      "value": "default"
    },
    {
      "name": "workload",
      "value": "hello"
    },
    {
      "name": "resource_type",
      "value": "deployment"
    },
    {
      "name": "label_component",
      "value": "greeting"
    },
    {
      "name": "label_foo",
      "value": "bar"
    }
  ],
  "samples": [
    {
      "value": 1.0,
      "timestamp": "1733378003953"
    }
  ]
}
  1. Namespace Metrics
Metric Names
  • cloudzero_namespace_labels
  • cloudzero_namespace_annotations
Required Fields
  • __name__; will be one of the valid namespace metric names
  • namespace; the name of the namespace
  • resource_type; will always be namespace for namespace metrics
Example
{
  "labels": [
    {
      "name": "__name__",
      "value": "cloudzero_namespace_labels"
    },
    {
      "name": "namespace",
      "value": "default"
    },
    {
      "name": "resource_type",
      "value": "namespace"
    },
    {
      "name": "label_engr.os.com/component",
      "value": "foo"
    },
    {
      "name": "label_kubernetes.io/metadata.name",
      "value": "default"
    }
  ],
  "samples": [
    {
      "value": 1.0,
      "timestamp": "1733880410225"
    }
  ]
}
  1. Node Metrics
Metric Names
  • cloudzero_node_labels
  • cloudzero_node_annotations
Required Fields
  • __name__; will be one of the valid node metric names
  • node; the name of the node
  • resource_type; will always be node for node metrics
Example
{
  "labels": [
    {
      "name": "__name__",
      "value": "cloudzero_node_labels"
    },
    {
      "name": "resource_type",
      "value": "node"
    },
    {
      "name": "label_alpha.eksctl.io/nodegroup-name",
      "value": "spot-nodes"
    },
    {
      "name": "label_beta.kubernetes.io/arch",
      "value": "amd64"
    }
  ],
  "samples": [
    {
      "value": 1.0,
      "timestamp": "1733880410225"
    }
  ]
}

🤝 How to Contribute

We appreciate feedback and contribution to this repo! Before you get started, please see the following:

🤔 Support + Feedback

Contact support@cloudzero.com for usage, questions, specific cases. See the CloudZero Docs for general information on CloudZero.

🛡️ Vulnerability Reporting

Please do not report security vulnerabilities on the public GitHub issue tracker. Email security@cloudzero.com instead.

☁️ What is CloudZero?

CloudZero is the only cloud cost intelligence platform that puts engineering in control by connecting technical decisions to business results.:

  • Cost Allocation And Tagging Organize and allocate cloud spend in new ways, increase tagging coverage, or work on showback.
  • Kubernetes Cost Visibility Understand your Kubernetes spend alongside total spend across containerized and non-containerized environments.
  • FinOps And Financial Reporting Operationalize reporting on metrics such as cost per customer, COGS, gross margin. Forecast spend, reconcile invoices and easily investigate variance.
  • Engineering Accountability Foster a cost-conscious culture, where engineers understand spend, proactively consider cost, and get immediate feedback with fewer interruptions and faster and more efficient innovation.
  • Optimization And Reducing Waste Focus on immediately reducing spend by understanding where we have waste, inefficiencies, and discounting opportunities.

Learn more about CloudZero on our website www.cloudzero.com

📜 License

This project is licensed under the Apache 2.0 LICENSE.

Directories

Path Synopsis
app
build
Package build contains build information for the application.
Package build contains build information for the application.
compress
Package compress provides functionality to compress a file into a tar.gz archive.
Package compress provides functionality to compress a file into a tar.gz archive.
config
Package config contains code for all configs to share
Package config contains code for all configs to share
config/gator
Package config implements the configuration for the aggregator..
Package config implements the configuration for the aggregator..
config/validator
Package config contains configuration settings.
Package config contains configuration settings.
config/webhook
Package config contains the configuration for the application.
Package config contains the configuration for the application.
domain
Package domain provides domain logic.
Package domain provides domain logic.
domain/diagnostic
Package diagnostic contains an interface to be implemented by diagnostics providers.
Package diagnostic contains an interface to be implemented by diagnostics providers.
domain/diagnostic/catalog
Package catalog contains the registry of diagnostics.
Package catalog contains the registry of diagnostics.
domain/diagnostic/common
Package common contains common utilities.
Package common contains common utilities.
domain/diagnostic/cz
Package cz contains code for checking a CloudZero API token.
Package cz contains code for checking a CloudZero API token.
domain/diagnostic/k8s/namespace
Package namespace contains code for checking the Kubernetes configuration.
Package namespace contains code for checking the Kubernetes configuration.
domain/diagnostic/k8s/provider
Package provider contains code for checking the Kubernetes configuration.
Package provider contains code for checking the Kubernetes configuration.
domain/diagnostic/k8s/version
Package version contains code for checking the Kubernetes configuration.
Package version contains code for checking the Kubernetes configuration.
domain/diagnostic/kms
Package kms contains code for testing access the Kubernetes Management Service.
Package kms contains code for testing access the Kubernetes Management Service.
domain/diagnostic/prom/config
Package config contains a diagnostic provider for checking the Prometheus configuration.
Package config contains a diagnostic provider for checking the Prometheus configuration.
domain/diagnostic/prom/version
Package version contains a diagnostic provider for checking the Prometheus version.
Package version contains a diagnostic provider for checking the Prometheus version.
domain/diagnostic/runner
Package runner contains tools for running diagnostics.
Package runner contains tools for running diagnostics.
domain/diagnostic/stage
Package stage contains a diagnostic provider for checking the stage.
Package stage contains a diagnostic provider for checking the stage.
domain/diagnostic/webhook
Package webhook contains code for checking a CloudZero API token.
Package webhook contains code for checking a CloudZero API token.
domain/filter
Package filter provides low-level functionality for metric filtering.
Package filter provides low-level functionality for metric filtering.
domain/healthz
Package healthz provides a simple and extensible health check mechanism for HTTP services.
Package healthz provides a simple and extensible health check mechanism for HTTP services.
domain/housekeeper
Package housekeeper provides a mechanism for cleaning up stale data in a resource store.
Package housekeeper provides a mechanism for cleaning up stale data in a resource store.
domain/k8s
Package k8s gives a unified interface for k8s information to be retrieved.
Package k8s gives a unified interface for k8s information to be retrieved.
domain/monitor
Package monitor provides functionality to manage and reload secrets dynamically.
Package monitor provides functionality to manage and reload secrets dynamically.
domain/pusher
Package pusher provides a mechanism for pushing metrics to a remote write endpoint.
Package pusher provides a mechanism for pushing metrics to a remote write endpoint.
domain/shipper
Package shipper provides domain logic for the shipper.
Package shipper provides domain logic for the shipper.
domain/webhook
Package webhook provides kubernetes webhook resource business logic.
Package webhook provides kubernetes webhook resource business logic.
domain/webhook/backfiller
Package backfiller provides functionality to backfill Kubernetes Resource objects, and if enabled invokes the webhook domain logic
Package backfiller provides functionality to backfill Kubernetes Resource objects, and if enabled invokes the webhook domain logic
domain/webhook/handler
Package handler admission webhook handlers (hook.Handler) for various resource types.
Package handler admission webhook handlers (hook.Handler) for various resource types.
domain/webhook/helper
Package helper contains decode helper methods for transforming kubernetes metav1.Objects into K8sObjects
Package helper contains decode helper methods for transforming kubernetes metav1.Objects into K8sObjects
domain/webhook/hook
Package hook contains structures and interfaces for implementing admission webhooks handlers.
Package hook contains structures and interfaces for implementing admission webhooks handlers.
functions/agent-validator/config
Package config contains a CLI for managing configuration files.
Package config contains a CLI for managing configuration files.
functions/agent-validator/diagnose
Package diagnose contains a CLI for running diagnostics.
Package diagnose contains a CLI for running diagnostics.
functions/agent-validator/install
Package install contains a CLI for copying the executable to a destination.
Package install contains a CLI for copying the executable to a destination.
functions/cluster-config/loader
Package loader provides code to load all the different config types
Package loader provides code to load all the different config types
functions/helmless
Package main implements a tool for comparing configured values against default values from a Helm chart.
Package main implements a tool for comparing configured values against default values from a Helm chart.
functions/helmless/overrides
Package overrides provides functionality for extracting configuration overrides by comparing configured values against default values from Helm charts.
Package overrides provides functionality for extracting configuration overrides by comparing configured values against default values from Helm charts.
handlers
Package handlers provides HTTP handlers.
Package handlers provides HTTP handlers.
http/client
Package http contains utilities for making HTTP requests.
Package http contains utilities for making HTTP requests.
http/middleware
Package middleware provides standard app middlware implementations
Package middleware provides standard app middlware implementations
inspector
Package inspector provides a way to inspect HTTP responses from the CloudZero API to diagnose issues.
Package inspector provides a way to inspect HTTP responses from the CloudZero API to diagnose issues.
logging
Package logging provides a wrapper around Zerolog.
Package logging provides a wrapper around Zerolog.
logging/instr
Package instr provides instrumentation utilities.
Package instr provides instrumentation utilities.
logging/validator
Package logging contains utilities for logging.
Package logging contains utilities for logging.
storage/core
Package core provides core functionalities for database repository implementations.
Package core provides core functionalities for database repository implementations.
storage/disk
Package disk provides storage functionality.
Package disk provides storage functionality.
storage/repo
Package repo provides implementations for resource repository interfaces.
Package repo provides implementations for resource repository interfaces.
storage/sqlite
Package sqlite provides implementations for resource repository interfaces using SQLite as the underlying database.
Package sqlite provides implementations for resource repository interfaces using SQLite as the underlying database.
types
Package types contains types and errors used throughout the application.
Package types contains types and errors used throughout the application.
types/clusterconfig
Package clusterconfig provides clusterconfig
Package clusterconfig provides clusterconfig
types/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
types/status
Package status contains generated code for reading and writing the ClusterStatus protobuf.
Package status contains generated code for reading and writing the ClusterStatus protobuf.
utils
Package utils contains utility functions and types used throughout the application.
Package utils contains utility functions and types used throughout the application.
utils/k8s
Package k8s contains helpers for working with the Kubernetes Client.
Package k8s contains helpers for working with the Kubernetes Client.
utils/lock
Package lock provides an interface for file-based locking.
Package lock provides an interface for file-based locking.
utils/parallel
Package parallel provides utilities for running tasks in parallel.
Package parallel provides utilities for running tasks in parallel.
utils/scout
Package scout provides cloud environment detection and metadata retrieval capabilities for cloud environments.
Package scout provides cloud environment detection and metadata retrieval capabilities for cloud environments.
utils/scout/auto
Package auto provides auto-detection capabilities for the CloudZero Scout.
Package auto provides auto-detection capabilities for the CloudZero Scout.
utils/scout/aws
Package aws provides AWS cloud environment detection and metadata retrieval capabilities using the EC2 instance metadata service (IMDS) v2.
Package aws provides AWS cloud environment detection and metadata retrieval capabilities using the EC2 instance metadata service (IMDS) v2.
utils/scout/azure
Package azure provides Azure cloud environment detection and metadata retrieval capabilities using the Azure Instance Metadata Service (IMDS).
Package azure provides Azure cloud environment detection and metadata retrieval capabilities using the Azure Instance Metadata Service (IMDS).
utils/scout/google
Package google provides functionality for detecting and gathering environment information from Google Cloud metadat services.
Package google provides functionality for detecting and gathering environment information from Google Cloud metadat services.
utils/scout/types
Package types defines core types and interfaces for cloud environment detection and metadata retrieval.
Package types defines core types and interfaces for cloud environment detection and metadata retrieval.
utils/scout/types/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
utils/telemetry
Package telemetry contains code for posting telemetry data to the CloudZero API.
Package telemetry contains code for posting telemetry data to the CloudZero API.
mock
controller/controller
Package controller provides a mock insights controller.
Package controller provides a mock insights controller.
metrics
Package metrics provides utilities for generating metrics.
Package metrics provides utilities for generating metrics.
remotewrite/pkg
Package remotewrite provides a mock remote write server.
Package remotewrite provides a mock remote write server.
tests
backfiller
Package backfiller provides utilities for testing label backfiller integrations.
Package backfiller provides utilities for testing label backfiller integrations.
integration
Package integration provides integration tests.
Package integration provides integration tests.
smoke
Package smoke provides smoke tests.
Package smoke provides smoke tests.
utils
Package utils provides utilities supporting the smoke tests.
Package utils provides utilities supporting the smoke tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL