README

BuildStatus Build status Docker Pulls

Cloudprober

cloudprober.org

Cloudprober is a monitoring software that makes it super-easy to monitor availability and performance of various components of your system. Cloudprober employs the "active" monitoring model. It runs probes against (or on) your components to verify that they are working as expected. For example, it can run a probe to verify that your frontends can reach your backends. Similarly it can run a probe to verify that your in-Cloud VMs can actually reach your on-premise systems. This kind of monitoring makes it possible to monitor your systems' interfaces regardless of the implementation and helps you quickly pin down what's broken in your system.

Cloudprober UseCase

Features

  • Integration with open source monitoring stack of Prometheus and Grafana. Cloudprober exports probe results as counter based metrics that work well with Prometheus and Grafana.
  • Automated target discovery for Cloud targets. GCE and Kubernetes are supported out-of-the-box; other Cloud providers can be added easily.
  • Integration with StackDriver. If configured, Cloudprober exports probe results to StackDriver as custom metrics.
  • Built-in implementations for the most common probe types: PING, HTTP, UDP, DNS.
  • Arbitrary, complex probes can be run through the external probe type. For example, you could write a simple script to insert and delete a row in your database, and execute this script through the 'EXTERNAL' probe type.
  • Fast and efficient ping prober implementation that allows probing thousands of hosts with minimal resources.
  • Strong focus on ease of deployment. Cloudprober is written entirely in Go and compiles into a static binary. It can be easily deployed through docker containers. Thanks to the automated target discovery, there is usually no need to re-deploy or re-configure cloudprober in response to the most of the changes.
  • Go templates based config file with substitutions for standard variables like project, zone, instance names etc allows for using same config file across the fleet.
  • Low footprint. Cloudprober docker image is small, containing just the statically compiled binary and it takes very little CPU and RAM to run even a large number of probes.
  • Extensible architecture. Cloudprober can be easily extended along most of the dimensions. Adding support for other Cloud targets, monitoring systems (e.g. Graphite, Amazon Cloudwatch) and even a new probe type, is straight-forward and fairly easy.

Visit cloudprober.org to get started with Cloudprober.

We'd love to hear your feedback. If you're using Cloudprober, would you please mind sharing how you use it by adding a comment to the issue #123. It will be a great help in planning Cloudprober's future progression.

Join Cloudprober Slack or mailing list for questions and discussion about Cloudprober.

Expand ▾ Collapse ▴

Documentation

Overview

    Package cloudprober provides a prober for running a set of probes.

    Cloudprober takes in a config proto which dictates what probes should be created with what configuration, and manages the asynchronous fan-in/fan-out of the metrics data from these probes.

    Index

    Constants

    View Source
    const (
    	DefaultServerHost = ""
    	DefaultServerPort = 9313
    	ServerHostEnvVar  = "CLOUDPROBER_HOST"
    	ServerPortEnvVar  = "CLOUDPROBER_PORT"
    )

      Constants defining the default server host and port.

      Variables

      This section is empty.

      Functions

      func GetConfig

      func GetConfig() string

        GetConfig returns the prober config.

        func GetInfo

          GetInfo returns information on all the probes, servers and surfacers.

          func InitFromConfig

          func InitFromConfig(configFile string) error

            InitFromConfig initializes Cloudprober using the provided config.

            func Start

            func Start(ctx context.Context)

              Start starts a previously initialized Cloudprober.

              Types

              This section is empty.

              Source Files

              Directories

              Path Synopsis
              Binary cloudprober is a tool for running a set of probes and metric surfacers on a GCE VM.
              Binary cloudprober is a tool for running a set of probes and metric surfacers on a GCE VM.
              common
              file
              Package file implements utilities to read files from various backends.
              Package file implements utilities to read files from various backends.
              iputils
              Package iputils implements utilities to work with IP addresses.
              Package iputils implements utilities to work with IP addresses.
              message
              Package message implements wrappers for sending and receiving messages with sequence numbers and timestamps.
              Package message implements wrappers for sending and receiving messages with sequence numbers and timestamps.
              oauth
              Package oauth implements OAuth related utilities for Cloudprober.
              Package oauth implements OAuth related utilities for Cloudprober.
              tlsconfig
              Package tlsconfig implements utilities to parse TLSConfig.
              Package tlsconfig implements utilities to parse TLSConfig.
              Package config provides parser for cloudprober configs.
              Package config provides parser for cloudprober configs.
              runconfig
              Package runconfig stores cloudprober config that is specific to a single invocation.
              Package runconfig stores cloudprober config that is specific to a single invocation.
              contrib
              gcp/bigquery
              Package bigquery connects to the GCP BigQuery API and computes metrics suitable for blackbox-probing.
              Package bigquery connects to the GCP BigQuery API and computes metrics suitable for blackbox-probing.
              gcp/cmd
              Bigquery_probe is an external Cloudprober probe suitable for blackbox-probing the GCP BigQuery API.
              Bigquery_probe is an external Cloudprober probe suitable for blackbox-probing the GCP BigQuery API.
              examples
              external
              Binary redis_probe_once implements an example cloudprober external probe that puts a key-value in a redis server, retrieves it and reports time taken for both operations.
              Binary redis_probe_once implements an example cloudprober external probe that puts a key-value in a redis server, retrieves it and reports time taken for both operations.
              Package logger provides a logger that logs to Google Cloud Logging.
              Package logger provides a logger that logs to Google Cloud Logging.
              Package metrics implements data types for probes generated data.
              Package metrics implements data types for probes generated data.
              payload
              Package payload provides utilities to work with the metrics in payload.
              Package payload provides utilities to work with the metrics in payload.
              testutils
              Package testutils provides utilities for tests.
              Package testutils provides utilities for tests.
              Package prober provides a prober for running a set of probes.
              Package prober provides a prober for running a set of probes.
              cmd
              This binary implements a Cloudprober gRPC client.
              This binary implements a Cloudprober gRPC client.
              Package probes provides an interface to initialize probes using prober config.
              Package probes provides an interface to initialize probes using prober config.
              common/statskeeper
              Package statskeeper implements utilities that are shared across multiple probe types.
              Package statskeeper implements utilities that are shared across multiple probe types.
              dns
              Package dns implements a DNS prober.
              Package dns implements a DNS prober.
              dns/cmd
              Dns_bin implements a stand-alone dns prober binary using the cloudprober/probes/dns package.
              Dns_bin implements a stand-alone dns prober binary using the cloudprober/probes/dns package.
              external
              Package external implements an external probe type for cloudprober.
              Package external implements an external probe type for cloudprober.
              external/cmd
              This program implements a stand-alone external prober binary using the cloudprober/probes/external package.
              This program implements a stand-alone external prober binary using the cloudprober/probes/external package.
              external/serverutils
              Package serverutils provides utilities to work with the cloudprober's external probe.
              Package serverutils provides utilities to work with the cloudprober's external probe.
              grpc
              Package grpc implements a gRPC probe.
              Package grpc implements a gRPC probe.
              http
              Package http implements HTTP probe type.
              Package http implements HTTP probe type.
              http/cmd
              This program implements a stand-alone http prober binary using the cloudprober/http package.
              This program implements a stand-alone http prober binary using the cloudprober/http package.
              options
              Package options provides a shared interface to common probe options.
              Package options provides a shared interface to common probe options.
              ping
              Package ping implements a fast ping prober.
              Package ping implements a fast ping prober.
              ping/cmd
              This program implements a stand-alone ping prober binary using the cloudprober/ping package.
              This program implements a stand-alone ping prober binary using the cloudprober/ping package.
              probeutils
              Package probeutils implements utilities that are shared across multiple probe types.
              Package probeutils implements utilities that are shared across multiple probe types.
              udp
              Package udp implements a UDP prober.
              Package udp implements a UDP prober.
              udp/cmd
              Udp_bin implements a stand-alone udp prober binary using the cloudprober/probes/udp package.
              Udp_bin implements a stand-alone udp prober binary using the cloudprober/probes/udp package.
              udplistener
              Package udplistener implements a UDP listener.
              Package udplistener implements a UDP listener.
              rds
              client
              Package client implements a ResourceDiscovery service (RDS) client.
              Package client implements a ResourceDiscovery service (RDS) client.
              client/cmd
              This binary implements a standalone ResourceDiscovery service (RDS) client.
              This binary implements a standalone ResourceDiscovery service (RDS) client.
              gcp
              Package gcp implements a GCP (Google Compute Platform) resources provider for ResourceDiscovery server.
              Package gcp implements a GCP (Google Compute Platform) resources provider for ResourceDiscovery server.
              kubernetes
              Package kubernetes implements a kubernetes resources provider for ResourceDiscovery server.
              Package kubernetes implements a kubernetes resources provider for ResourceDiscovery server.
              server
              Package server provides a ResourceDiscovery gRPC server implementation.
              Package server provides a ResourceDiscovery gRPC server implementation.
              server/cmd
              This binary implements a stand-alone ResourceDiscovery server.
              This binary implements a stand-alone ResourceDiscovery server.
              server/filter
              Package filter implements common filters for the RDS (resource discovery service) providers.
              Package filter implements common filters for the RDS (resource discovery service) providers.
              Package servers provides an interface to initialize cloudprober servers using servers config.
              Package servers provides an interface to initialize cloudprober servers using servers config.
              grpc
              Package grpc provides a simple gRPC server that acts as a probe target.
              Package grpc provides a simple gRPC server that acts as a probe target.
              http
              Package http implements an HTTP server that simply returns 'ok' for any URL and sends stats on a string channel.
              Package http implements an HTTP server that simply returns 'ok' for any URL and sends stats on a string channel.
              udp
              Package udp implements a UDP server.
              Package udp implements a UDP server.
              udp/cmd
              This binary implements a stand-alone UDP server using the cloudprober/servers/udp/udp package.
              This binary implements a stand-alone UDP server using the cloudprober/servers/udp/udp package.
              Package surfacers is the base package for creating Surfacer objects that are used for writing metics data to different monitoring services.
              Package surfacers is the base package for creating Surfacer objects that are used for writing metics data to different monitoring services.
              common/compress
              Package compress implements compression related utilities.
              Package compress implements compression related utilities.
              file
              Package file implements the "file" surfacer.
              Package file implements the "file" surfacer.
              postgres
              Package postgres implements "postgres" surfacer.
              Package postgres implements "postgres" surfacer.
              prometheus
              Package prometheus provides a prometheus surfacer for Cloudprober.
              Package prometheus provides a prometheus surfacer for Cloudprober.
              pubsub
              Package pubsub implements the "pubsub" surfacer.
              Package pubsub implements the "pubsub" surfacer.
              stackdriver
              Package stackdriver implements the Stackdriver version of the Surfacer object.
              Package stackdriver implements the Stackdriver version of the Surfacer object.
              Package sysvars implements a system variables exporter.
              Package sysvars implements a system variables exporter.
              Package targets provides the means to list and resolve targets for probers in the cloudprober framework.
              Package targets provides the means to list and resolve targets for probers in the cloudprober framework.
              endpoint
              Package endpoint provides the type Endpoint, to be used with the targets.Targets interface.
              Package endpoint provides the type Endpoint, to be used with the targets.Targets interface.
              file
              Package file implements a file-based targets for cloudprober.
              Package file implements a file-based targets for cloudprober.
              gce
              Package gce implements Google Compute Engine (GCE) targets for Cloudprober.
              Package gce implements Google Compute Engine (GCE) targets for Cloudprober.
              lameduck
              Package lameduck implements a lameducks provider.
              Package lameduck implements a lameducks provider.
              resolver
              Package resolver provides a caching, non-blocking DNS resolver.
              Package resolver provides a caching, non-blocking DNS resolver.
              rtc/rtcservice
              Package rtcservice provides utility functions for the cloudprober project used when dealing with the Runtime Configurator API.
              Package rtcservice provides utility functions for the cloudprober project used when dealing with the Runtime Configurator API.
              Package validators provides an entrypoint for the cloudprober's validators framework.
              Package validators provides an entrypoint for the cloudprober's validators framework.
              http
              Package http provides an HTTP validator for the Cloudprober's validator framework.
              Package http provides an HTTP validator for the Cloudprober's validator framework.
              integrity
              Package integrity provides data integrity validator for the Cloudprober's validator framework.
              Package integrity provides data integrity validator for the Cloudprober's validator framework.
              regex
              Package regex provides regex validator for the Cloudprober's validator framework.
              Package regex provides regex validator for the Cloudprober's validator framework.
              web
              Package web provides web interface for cloudprober.
              Package web provides web interface for cloudprober.
              formatutils
              Package formatutils provides web related utils for the cloudprober sub-packages.
              Package formatutils provides web related utils for the cloudprober sub-packages.