monitoring

Documentation

Overview

Package monitoring provides a monitoring endpoint and Continuous Profiling for Go service processes.

- The monitoring endpoint will expose OpenMetrics at `/metrics`.

• Pprof service endpoints at `/debug/pprof`. More information about the pprof endpoints is available at https://golang.org/pkg/net/http/pprof/

- Support the gitlab_build_info metric for exposing application version information

This package includes optional support for continuous profiling, which is responsible for heap and cpu profiling of the Go process. Currently, the only supported driver is Google Cloud Platform's Stackdriver Profiler (see https://cloud.google.com/profiler/).

The profiler is initialized upon `monitoring.Start` call.

*Compiling applications with Profiler support*

For compiling binaries with Stackdriver Profiler support (including its dependencies), build your binary with the `continuous_profiler_stackdriver` compilation tag:

* `go build -tags="continuous_profiler_stackdriver"“

*Initializing the Profiler*

Use the following pattern in the `GITLAB_CONTINUOUS_PROFILING` environment variable for getting it started:

* `GITLAB_CONTINUOUS_PROFILING="stackdriver?service=gitaly&service_version=1.0.1&project_id=test-123"`

For more information about each argument see https://godoc.org/cloud.google.com/go/profiler#Config. Most of these shouldn't be required in GCP environments, so a simpler version could be used:

* `GITLAB_CONTINUOUS_PROFILING="stackdriver"`

See https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application for authentication details.

*Profiler overhead*

Google Cloud claim Stackdriver Profiler adds a 5% performance overhead to processes when it is enabled. This cost is only incurred when the profiler is switched on (through the `GITLAB_CONTINUOUS_PROFILING`) environment variable. More details can be found at https://medium.com/google-cloud/continuous-profiling-of-go-programs-96d4416af77b.

*How to make use of profiles*

- https://youtu.be/MxGQ7WZ2N74 : A look at how one can use Go profiles in practice

Index

• Constants
• func Serve(options ...Option) error
• func Start(options ...Option) error
• type Option
  • func WithBuildExtraLabels(labels map[string]string) Option
  • func WithBuildInformation(version string, buildTime string) Option
  • func WithGoBuildInformation(info *debug.BuildInfo) Option
  • func WithListener(listener net.Listener) Option
  • func WithListenerAddress(addr string) Option
  • func WithMetricsHandlerPattern(pattern string) Option
  • func WithProfilerCredentialsFile(path string) Option
  • func WithPrometheusGatherer(gatherer prometheus.Gatherer) Option
  • func WithPrometheusRegisterer(registerer prometheus.Registerer) Option
  • func WithServeMux(mux *http.ServeMux) Option
  • func WithServer(s *http.Server) Option
  • func WithoutContinuousProfiling() Option
  • func WithoutMetrics() Option
  • func WithoutPprof() Option

Examples

• Start

Constants

const (
	// GitlabBuildInfoGaugeMetricName is the name of the label containing
	// build information for this process.
	GitlabBuildInfoGaugeMetricName = "gitlab_build_info"
)

Functions

func Serve

func Serve(options ...Option) error

Serve will start a new monitoring service listening on the address configured through the option arguments. Additionally, it'll start a Continuous Profiler configured through environment variables (see more at https://gitlab.com/gitlab-org/labkit/-/blob/master/monitoring/doc.go).

If `WithListenerAddress` option is provided, Serve will block or return a non-nil error, similar to `http.ListenAndServe` (for instance).

Deprecated: Use Start instead.

func Start

func Start(options ...Option) error

Start will start a new monitoring service listening on the address configured through the option arguments. Additionally, it'll start a Continuous Profiler configured through environment variables (see more at https://gitlab.com/gitlab-org/labkit/-/blob/master/monitoring/doc.go).

If `WithListenerAddress` option is provided, Start will block or return a non-nil error, similar to `http.ListenAndServe` (for instance).

Example

package main

import (
	"log"
	"runtime/debug"

	"gitlab.com/gitlab-org/labkit/monitoring"
)

func main() {
	go func() {
		// Add the standard version and build time labels
		buildInfoOption := monitoring.WithBuildInformation("0.1.1", "2019-09-01T00:22:00Z")
		if buildInfo, ok := debug.ReadBuildInfo(); ok {
			// If we're able to read the Go build information from
			// the currently running binary then we use that as
			// build info option instead. Binaries that are built
			// from a Go project with module support should always
			// have build info available.
			buildInfoOption = monitoring.WithGoBuildInformation(buildInfo)
		}

		log.Fatal(monitoring.Start(
			// Listen on port 7822 on all interfaces
			monitoring.WithListenerAddress(":7822"),
			// Add the standard version and build time labels
			buildInfoOption,
			// Add any additional application-specific labels to the `gitlab_build_info` metric
			monitoring.WithBuildExtraLabels(map[string]string{
				"git_version": "2.0.0",
			}),
		))
	}()
}

Types

type Option

type Option func(*optionsConfig)

Option is used to pass options to NewListener.

func WithBuildExtraLabels

func WithBuildExtraLabels(labels map[string]string) Option

WithBuildExtraLabels will configure extra labels on the `gitlab_build_info` metric.

func WithBuildInformation

func WithBuildInformation(version string, buildTime string) Option

WithBuildInformation will configure the `gitlab_build_info` metric with appropriate labels.

func WithGoBuildInformation

func WithGoBuildInformation(info *debug.BuildInfo) Option

WithGoBuildInformation will configure the `gitlab_build_info` metric with appropriate labels derived from the runtime/debug.BuildInfo.

func WithListener

func WithListener(listener net.Listener) Option

WithListener will configure the health check endpoint to use the provided listener.

func WithListenerAddress

func WithListenerAddress(addr string) Option

WithListenerAddress will configure the health check endpoint to use the provided listener.

func WithMetricsHandlerPattern

func WithMetricsHandlerPattern(pattern string) Option

WithMetricsHandlerPattern configures the pattern that the metrics handler is registered for (defaults to `/metrics`).

func WithProfilerCredentialsFile

func WithProfilerCredentialsFile(path string) Option

WithProfilerCredentialsFile configures the credentials file to be used for the profiler service.

func WithPrometheusGatherer

func WithPrometheusGatherer(gatherer prometheus.Gatherer) Option

WithPrometheusGatherer sets the prometheus.Gatherer to expose in the metrics endpoint.

func WithPrometheusRegisterer

func WithPrometheusRegisterer(registerer prometheus.Registerer) Option

WithPrometheusRegisterer sets the prometheus.Registerer to use to register metrics from this package.

func WithServeMux

func WithServeMux(mux *http.ServeMux) Option

WithServeMux will configure the health check endpoint to use the provided http.ServeMux.

func WithServer

func WithServer(s *http.Server) Option

WithServer will configure the health check endpoint to use the provided server.

func WithoutContinuousProfiling

func WithoutContinuousProfiling() Option

WithoutContinuousProfiling disables the continuous profiler.

func WithoutMetrics

func WithoutMetrics() Option

WithoutMetrics disables the metrics endpoint.

func WithoutPprof

func WithoutPprof() Option

WithoutPprof disables the pprof endpoint.