health

README

Health

A simple and flexible health check library for Go.

Documentation · Report Bug · Request Feature

Features

This library allows you to build health checks that do not simply return 200 HTTP status codes but actually check if the systems that your app requires to work are available.

This library provides the following features:

• Request based and fixed-schedule health checks.
• Request and check-based timeout management.
• Caching support to unburden checked systems during load peeks.
• Custom HTTP request middleware to pre- and postprocess HTTP requests and/or responses.
• Authentication middleware allows separating public and private health check information.
• Provides an http.Handler that can be easily used with any mux.
• Failure tolerance based on fail count and/or time thresholds.

Example

package main

import (
	"context"
	"database/sql"
	"github.com/alexliesenfeld/health"
	_ "github.com/mattn/go-sqlite3"
	"net/http"
	"time"
)

func main() {
	db, _ := sql.Open("sqlite3", "simple.sqlite")
	defer db.Close()

	router := http.NewServeMux()
	
	// Create a new new http.Handler that provides health check information.
	router.Handle("/health", health.NewHandler(
		
		// Configure a globally configured timeout that will be applied to all checks.
		health.WithTimeout(10*time.Second),
		
		// Our health endpoint requires authentication.
		health.WithBasicAuth("username", "password", true),
		
		// A simple check to see if database connection is up.
		health.WithCheck(health.Check{                          
			Name:  "database",
			Check: db.PingContext,
		}),
		
		// The following check will be executed periodically every 30 seconds.
		health.WithPeriodicCheck(30*time.Second, health.Check{  
			Name: "search",
			Timeout: 5*time.Second, // A check specific timeout.
			Check: func(ctx context.Context) error {
				_, err := http.Get("https://www.google.com")
				return err
			},
		}),
	))

	http.ListenAndServe(":3000", router)
}

The request curl -u username:password http://localhost:3000/health would then yield a response with HTTP status code 503 (Service Unavailable), and the following JSON response body:

{
   "status": "DOWN",
   "timestamp": "2021-07-01T08:05:08.522685Z",
   "details":{
      "database": {
         "status": "DOWN",
         "timestamp": "2021-07-01T08:05:14.603364Z",
         "error": "check timed out"
      },
      "search": {
         "status": "UP",
         "timestamp": "2021-07-01T08:05:08.522685Z"
      }
   }
}

Caching

Health responses are cached to avoid burdening the services that your program checks and to mitigate "denial of service" attacks. Caching can be configured globally and/or be fine-tuned per check. If you do not want to use caching altogether, you can disable it using the health.WithDisabledCache() configuration option.

Security

The data that is returned as part of health check results usually contains sensitive information (such as service names, error messages, etc.). You probably do not want to expose this information to everyone. For this reason, this library provides support for authentication middleware that allows you to hide health details or entirely block requests based on authentication success.

Example

Based on the example below, the authentication middleware will respond with a JSON response body that only contains the health status, and the corresponding HTTP status code (in this case HTTP status code 503 (Service Unavailable) and JSON response body { "status":"DOWN" }).

health.NewHandler(
	health.WithBasicAuth("username", "password", true), 
	health.WithCustomAuth(true, func(r *http.Request) error {
		return fmt.Errorf("this simulates authentication failure")
	}), 
	health.WithCheck(health.Check{
		Name:    "database",
		Check: db.PingContext,
	}), 
)

Details, such as error messages, services names, etc. are not exposed to the caller. This allows you to open health endpoints to the public but only provide details to authenticated sources.

Periodic Checks

Rather than executing a health check function on every request that is received over the health endpoint, periodic checks execute the check function on a fixed schedule. This allows to respond to HTTP requests instantly without waiting for the check function to complete. This is especially useful if you either expect a higher request rate on the health endpoint, or your checks take a relatively long time to complete.

health.NewHandler(
	health.WithPeriodicCheck(15*time.Second, health.Check{
		Name:    "slow-check",
		Check:   myLongRunningCheckFunc, // your custom long running check function
	}),
)

Failure Tolerant Checks

This library lets you configure failure tolerant checks that allow some degree of failure. The check is only considered failed, when tolerance thresholds have been crossed.

Example

Let's assume that your app provides a REST API but also consumes messages from a Kafka topic. If the connection to Kafka is down, your app can still serve API requests, but it will not process any messages during this time. If the Kafka health check is configured without any failure tolerance, and the connection to Kafka is temporarily down, your whole application will look unhealthy in health checks. This is most likely not what you want. However, if Kafka is down for too long, there may indeed be a problem that requires attention. In this case, you still may want to flag your app unhealthy by returning a failing health check, so that it can be automatically restarted by your infrastructure.

To allow some degree of failure tolerance, please have a look at the FailureTolerance and FailureToleranceThreshold attributes in your check configuration.

License

health is free software: you can redistribute it and/or modify it under the terms of the MIT Public License.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the MIT Public License for more details.

Documentation

Index

• func NewHandler(options ...option) http.Handler
• func StartPeriodicChecks(healthHandler http.Handler)
• func StopPeriodicChecks(healthHandler http.Handler)
• func WithBasicAuth(username string, password string, sendStatusOnAuthFailure bool) option
• func WithCacheDuration(duration time.Duration) option
• func WithCheck(check Check) option
• func WithCustomAuth(sendStatusOnAuthFailure bool, authFunc func(r *http.Request) error) option
• func WithDisabledCache() option
• func WithManualPeriodicCheckStart() option
• func WithMaxErrorMessageLength(length uint) option
• func WithMiddleware(mw Middleware) option
• func WithPeriodicCheck(refreshPeriod time.Duration, check Check) option
• func WithTimeout(timeout time.Duration) option
• type Check
• type Middleware

Functions

func NewHandler

func NewHandler(options ...option) http.Handler

NewHandler creates a new health check http.Handler. If periodic checks have been configured (see WithPeriodicCheck), they will be started as well (if not explicitly turned off using WithManualPeriodicCheckStart).

func StartPeriodicChecks

func StartPeriodicChecks(healthHandler http.Handler)

StartPeriodicChecks allows to start periodic checks manually if the health check was configured using WithManualPeriodicCheckStart or when checks have been stopped earlier using health.StopPeriodicChecks. This function has no effect otherwise.

func StopPeriodicChecks

func StopPeriodicChecks(healthHandler http.Handler)

StopPeriodicChecks stops all periodic checks. This function has only effect after automatic startup (i.e. when the handler was not configured using WithManualPeriodicCheckStart) or when peridic checks have been started before manually using StartPeriodicChecks. This function will have no effect otherwise. It is usually not necessary to call this function manually. Attention: This function does not block until all checks have been stopped!

func WithBasicAuth

func WithBasicAuth(username string, password string, sendStatusOnAuthFailure bool) option

WithBasicAuth adds a basic authentication middleware. Parameter sendStatusOnAuthFailure=true allows to pass the HTTP status code to the user even if authentication was not successful. This is useful to allow publicly available health checks that do not expose system details. Example: If authentication is not successful and sendStatusOnAuthFailure=true, then HTTP status code 200 (OK) will be returned in case the service is up or 503 (Service Unavailable) if the service is considered down. Additionally, the response body will only contain an aggregated status but without any details (e.g. { "status" : "UP" }). In case sendStatusOnAuthFailure=false and authentication fails, then an empty response body will be returned along with HTTP status code 401 (Unauthorized).

func WithCacheDuration

func WithCacheDuration(duration time.Duration) option

WithCacheDuration sets the duration for how long the aggregated health check result will be cached. This is set to 1 second by default. Caching will prevent that each incoming HTTP request triggers a new health check. A duration of 0 will effectively disable the cache and has the same effect as WithDisabledCache.

func WithCheck

func WithCheck(check Check) option

WithCheck adds a new health check that contributes to the overall service availability status. This check will be triggered each time the health check HTTP endpoint is called (and the cache has expired, see WithCacheDuration). If health checks are expensive or you expect a lot of calls to the health endpoint, consider using WithPeriodicCheck instead.

func WithCustomAuth

func WithCustomAuth(sendStatusOnAuthFailure bool, authFunc func(r *http.Request) error) option

WithCustomAuth adds a custom authentication middleware so you can use you own authentication functionality to integrate with this library. Parameter sendStatusOnAuthFailure=true allows to return the HTTP status code and the aggregated overall status without the details even if authentication was not successful. This is useful to allow publicly available health checks that do not expose system details. Example: If authentication is not successful and sendStatusOnAuthFailure=true, then HTTP status code 200 (OK) will be returned in case the service is up or 503 (Service Unavailable) if the service is considered down. Additionally, the response body will only contain an aggregated status but without any details (e.g. { "status" : "UP" }). In case sendStatusOnAuthFailure=false and authentication fails, then an empty response body will be returned along with HTTP status code 401 (Unauthorized).

func WithDisabledCache

func WithDisabledCache() option

WithDisabledCache disabled the check cache. This is not recommended in most cases. This will effectively lead to a health endpoint that initiates a new health check for each incoming HTTP request. This may have an impact on the systems that are being checked (especially if health checks are expensive). Caching also mitigates "denial of service" attacks.

func WithManualPeriodicCheckStart

func WithManualPeriodicCheckStart() option

WithManualPeriodicCheckStart prevents an automatic start of periodic checks (see NewHandler). If this configuation option is used and you want to start periodic checks yourself, you need to start them by using StartPeriodicChecks.

func WithMaxErrorMessageLength

func WithMaxErrorMessageLength(length uint) option

WithMaxErrorMessageLength limits maximum number of characters in error messages.

func WithMiddleware

func WithMiddleware(mw Middleware) option

WithMiddleware allows to add a Middleware to the processing chain of HTTP requests. Middleware is a wrapper for HTTP handlers. This allows to pre- and postprocess HTTP requests/responses before and/or after running the health checks.

func WithPeriodicCheck

func WithPeriodicCheck(refreshPeriod time.Duration, check Check) option

WithPeriodicCheck adds a new health check that contributes to the overall service availability status. The health check will be performed on a fixed schedule and will not be executed for each HTTP request (as in contrast to WithCheck). This allows to process a much higher number of HTTP requests without actually calling the checked services too often or to execute long running checks. The health endpoint always returns the last result of the periodic check. When periodic checks are started (happens automatically if WithManualPeriodicCheckStart is not used) they are also executed for the first time. Until all periodic checks have not been executed at least once, the overall availability status will be "UNKNOWN" with HTTP status code 503 (Service Unavailable).

func WithTimeout

func WithTimeout(timeout time.Duration) option

WithTimeout globally defines a timeout duration for all checks. You can still override this timeout by using the timeout value in the Check configuration. Default value is 30 seconds.

Types

type Check

type Check struct {
	// The Name must be unique among all checks. Name is a required attribute.
	Name string
	// Check is the check function that will be executed to check availability.
	// This function must return an error if the checked service is considered
	// not available. Check is a required attribute.
	Check func(ctx context.Context) error
	// Timeout will override the global timeout value, if it is smaller than
	// the global timeout (see WithTimeout).
	Timeout time.Duration
	// FailureTolerance will set a duration for how long a service must be
	// "in error" until it is considered unavailable.
	FailureTolerance time.Duration
	// FailureToleranceThreshold will set a maximum number of consecutive
	// check fails until the service is considered unavailable.
	FailureToleranceThreshold uint
	// contains filtered or unexported fields
}

Check allows to configure health checks.

type Middleware

type Middleware func(next http.Handler) http.Handler

Middleware allows to define a wrapper function for HTTP handlers. This allows to pre- and postprocess HTTP requests/responses before and/or after running health checks.