gopatrol

README

gopatrol

self-hosted endpoint monitoring daemon with centralized events log using MongoDB based on https://github.com/sourcegraph/checkup

Installation | Setting API Server | Setting Daemon | CLI | CLI | Slack Notifier | Email Notifier | License

gopatrol is self-hosted health checks, written in Go using checkup (instead of using them as dependency i decide to copy the file to this project) and restful api to interact with.

There is much work to do for this project to be complete. Use it carefully.

gopatrol currently supports:

• Checking HTTP endpoints
• Checking TCP endpoints (TLS supported)
• Checking of DNS services & record existence
• Storing events in MongoDB
• Add delete update checkers with dashboard/cli
• Easy to setup and deploy
• Get notified via slack and email (need help with email notifier)

Installation

• git clone
• make

NAME:
   gopatrol-cli - gopatrol cli 

USAGE:
   gopatrol-cli [global options] command [command options] [arguments...]

VERSION:
   1.0.0

AUTHOR:
   Adhitya Ramadhanus <adhitya.ramadhanus@gmail.com>

COMMANDS:
     add-http  Add endpoints to checkup
     add-tcp   Add tcp endpoints to checkup
     add-dns   Add dns endpoints to checkup
     list      list endpoint
     delete    delete endpoint
     help, h   Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version

NAME:
   gopatrol - gopatrol daemon 

USAGE:
   gopatrol [global options] command [command options] [arguments...]

VERSION:
   1.0.0

AUTHOR:
   Adhitya Ramadhanus <adhitya.ramadhanus@gmail.com>

COMMANDS:
     daemon   run gopatrol checking daemon
     api      run gopatrol api server
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version

Setting up gopatrol API Server

There's 2 different setup configuration for cli and dashboard, you can interact with the api using cli only if you run the api in unix domain socket

For these two setup you also need to set env var, i suggest using .env file to store your config

Example of .env

MONGODB_URI="mongodb://localhost:27017/gopatrol"
JWT_SECRET="Something"
ENV="Development"

1. Setup For CLI

$ gopatrol api --log=<log output, stdout, stderr or filename> --proto=unix

it will run on a socket file /tmp/gopatrol.sock

1. Setup For Dashboard

$ gopatrol api --log=<log output, stdout, stderr or filename> --proto=http --address=:3000

it will run on port 3000

Setting up gopatrol daemon

You also need to set env var for daemon to work mostly for mongodb and slack notifier, i suggest using .env file to store your config

Example of .env

MONGODB_URI="mongodb://localhost:27017/gopatrol"
SLACK_TOKEN="xoxb-something-something" //bot users token
SLACK_CHANNEL="something" // channel ID 
ENV="Development"

$ gopatrol daemon --log=<log output, stdout, stderr or filename> interval (10s, 1m, etc)

Interacting with API using CLI

1. Adding Tcp endpoint

NAME:
   gopatrol-cli add-tcp - Add tcp endpoints to checkup

USAGE:
   gopatrol-cli add-tcp [command options] name url

OPTIONS:
   --attempts value, -a value         how many times to check endpoint (default: 5)
   --thresholdrtt value, --rtt value  Threshold Rtt to define a degraded endpoint (default: 0)
   --tls-enabled                      Enable TLS connection to endpoint
   --tls-ca value                     Certificate file to established tls connection
   --tls-skip-verify                  Skip verify tls certificate
   --timeout value                    Timeout to established a tls connection (default: 3000000000)

1. Adding Http endpoint

NAME:
   gopatrol-cli add-http - Add endpoints to checkup

USAGE:
   gopatrol-cli add-http [command options] name url

OPTIONS:
   --attempts value, -a value         how many times to check endpoint (default: 5)
   --thresholdrtt value, --rtt value  Threshold Rtt to define a degraded endpoint (default: 0)
   --mustcontain value                HTML content that a page should contain to determine whether a page is up or down
   --mustnotcontain value             HTML content that a page should not contain to determine whether a page is up or down
   --headers value                    Http Headers to send along the check request
   --upstatus value                   Http status code to define a healthy page (default: 200)

1. Adding DNS endpoint

NAME:
   gopatrol-cli add-dns - Add dns endpoints to checkup

USAGE:
   gopatrol-cli add-dns [command options] name url hostname

OPTIONS:
   --tls                              Send request over tls
   --host value                       grpc server address (default: "/tmp/gopatrol.sock")
   --attempts value, -a value         how many times to check endpoint (default: 5)
   --thresholdrtt value, --rtt value  Threshold Rtt to define a degraded endpoint (default: 0)
   --timeout value                    Timeout to established a tls connection (default: 3000000000)

1. Deleting Endpoint

NAME:
   gopatrol-cli delete - delete endpoint

USAGE:
   gopatrol-cli delete [command options] slug

1. Listing Endpoint

NAME:
   gopatrol-cli list - list endpoint

USAGE:
   gopatrol-cli list [command options] [arguments...]

Interacting with API using Dashboard

Still in progress

Notifier (Slack)

• To use this notifier you need bot integration in your team and channel id where this bot will notify you, refer to this link https://api.slack.com/bot-users
• After you get the tokens and channel ID, set it in env var ot use .env file

Notifier (Email)

• NIP, not in progress, but definitely in to-do list

License

MIT © [Adhitya Ramadhanus]

Documentation

Overview

Package gopatrol provides means for checking and reporting the status and performance of various endpoints in a distributed, lock-free, self-hosted fashion.

Index

• Constants
• Variables
• type Attempt
• type Attempts
  • func (a Attempts) Len() int
  • func (a Attempts) Less(i, j int) bool
  • func (a Attempts) Swap(i, j int)
• type CacheService
• type Checker
• type CheckersService
• type Checkup
  • func (c Checkup) Check() ([]Result, error)
• type DNSChecker
  • func (c DNSChecker) Check() (Result, error)
• type Errors
  • func (e Errors) Empty() bool
  • func (e Errors) Error() string
• type Event
  • func NewEvent(result Result) Event
• type EventService
• type HTTPChecker
  • func (c HTTPChecker) Check() (Result, error)
• type Notifier
• type Result
  • func (r Result) ComputeStats() Stats
  • func (r Result) Status() string
  • func (r Result) String() string
• type SlackNotifier
  • func NewSlackNotifier(slackToken string, channelID string) *SlackNotifier
  • func (t SlackNotifier) GetType() string
  • func (t SlackNotifier) Notify(result Result) error
• type Stats
• type TCPChecker
  • func (c TCPChecker) Check() (Result, error)
• type User
• type UsersService

Constants

const (
	Healthy  = "healthy"
	Degraded = "degraded"
	Down     = "down"
	Unknown  = "unknown"
)

Text representations for the status of a check.

Variables

var DefaultConcurrentChecks = 5

DefaultConcurrentChecks is how many checks, at most, to perform concurrently.

var DefaultHTTPClient = &http.Client{
	Transport: &http.Transport{
		Proxy: http.ProxyFromEnvironment,
		Dial: (&net.Dialer{
			Timeout:   10 * time.Second,
			KeepAlive: 0,
		}).Dial,
		TLSHandshakeTimeout:   5 * time.Second,
		ExpectContinueTimeout: 1 * time.Second,
		MaxIdleConnsPerHost:   1,
		DisableCompression:    true,
		DisableKeepAlives:     true,
		ResponseHeaderTimeout: 5 * time.Second,
	},
	CheckRedirect: func(req *http.Request, via []*http.Request) error {
		return http.ErrUseLastResponse
	},
	Timeout: 10 * time.Second,
}

DefaultHTTPClient is used when no other http.Client is specified on a HTTPChecker.

Types

type Attempt

type Attempt struct {
	RTT   time.Duration `json:"rtt"`
	Error string        `json:"error,omitempty"`
}

Attempt is an attempt to communicate with the endpoint.

type Attempts

type Attempts []Attempt

Attempts is a list of Attempt that can be sorted by RTT.

func (Attempts) Len

func (a Attempts) Len() int

func (Attempts) Less

func (a Attempts) Less(i, j int) bool

func (Attempts) Swap

func (a Attempts) Swap(i, j int)

type CacheService

type CacheService interface {
	// serialize to bytes
	Set(key string, value []byte) error
	Get(key string) ([]byte, error)
}

type Checker

type Checker interface {
	Check() (Result, error)
}

Checker can create a Result.

type CheckersService

type CheckersService interface {
	InsertChecker(checker interface{}) error
	GetAllCheckers(query map[string]interface{}) ([]interface{}, error)
	GetCheckerBySlug(slug string) (interface{}, error)
	CountCheckers(query map[string]interface{}) (int, error)
	DeleteCheckerBySlug(slug string) error
	UpdateCheckerBySlug(slug string, updateData interface{}) error
}

type Checkup

type Checkup struct {
	// Checkers is the list of Checkers to use with
	// which to perform checks.
	Checkers []Checker `json:"checkers,omitempty"`

	// ConcurrentChecks is how many checks, at most, to
	// perform concurrently. Default is
	// DefaultConcurrentChecks.
	ConcurrentChecks int `json:"concurrent_checks,omitempty"`

	// Timestamp is the timestamp to force for all checks.
	// Useful if wanting to perform distributed check
	// "at the same time" even if they might actually
	// be a few milliseconds or seconds apart.
	Timestamp time.Time `json:"timestamp,omitempty"`

	// Notifier is a notifier that will be passed the
	// results after checks from all checkers have
	// completed. Notifier may evaluate and choose to
	// send a notification of potential problems.
	Notifier []Notifier `json:"notifier,omitempty"`
}

Checkup performs a routine checkup on endpoints or services.

func (Checkup) Check

func (c Checkup) Check() ([]Result, error)

Check performs the health checks. An error is only returned in the case of a misconfiguration or if any one of the Checkers returns an error.

type DNSChecker

type DNSChecker struct {
	Slug string `json:"slug" valid:"required"`
	// Name is the name of the endpoint.
	Name string `json:"name,omitempty" valid:"required"`
	// URL is the URL of the endpoint.
	URL  string `json:"url,omitempty" valid:"required"`
	Type string `json:"type,omitempty" valid:"required"`
	// ThresholdRTT is the maximum round trip time to
	// allow for a healthy endpoint. If non-zero and a
	// request takes longer than ThresholdRTT, the
	// endpoint will be considered unhealthy. Note that
	// this duration includes any in-between network
	// latency.
	ThresholdRTT time.Duration `json:"threshold_rtt,omitempty"`
	// Attempts is how many requests the client will
	// make to the endpoint in a single check.
	Attempts int    `json:"attempts,omitempty"` // This is the fqdn of the target server to query the DNS server for.
	Host     string `json:"hostname_fqdn,omitempty"`
	// Timeout is the maximum time to wait for a
	// TCP connection to be established.
	Timeout     time.Duration `json:"timeout,omitempty"`
	LastChecked time.Time     `json:"last_checked"`
	LastChange  time.Time     `json:"last_change"`
	LastStatus  string        `json:"last_status"`
}

DNSChecker implements a Checker for TCP endpoints.

func (DNSChecker) Check

func (c DNSChecker) Check() (Result, error)

Check performs checks using c according to its configuration. An error is only returned if there is a configuration error.

type Errors

type Errors []error

Errors is an error type that concatenates multiple errors.

func (Errors) Empty

func (e Errors) Empty() bool

Empty returns whether e has any non-nil errors in it.

func (Errors) Error

func (e Errors) Error() string

Error returns a string containing all the errors in e.

type Event

type Event struct {
	Slug      string    `json:"slug" bson:"slug"`
	Name      string    `json:"name" bson:"name"`
	URL       string    `json:"url" bson:"url"`
	Message   string    `json:"message" bson:"message"`
	Type      string    `json:"type" bson:"type"`
	Timestamp time.Time `json:"timestamp" bson:"timestamp"`
	Reason    string    `json:"reason" bson:"reason"`
}

Event is an event (up or down) of a checker

func NewEvent

func NewEvent(result Result) Event

NewEvent create an event from a result returned by a checker

type EventService

type EventService interface {
	InsertEvent(event Event) error
	GetAllEvents(query map[string]interface{}) ([]Event, error)
	CountEvents(query map[string]interface{}) (int, error)
}

type HTTPChecker

type HTTPChecker struct {
	Slug string `json:"slug" valid:"required"`
	// Name is the name of the endpoint.
	Name string `json:"name,omitempty" valid:"required"`
	// URL is the URL of the endpoint.
	URL  string `json:"url,omitempty" valid:"required"`
	Type string `json:"type,omitempty" valid:"required"`
	// ThresholdRTT is the maximum round trip time to
	// allow for a healthy endpoint. If non-zero and a
	// request takes longer than ThresholdRTT, the
	// endpoint will be considered unhealthy. Note that
	// this duration includes any in-between network
	// latency.
	ThresholdRTT time.Duration `json:"threshold_rtt,omitempty"`
	// Attempts is how many requests the client will
	// make to the endpoint in a single check.
	Attempts int `json:"attempts,omitempty"`
	// UpStatus is the HTTP status code expected by
	// a healthy endpoint. Default is http.StatusOK.
	UpStatus int `json:"up_status,omitempty"`
	// MustContain is a string that the response body
	// must contain in order to be considered up.
	// NOTE: If set, the entire response body will
	// be consumed, which has the potential of using
	// lots of memory and slowing down checks if the
	// response body is large.
	MustContain string `json:"must_contain,omitempty"`
	// MustNotContain is a string that the response
	// body must NOT contain in order to be considered
	// up. If both MustContain and MustNotContain are
	// set, they are and-ed together. NOTE: If set,
	// the entire response body will be consumed, which
	// has the potential of using lots of memory and
	// slowing down checks if the response body is large.
	MustNotContain string `json:"must_not_contain,omitempty"`
	// Client is the http.Client with which to make
	// requests. If not set, DefaultHTTPClient is
	// used.
	Client *http.Client `json:"-" bson:"-"`
	// Headers contains headers to added to the request
	// that is sent for the check
	Headers http.Header `json:"headers,omitempty"`

	LastChecked time.Time `json:"last_checked"`
	LastChange  time.Time `json:"last_change"`
	LastStatus  string    `json:"last_status"`
}

HTTPChecker implements a Checker for HTTP endpoints.

func (HTTPChecker) Check

func (c HTTPChecker) Check() (Result, error)

Check performs checks using c according to its configuration. An error is only returned if there is a configuration error.

type Notifier

type Notifier interface {
	Notify(Result) error
}

Notifier can notify ops or sysadmins of potential problems. A Notifier should keep state to avoid sending repeated notices more often than the admin would like.

type Result

type Result struct {
	Slug string
	// Title is the title (or name) of the thing that was checked.
	// It should be unique, as it acts like an identifier to users.
	Name string
	// Endpoint is the URL/address/path/identifier/locator/whatever
	// of what was checked.
	URL string
	// Timestamp is when the check occurred; UTC UnixNano format.
	Timestamp time.Time
	// Times is a list of each individual check attempt.
	Times Attempts
	// ThresholdRTT is the maximum RTT that was tolerated before
	// considering performance to be degraded. Leave 0 if irrelevant.
	ThresholdRTT time.Duration
	// Healthy, Degraded, and Down contain the ultimate conclusion
	// about the endpoint. Exactly one of these should be true;
	// any more or less is a bug.
	Healthy  bool
	Degraded bool
	Down     bool
	// Notice contains a description of some condition of this
	// check that might have affected the result in some way.
	// For example, that the median RTT is above the threshold.
	Notice string
	// Message is an optional message to show on the status page.
	Message string
	// Flag to determine whether this result is an event or notification
	Event        bool
	Notification bool
}

Result is the result of a health check.

func (Result) ComputeStats

func (r Result) ComputeStats() Stats

ComputeStats computes basic statistics about r.

func (Result) Status

func (r Result) Status() string

Status returns a text representation of the overall status indicated in r.

func (Result) String

func (r Result) String() string

String returns a human-readable rendering of r.

type SlackNotifier

type SlackNotifier struct {
	ID         string        `json:"-"`
	RTM        *slack.RTM    `json:"-"`
	SlackAPI   *slack.Client `json:"-"`
	ChannelID  string        `json:"channel" validate:"required"`
	SlackToken string        `json:"token" validate:"required"`
	Type       string        `json:"type" validate:"required"`
}

SlackNotifier is the main struct consist of all the sub component including slack api, real-time messaing api and face detector

func NewSlackNotifier

func NewSlackNotifier(slackToken string, channelID string) *SlackNotifier

NewSlackNotifier create new Thug bot

func (SlackNotifier) GetType

func (t SlackNotifier) GetType() string

GetType return what type of notifier is thiss

func (SlackNotifier) Notify

func (t SlackNotifier) Notify(result Result) error

Notify implements notifier interface, send slack message about an event

type Stats

type Stats struct {
	Total  time.Duration `json:"total,omitempty"`
	Mean   time.Duration `json:"mean,omitempty"`
	Median time.Duration `json:"median,omitempty"`
	Min    time.Duration `json:"min,omitempty"`
	Max    time.Duration `json:"max,omitempty"`
}

Stats is a type that holds information about a Result, especially its various Attempts.

type TCPChecker

type TCPChecker struct {
	Slug string `json:"slug" valid:"required"`
	// Name is the name of the endpoint.
	Name string `json:"name,omitempty" valid:"required"`
	// URL is the URL of the endpoint.
	URL  string `json:"url,omitempty" valid:"required"`
	Type string `json:"type,omitempty" valid:"required"`
	// ThresholdRTT is the maximum round trip time to
	// allow for a healthy endpoint. If non-zero and a
	// request takes longer than ThresholdRTT, the
	// endpoint will be considered unhealthy. Note that
	// this duration includes any in-between network
	// latency.
	ThresholdRTT time.Duration `json:"threshold_rtt,omitempty"`
	// Attempts is how many requests the client will
	// make to the endpoint in a single check.
	Attempts int `json:"attempts,omitempty"`
	// TLSEnabled controls whether to enable TLS or not.
	// If set, TLS is enabled.
	TLSEnabled bool `json:"tls,omitempty"`
	// TLSSkipVerify controls whether to skip server TLS
	// certificat validation or not.
	TLSSkipVerify bool `json:"tls_skip_verify,omitempty"`
	// TLSCAFile is the Certificate Authority used
	// to validate the server TLS certificate.
	TLSCAFile string `json:"tls_ca_file,omitempty"`
	// Timeout is the maximum time to wait for a
	// TCP connection to be established.
	Timeout     time.Duration `json:"timeout,omitempty"`
	LastChecked time.Time     `json:"last_checked"`
	LastChange  time.Time     `json:"last_change"`
	LastStatus  string        `json:"last_status"`
}

TCPChecker implements a Checker for TCP endpoints.

func (TCPChecker) Check

func (c TCPChecker) Check() (Result, error)

Check performs checks using c according to its configuration. An error is only returned if there is a configuration error.

type User

type User struct {
	Name         string `json:"name" bson:"name" valid:"required,email"`
	Email        string `json:"email" bson:"email" valid:"required,email"`
	HashPassword string `json:"-" bson:"hashpassword"`
	Role         string `json:"role" bson:"role"`
}

type UsersService

type UsersService interface {
	Register(name, email, role, plainPassword string) error
	// UpdateUser(email string, updateData interface{}) error
	Login(email, plainPassword string) (User, error)
}