procjon

README

procjon

Procjon is simple monitoring tool written in Go. It is a deamon which sends updates to Slack webhook when status of monitored service changes. Service status is sent to procjon by procjonagent. Service is abstracted by Servicer interface. It can be for example host itself, systemd unit or elasticsearch node.

How does procjon work?

• Service must be registered first. Service identifier, service timeout and array of human redable statuses are sent.
• Then, service is periodically pinged - only identifier and status index is sent.
• After first ping, procjon sends service availability update to Slack, indicating that service is available.
• When status sent in ping changes, procjon sends service status update to Slack.
• When status sent in first ping is non-zero, procjon sends service status update to Slack.

Important operating principle of procjon is that reliability of monitoring server is higher that infrastructure to monitor. Procjon was designed to deal with unreliable internal infrastructure and many processes which were set-up and then left forgotten (e.g. long term tests). HA is not yet planned.

Running procjon

To run procjon, certificates must be generated. See script used for testing purposes. Script will generate certificates to .certs directory. Having certificates, get binaries from releases or install it using go:

$ go install github.com/PiotrKozimor/procjon/procjon
$ go install github.com/PiotrKozimor/procjon/procjonagent

Then run procjon (assuming certificates in .certs directory):

$ ./procjon -l debug

and procjonagent:

$ ./procjonagent ping -l debug

When you stop procjonagent, procjon will sent message to Slack after 10s (default timeout) that service foo is not available.

Procjonagent

Three procjonagents are implemented: ping, elasticsearch and systemd. Each agent is subcommand in procjonagent.

Each agent implements Servicer interface. Own agent can be easily implemented and integrated into existing application. Please see example.

Documentation

Index

• func LoadService(db *badger.DB, service *pb.Service, status *pb.ServiceStatus) error
• func MustConnectOnBuffer(sender Sender) *grpc.ClientConn
• func SaveService(db *badger.DB, service *pb.Service) error
• type Availability
  • func NewAvailability(timeout time.Duration, callback func(bool)) *Availability
  • func (a *Availability) Ping()
  • func (a *Availability) Run()
• type Sender
• type Server
  • func (s *Server) RegisterService(ctx context.Context, service *pb.Service) (*pb.Empty, error)
  • func (s *Server) SendServiceStatus(stream pb.Procjon_SendServiceStatusServer) error
• type StatusCode
  • func (stc *StatusCode) HasChanged(new uint32) (changed bool)

Functions

func LoadService

func LoadService(db *badger.DB, service *pb.Service, status *pb.ServiceStatus) error

LoadService from badger KV store. Service is uniquely identified by pb.ServiceStatus.Identifier.

func MustConnectOnBuffer

func MustConnectOnBuffer(sender Sender) *grpc.ClientConn

func SaveService

func SaveService(db *badger.DB, service *pb.Service) error

SaveService to badger KV store using protobuf. If service with given pb.Service.Identifier exists it will be overwritten.

Types

type Availability

type Availability struct {
	// contains filtered or unexported fields
}

Availability triggers action with respect to provided timeout and Ping calls.

func NewAvailability

func NewAvailability(timeout time.Duration, callback func(bool)) *Availability

NewAvailability creates new Availability struct with provided timeout and callback. Use av.Run() function to start detecting availability changes. Call av.Ping() to renew timeout. Callback will be called after first Ping() call. We assume initial inavailability. New Timer is created, so callback will be called unless Availability is pinged in prvided timeout.

func (*Availability) Ping

func (a *Availability) Ping()

Ping to renew timeout. Must call Run() before in seperate goroutine.

func (*Availability) Run

func (a *Availability) Run()

Run will detect availability changes. Should be run in seperate goroutine. Callbacks are called in seperate goroutine. Callback will be called with false when timeout has expired since last Ping

type Sender

type Sender interface {
	SendAvailability(service string, availability bool) error
	SendStatus(service string, status string) error
}

Sender is used to send availability and status (e.g. to Slack) when change is detected.

type Server

type Server struct {
	pb.UnimplementedProcjonServer
	Sender Sender
	DB     *badger.DB
}

func (*Server) RegisterService

func (s *Server) RegisterService(ctx context.Context, service *pb.Service) (*pb.Empty, error)

RegisterService in procjon and save it in badger KV store. This registration will persist across procjon restarts.

func (*Server) SendServiceStatus

func (s *Server) SendServiceStatus(stream pb.Procjon_SendServiceStatusServer) error

SendServiceStatus. Service must be registered first with RegisterService.

type StatusCode

type StatusCode struct {
	// contains filtered or unexported fields
}

StatusCode just holds the value provided in last HasChanged() call.

func (*StatusCode) HasChanged

func (stc *StatusCode) HasChanged(new uint32) (changed bool)

HasChanged returns true if new is different than value from previous call. Otherwise, it returns false.