notify

Documentation

Overview

Package notify runs the scripts and writes the FIFO lines that tell the rest of the system a VRRP instance or sync group changed state.

It is the seam between a routing decision and everything an operator built on top of it — the script that reconfigures an application, the FIFO a monitoring agent reads, the alert that pages someone. keepalived treats all of it as best-effort and so does this: by the time a notification is sent the state change has already happened on the wire. A notification that fails must not unwind it, because there is nothing to unwind to.

Index

• Constants
• Variables
• func BuildMessage(cfg SMTPConfig, a Alert, now time.Time) string
• func OpenFIFO(path string) (*os.File, error)
• func SendAlert(ctx context.Context, cfg SMTPConfig, a Alert) error
• type Alert
  • func (a Alert) SubjectLine() string
• type AlertKind
• type Alerter
  • func NewAlerter(cfg SMTPConfig, logf func(string, ...any)) *Alerter
  • func (a *Alerter) Close()
  • func (a *Alerter) Dropped() int
  • func (a *Alerter) Send(al Alert)
  • func (a *Alerter) SetSendForTest(f func(context.Context, SMTPConfig, Alert) error)
• type Config
• type Event
  • func (e Event) FIFOLine() string
  • func (e Event) ScriptArgs() []string
• type Kind
• type Notifier
  • func New(cfg Config) *Notifier
  • func (n *Notifier) Close() error
  • func (n *Notifier) Notify(e Event, specific, generic *Script)
  • func (n *Notifier) NotifyFIFO(e Event)
• type SMTPConfig
  • func NewSMTPConfig(server, port, from string, to []string, helo string, timeout time.Duration) SMTPConfig
• type Script
• type State

Constants

const (
	MaxSubject = 256
	MaxBody    = 512
)

Message-length bounds (MAX_HEADERS_LENGTH and MAX_BODY_LENGTH, smtp.h).

const DefaultHelo = "localhost"

DefaultHelo is the name sent when none is configured.

Variables

var ErrNotConfigured = errors.New("notify: smtp alerts are not configured")

ErrNotConfigured reports that alerts are not set up.

C returns early when there is no server or no recipient (smtp.c:604), and it is not an error: not configuring alerts is the common case and must not produce a log line every time something changes state.

Functions

func BuildMessage

func BuildMessage(cfg SMTPConfig, a Alert, now time.Time) string

BuildMessage renders the DATA payload (smtp.c:450-482).

The trailing "\r\n.\r\n" is the SMTP end-of-data marker and is what makes the server accept the message. It is written here rather than by the caller because forgetting it produces a connection that hangs rather than an error.

func OpenFIFO

func OpenFIFO(path string) (*os.File, error)

OpenFIFO opens a notify FIFO for writing without blocking.

O_NONBLOCK on a write end whose reader has not opened yet gives ENXIO rather than blocking, and that is the behaviour wanted: a router must start whether or not the monitoring agent is up. The caller retries or gives up; it does not wait.

func SendAlert

func SendAlert(ctx context.Context, cfg SMTPConfig, a Alert) error

SendAlert delivers one alert.

The whole exchange is one connection and it is not reused. keepalived opens a connection per alert, which is wasteful and is also what makes an alert about a failing router independent of every alert before it.

Types

type Alert

type Alert struct {
	Kind AlertKind
	// RouterID prefixes every subject, so mail from several routers can be
	// told apart in one inbox.
	RouterID string
	// Name is the instance, group or virtual server.
	Name string
	// RealServer is the backend, for AlertRealServer.
	RealServer string
	// Subject is the event description appended after the identity.
	Subject string
	Body    string
}

Alert is one message to send.

func (Alert) SubjectLine

func (a Alert) SubjectLine() string

SubjectLine renders the subject the way C does (smtp_alert, smtp.c:590-660).

Every shape starts with the router id in brackets. That is what makes the alerts sortable in a mailbox that receives them from a whole fleet, and it is why the id is worth setting even though nothing else uses it.

type AlertKind

type AlertKind uint8

AlertKind selects the subject's shape.

const (
	// AlertVRRP is a VRRP instance changing state.
	AlertVRRP AlertKind = iota
	// AlertGroup is a sync group changing state.
	AlertGroup
	// AlertRealServer is a backend going up or down.
	AlertRealServer
	// AlertVirtualServer is a quorum change.
	AlertVirtualServer
	// AlertGeneral is anything else.
	AlertGeneral
)

type Alerter

type Alerter struct {
	// contains filtered or unexported fields
}

Alerter sends alerts without making the caller wait.

Why this is not a direct call

SendAlert dials a mail server and runs an SMTP exchange, which takes as long as the network says it does — up to the timeout, which defaults to thirty seconds. The callers are state-machine transitions. A router that took thirty seconds to finish entering MASTER state because a mail server was unreachable would be a router whose failover is gated on its monitoring, which is exactly backwards: the alert exists to report the failover, not to participate in it.

So the queue is bounded and a full one drops. That is the same discipline the notify FIFOs use and for the same reason — a slow consumer must not be able to stop a router failing over. Dropped alerts are counted so the condition is visible rather than silent.

func NewAlerter

func NewAlerter(cfg SMTPConfig, logf func(string, ...any)) *Alerter

NewAlerter returns an alerter, or nil if alerts are not configured.

Nil is a usable value: every method tolerates it, so a caller does not have to test whether alerting is on before reporting something.

func (*Alerter) Close

func (a *Alerter) Close()

Close stops the sender and waits for the alert in flight.

The queued ones are delivered first: the last alert before a shutdown is the one saying the router is stopping, and dropping it defeats the point.

func (*Alerter) Dropped

func (a *Alerter) Dropped() int

Dropped reports how many alerts were discarded for a full queue.

func (*Alerter) Send

func (a *Alerter) Send(al Alert)

Send queues an alert. It never blocks.

func (*Alerter) SetSendForTest

func (a *Alerter) SetSendForTest(f func(context.Context, SMTPConfig, Alert) error)

SetSendForTest substitutes the delivery function.

Exported because the gating that decides *whether* to alert lives in another package, and testing it against a real mail server would make a test of keepalived's logic depend on the network.

It must be called before the first Send. The sender goroutine reads the field only after receiving from the queue, so a write that precedes every Send is ordered before every read by the channel; one that does not is a data race.

type Config

type Config struct {
	// FIFOs are open notify FIFOs. Both the global and the VRRP-specific
	// one may be configured and both receive every line
	// (vrrp_notify.c:138-144).
	FIFOs []*os.File

	// Generic is the script run for every notification, receiving the four
	// positional arguments. Nil disables it.
	Generic *Script

	// StopTimeout bounds a STOP notification. C waits for one
	// (`system_call_script(..., TIMER_HZ, ...)`, vrrp_notify.c:188) rather
	// than firing and forgetting, because the process is about to exit and
	// an un-waited child would be orphaned mid-run. Every other state is
	// fire-and-forget.
	StopTimeout time.Duration

	// Logf receives failures. Notifications are best-effort, so failures are
	// logged and never returned.
	Logf func(format string, args ...any)
}

Config is the notifier's configuration.

type Event

type Event struct {
	Kind     Kind
	Name     string
	State    State
	Priority uint8
}

Event is one notification.

func (Event) FIFOLine

func (e Event) FIFOLine() string

FIFOLine renders the event as the line written to a notify FIFO (notify_fifo, vrrp_notify.c:136):

{GROUP|INSTANCE} "NAME" {MASTER|BACKUP|FAULT|STOP|DELETED} PRIO

The name is quoted and the others are not, which is not tidy but is what every existing FIFO consumer parses. The priority is always zero for a group, because a group has no priority of its own — the field exists so a consumer can use one parser for both.

func (Event) ScriptArgs

func (e Event) ScriptArgs() []string

ScriptArgs returns the four arguments appended to a generic notify script (notify_script_exec, vrrp_notify.c:162-192):

<script> {GROUP|INSTANCE} NAME {MASTER|BACKUP|...} PRIO

Note the order: type, name, state, priority. The FIFO line puts the name in quotes and the arguments do not, because argv needs no quoting and the FIFO is a text stream where a name with a space would otherwise be unparseable.

type Kind

type Kind string

Kind distinguishes an instance notification from a group one.

const (
	KindInstance Kind = "INSTANCE"
	KindGroup    Kind = "GROUP"
)

type Notifier

type Notifier struct {
	// contains filtered or unexported fields
}

Notifier runs notifications.

func New

func New(cfg Config) *Notifier

New returns a notifier.

func (*Notifier) Close

func (n *Notifier) Close() error

Close waits for outstanding fire-and-forget scripts.

func (*Notifier) Notify

func (n *Notifier) Notify(e Event, specific, generic *Script)

Notify sends one event: the per-instance script, the generic script, and every FIFO.

The order is C's — specific script, generic script, FIFO (vrrp_notify.c:294-308) — and it is observable: a consumer that watches both a script and the FIFO sees the script's side effects first.

func (*Notifier) NotifyFIFO

func (n *Notifier) NotifyFIFO(e Event)

NotifyFIFO writes the FIFO line without running any script.

The pseudo-events go this way. A priority change is not a state change, and the scripts an operator configured are keyed on state — running notify_master because the priority moved would tell it something that did not happen. C makes the same split: send_instance_priority_notifies calls notify_fifo alone (vrrp_notify.c:346-353), while send_instance_notifies runs the scripts too.

type SMTPConfig

type SMTPConfig struct {
	// Server is the mail server's address and port.
	Server string
	// From is the envelope and header sender.
	From string
	// To are the recipients.
	To []string
	// HeloName is sent in the greeting.
	HeloName string
	// Timeout bounds the whole exchange.
	Timeout time.Duration
}

SMTPConfig is where and how to send.

func NewSMTPConfig

func NewSMTPConfig(server, port, from string, to []string, helo string, timeout time.Duration) SMTPConfig

NewSMTPConfig assembles the settings from the pieces global_defs holds them in.

The joining rule is the only part with a decision in it and both daemons need the same answer, so it lives here rather than once in each: they must reach the same mail server, and two copies of this would eventually disagree about a default port or an IPv6 literal.

type Script

type Script struct {
	Path string
	Args []string
	// UID and GID are the credentials to drop to.
	UID, GID       uint32
	SetCredentials bool
}

Script is a configured notify script.

type State

type State string

State is the state name a notification carries. The spellings are C's (vrrp_notify.c:175-181) because scripts match on them.

const (
	StateMaster  State = "MASTER"
	StateBackup  State = "BACKUP"
	StateFault   State = "FAULT"
	StateStop    State = "STOP"
	StateDeleted State = "DELETED"
	// StateUnknown is C's fallback, spelled with the braces it uses.
	StateUnknown State = "{UNKNOWN}"

	// The pseudo-events. They travel the same FIFO as a state change and
	// occupy the same field, because a consumer should need one parser
	// rather than two — but nothing has *entered* these states, and no
	// per-state script exists for them.
	StateMasterRxLowerPri State = "MASTER_RX_LOWER_PRI"
	StateMasterPriority   State = "MASTER_PRIORITY"
	StateBackupPriority   State = "BACKUP_PRIORITY"
)