Documentation
¶
Overview ¶
Package framework is the health-check framework: per-checker retry state and per-virtual-server quorum with hysteresis.
It is pure — no sockets, no clock — so the retry counting and the quorum transitions can be exercised exhaustively. The individual checker types (TCP, HTTP, DNS, ...) plug in underneath by reporting success or failure; nothing here knows how a check is performed.
Index ¶
- type Checker
- type QuorumEvent
- type QuorumKind
- type RealServer
- type Result
- type SorryAction
- type Transition
- type VirtualServer
- func (vs *VirtualServer) AdjustWeight(name string, delta int) (QuorumEvent, error)
- func (vs *VirtualServer) LiveWeight() int
- func (vs *VirtualServer) SetAlive(name string, alive bool) (QuorumEvent, error)
- func (vs *VirtualServer) SetInitialQuorum()
- func (vs *VirtualServer) Threshold() int
- func (vs *VirtualServer) UpdateQuorum() QuorumEvent
- func (vs *VirtualServer) Weight(name string) (int, bool)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Checker ¶
type Checker struct {
Name string
// RealServer is the backend this check governs.
RealServer string
// DelayLoop is the interval between checks while the state is steady.
DelayLoop time.Duration
// Retry is how many *additional* attempts a failing check gets before
// the server is declared down. Zero means the first failure is
// decisive.
Retry uint
// DelayBeforeRetry is the interval between those retries, which is
// normally shorter than DelayLoop — the point of retrying is to resolve
// the ambiguity quickly.
DelayBeforeRetry time.Duration
// Alpha starts the checker down and suppresses notification until the
// first result. Without it a server is assumed healthy until proven
// otherwise, so a daemon restart sends traffic to a dead backend for
// one check interval.
Alpha bool
// Up is the current verdict.
Up bool
// HasRun reports whether any result has been recorded, which is what
// alpha mode keys off.
HasRun bool
// contains filtered or unexported fields
}
Checker is the retry and scheduling state for one health check.
func NewChecker ¶
NewChecker returns a checker in its initial state.
In alpha mode it starts down; otherwise it starts up, which is keepalived's default and is the optimistic assumption that a configured backend works until a check says otherwise.
func (*Checker) Init ¶
func (c *Checker) Init()
Init applies alpha mode, which must happen before the first check.
func (*Checker) Record ¶
func (c *Checker) Record(r Result, now time.Time) Transition
Record applies a check result and returns the transition.
func (*Checker) Schedule ¶
Schedule sets the next run time, used to stagger the first check across backends.
func (*Checker) ShouldNotify ¶
func (c *Checker) ShouldNotify(tr Transition) bool
ShouldNotify reports whether a result warrants a real-server notification — the notify_up/notify_down script, the notify FIFO and the SNMP trap (do_rs_notifies, ipwrapper.c:203-222).
A change always does. So does the *first* result even when it changes nothing, which is the part that is easy to miss (update_svr_checker_state, ipwrapper.c:742-750):
if (checker->is_up == alive) {
if (!checker->has_run) {
if (checker->alpha || !alive)
do_rs_notifies(checker->vs, checker->rs, false);
checker->has_run = true;
}
return;
}
An alpha-mode checker starts down by assumption, so a first check that fails confirms the assumption and changes nothing — and that is exactly the case an operator needs to hear about, because the backend was already dead when the daemon started. Gating on the transition alone leaves it silent: the backend is quietly absent from the pool and nothing was sent.
The `!alive` arm reaches the same population by another route, since a non-alpha checker starts up (check_api.c:152) and only alpha mode sets one down before its first run (ipwrapper.c:999, :1193).
Not the smtp_alert, which the individual checkers gate on the transition so that the mail can carry the checker's own wording — C says so where it leaves the mail out of this function (ipwrapper.c:216-218).
type QuorumEvent ¶
type QuorumEvent struct {
Kind QuorumKind
Weight int
Threshold int
// SorryServer reports what should happen to the sorry server as a
// result: it is installed when quorum is lost and removed when it
// returns.
SorryServer SorryAction
}
QuorumEvent describes a change in service availability.
type QuorumKind ¶
type QuorumKind uint8
QuorumKind is the direction of a quorum change.
const ( QuorumNoChange QuorumKind = iota QuorumGained QuorumLost )
func (QuorumKind) String ¶
func (k QuorumKind) String() string
type RealServer ¶
type RealServer struct {
Name string
// Weight contributes to the quorum sum while the server is alive. Zero
// is meaningful: it keeps the server in the table but contributes
// nothing, which is how a server is drained without dropping its
// existing connections.
Weight int
Alive bool
// Inhibit keeps a failed server in the IPVS table at weight zero
// instead of removing it, so established connections survive.
Inhibit bool
}
RealServer is one backend behind a virtual server.
type SorryAction ¶
type SorryAction uint8
SorryAction is what to do with the sorry server.
const ( SorryNone SorryAction = iota SorryInstall SorryRemove )
type Transition ¶
type Transition struct {
From, To bool
Changed bool
// Retrying reports that a failure was recorded but the server is not
// down yet, so the caller can schedule the shorter retry interval.
Retrying bool
// Retries is the failure count after this result.
Retries uint
// First reports that this was the checker's first result. It matters
// even when nothing changed: see ShouldNotify.
First bool
}
Transition reports whether a check changed the verdict.
type VirtualServer ¶
type VirtualServer struct {
Name string
// Quorum is the weight sum required for the service to be considered
// available.
Quorum int
// Hysteresis widens the gap between the thresholds for coming up and
// going down, so a server flapping around the quorum boundary does not
// take the whole service with it.
Hysteresis int
RealServers []*RealServer
// SorryServer is installed while quorum is lost, so clients get a
// controlled response instead of a connection refused.
SorryServer *RealServer
// QuorumUp is the current state. It is not derived on demand because
// the threshold depends on it — that is what hysteresis means.
QuorumUp bool
}
VirtualServer holds the real servers and the quorum policy.
func (*VirtualServer) AdjustWeight ¶
func (vs *VirtualServer) AdjustWeight(name string, delta int) (QuorumEvent, error)
AdjustWeight applies a weight delta to a real server and recomputes quorum (`checker->rs->effective_weight + new_status - previous_status`, track_file.c:771).
A delta rather than an absolute because several trackers can drive one server: a real server may have a FILE_CHECK and a conventional checker at once, and each contributes independently. Setting an absolute would make the last writer win and silently discard the others' opinions.
func (*VirtualServer) LiveWeight ¶
func (vs *VirtualServer) LiveWeight() int
LiveWeight is the summed weight of the alive real servers.
func (*VirtualServer) SetAlive ¶
func (vs *VirtualServer) SetAlive(name string, alive bool) (QuorumEvent, error)
SetAlive marks a real server up or down and returns the resulting quorum event.
func (*VirtualServer) SetInitialQuorum ¶
func (vs *VirtualServer) SetInitialQuorum()
SetInitialQuorum establishes the starting state without reporting a transition (set_quorum_states, ipwrapper.c:505-512).
At startup the threshold is the coming-up one, because nothing is up yet.
A virtual server that starts below quorum starts with its sorry server serving, which is what C's init pass arranges (ipwrapper.c:552-553, :659). Leaving the flag false would put the two views out of step: the applier installs the sorry server and the first UpdateQuorum to regain quorum then emits no SorryRemove, because it believes there is nothing to remove — so the sorry server would stay in the table taking traffic beside a healthy pool.
func (*VirtualServer) Threshold ¶
func (vs *VirtualServer) Threshold() int
Threshold is the weight sum the service must reach to change state (update_quorum_state, keepalived/check/ipwrapper.c:521).
threshold = quorum + (up ? -1 : +1) × hysteresis
The sign flips with the current state, which is the whole mechanism: coming up needs more than the quorum, going down needs less. Computing it from the quorum alone gives a service that oscillates whenever one server sits on the boundary.
func (*VirtualServer) UpdateQuorum ¶
func (vs *VirtualServer) UpdateQuorum() QuorumEvent
UpdateQuorum recomputes the quorum state and reports any transition (update_quorum_state, ipwrapper.c:515-570).
Source Files