keepalived-go

README

keepalived-go

A reimplementation of keepalived 2.4.3 in Go — VRRP, the LVS health checkers, and BFD.

The goal is a drop-in replacement: same configuration file, same diagnostics, same behaviour on the wire and in the kernel.

Status: not production software. Here be dragons

Why do this at all

Two reasons that hold up, and one honest cost.

Memory safety, in code that parses hostile input. The daemon reads bytes it does not control: VRRP advertisements off the wire, BFD packets, HTTP bodies and SMTP banners from health-checked backends. That is the kind of code where a bounds check in the wrong place costs you the machine, and Go removes the whole class for free. Every wire-format parser is also fuzzed for an hour a target, which found a stack overflow and an out-of-memory in this port's own configuration preprocessor.

A state machine you can actually test. internal/vrrp/fsm has no I/O of any kind — events in, a list of actions out, with the clock as a parameter. That makes the state × event cross-product and the 4×4 sync-group transition matrix exhaustively testable. They are not in the C original, where the state handlers call vrrp_send_adv, netlink and thread_add_timer directly.

The cost: Go is worse at hard deadlines, and the fix is not free. keepalived is one thread that waits for its deadline and does the work when it arrives, so a single sched_setscheduler covers the whole path. Go splits those — the advertisement is written by the runner's locked thread, but the wake-up comes from whichever M the runtime's timer happens to be on. At a 10 ms interval under CPU contention, worst inter-advert gap against a ~36 ms budget:

	keepalived	this port
no real-time priority	27.0 ms	61.5 ms
vrrp_rt_priority 50	10.7 ms	10.8 ms

Roughly twice as bad untuned. The mitigation is to promote every thread in /proc/self/task rather than one, which this does — and then the two are within a millisecond. If you run sub-100 ms intervals, set vrrp_rt_priority; the daemon warns at startup if you have not.

Compatibility

Concretely, and all of it checkable:

Configuration syntax	The same grammar, including $NAME= definitions, ~SEQ/~LST repetition, @id host conditionals and include
Accept / reject	66 recorded configurations, same verdict as keepalived 2.4.3
Diagnostics	81 messages, character for character
-t exit status	0 when accepted, 5 when rejected, as C does
Command-line flags	-f -P -C -D -n -t -V -X mean what they mean in keepalived
Wire format	VRRPv2 over IPv4, VRRPv3 over IPv4 and IPv6, byte-for-byte against 6 captured advertisements

See Migrating from keepalived for what is not the same.

Where it deliberately differs from the RFCs

The port is written against the keepalived C source, not against the RFCs, and where the two disagree it follows keepalived. An implementation that is more correct than its peers is one that cannot form a virtual router with them.

VRRPv2 here drops an advertisement whose interval differs from its own, for instance; RFC 3768 requires no such check, but keepalived does it. Every such case is listed in docs/keepalived-deltas.md with the C file and line it came from.

Migrating from keepalived

Your configuration file should not need to change. What changes is around it.

Logging. keepalived logs to syslog by default; this writes to stderr with a keepalived: prefix. Under systemd that is already what you want — the journal captures stderr — but a unit file that redirects to a log file needs adjusting, and -t --config-test=FILE has no equivalent here: -t writes its diagnostics to stderr and sets the exit status.

# keepalived.service
ExecStart=/usr/local/bin/keepalived -n -f /etc/keepalived/keepalived.conf
StandardOutput=journal
StandardError=journal

-n (don't fork) is the right mode under systemd with Type=simple, as it is for the C daemon.

Three binaries, not one. The supervisor forks keepalived-vrrp and keepalived-check, and looks for them next to itself. Install all three into the same directory. A container image needs all three in the layer.

Flags. Eight are compatible: -f -P -C -D -n -t -V -X. Check any others against --help; -v in particular is spelled -version here.

Capabilities. Same as keepalived: CAP_NET_RAW for the VRRP socket and CAP_NET_ADMIN for addresses, routes and nftables. A container needs both, or --privileged.

Subsystems that are thinner. If you poll SNMP, drive D-Bus, or rely on ha_suspend, read What is missing before switching — those three are implemented but not to C's full surface.

Quickstart

The supervisor forks two child binaries, so build all three into one directory:

go build -o bin/ ./cmd/...

That gives bin/keepalived, bin/keepalived-vrrp and bin/keepalived-check. The children are looked up next to the supervisor, so keep them together.

A minimal configuration:

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    virtual_ipaddress {
        192.168.1.100/24
    }
}

Check it, then run it:

bin/keepalived -t -f keepalived.conf     # exit 0 accepted, 5 rejected
sudo bin/keepalived -n -D -f keepalived.conf

-n keeps it in the foreground and -D turns on detailed logging. With no peer advertising a higher priority it takes the address within a few seconds:

keepalived-vrrp: (VI_1) Entering BACKUP STATE (init)
keepalived-vrrp: (VI_1) Entering MASTER STATE

It needs CAP_NET_RAW and CAP_NET_ADMIN — for a VRRP socket and for assigning the address.

No cgo. One dependency, golang.org/x/sys.

Architecture

Three binaries. keepalived is a supervisor that parses the configuration once to learn which children are needed, starts them, and then only supervises — restart a dead child, forward reload and dump signals, shut down within a bounded time. It holds no protocol state, which is what makes a crash in the VRRP child survivable. keepalived-vrrp and keepalived-check are the two children. keepalived splits the same way.

The VRRP state machine has no I/O. internal/vrrp/fsm takes an event and the current time and returns a list of actions; sockets, netlink, timers and logging all live outside it. internal/vrrp/runner is the single select loop that owns a domain, feeds the machine, and performs what comes back.

The unit of state is a domain — every instance in the process plus its sync groups — rather than a single instance, because sync-group propagation mutates sibling instances. One goroutine owns a sync group, or an instance that has no group. Nothing else touches that state, so there are no locks on the failover path.

Health checkers are one goroutine each, running their own probe-and-wait loop at that checker's delay_loop. Results funnel back through a single mutex before touching quorum arithmetic or the IPVS table, so the ordering of "backend went down" against "quorum lost" against "install the sorry server" is serialised rather than raced.

Reload rebuilds rather than mutates. A new configuration builds a whole new object graph, and instances whose configuration did not change carry their state machine and what they hold in the kernel across to it. Sockets, VMAC interfaces and the kernel parameters the daemon changed outlive a generation, because tearing them down and recreating them on reload would be a failover.

cmd/keepalived           supervisor
cmd/keepalived-vrrp      VRRP daemon
cmd/keepalived-check     health-check daemon

internal/config          the parser, byte-compatible with C's diagnostics
internal/vrrp/fsm        the state machine — pure, no I/O
internal/vrrp/runner     the loop that owns a domain and drives it
internal/vrrp/proto      wire format, checksums, HMAC auth extension
internal/vrrp/socket     raw sockets, IP_HDRINCL, multicast, GTSM
internal/vrrp/kernel     addresses, routes, rules, firewall
internal/vrrp/track      interface, script, file and process trackers
internal/vrrp/vmac       virtual MAC interfaces
internal/check           the checkers and their IPVS application
internal/bfd             BFD sessions
internal/firewall        nftables rules
internal/netlinkx        rtnetlink and generic netlink
internal/sysctl          per-interface kernel parameters

Testing

go test -race ./...          # skips what needs a capability, naming which
test/privileged/run.sh       # the rest, in a user namespace — no root needed
test/fuzz/gate.sh            # 30 fuzz targets, one hour each

The recorded corpus is in testdata/golden, so the compatibility tests run on a bare clone. KEEPALIVED_SRC points them at a keepalived checkout for re-recording.

Two tools measure how much of the configuration actually does something:

go run ./tools/parity        # settings that are read outside the parser
go run ./tools/inert         # settings no test notices being broken

What works

VRRP v2 and v3 over IPv4 and IPv6, unicast and multicast, with virtual MAC interfaces, virtual routes and rules, and accept_mode enforced through nftables.

• Sync groups, so several instances fail over together.
• Tracking — track_interface, track_script with rise/fall hysteresis, track_file, vrrp_track_process, track_bfd.
• Notifications — per-state and generic scripts, notify FIFOs, and SMTP alerts for instances, sync groups and real servers.
• LVS — ten checker types, IPVS application, quorum and sorry servers, session persistence.
• BFD sessions per RFC 5880/5881.

Each was compared against the running C daemon, not only against its source: on live interfaces for VRRP and tracking, through a recording SMTP server for the alerts, and against the kernel's IPVS table for persistence.

What is missing

Three subsystems are thinner than keepalived's:

• SNMP carries the columns a monitoring system polls, not all of C's tables.
• D-Bus PrintStats answers and returns nothing, because there are no per-instance counters yet.
• ha_suspend follows the virtual address rather than a VRRP instance's state.

Smaller gaps: the preferred_lft, track_group and use_vmac qualifiers on a virtual address are recognised and reported rather than applied; virtual_routes takes add, prepend and append but not replace; and the nftables rule for a VMAC's multicast membership reports is implemented in its drop-only form.

The daemon reports its own limits at startup, so a configuration relying on one of these says so when you run it.

Known limitations

Sub-100 ms advertisement intervals need vrrp_rt_priority. At a 10 ms interval a backup declares its master down after about 36 ms, and a contended general-purpose scheduler does not meet a 36 ms deadline — in either implementation. Untuned, this port's worst-case gap is roughly twice C's; with a real-time policy the two are within a millisecond. See Why do this at all for the numbers and the reason. The daemon warns at startup if you run short intervals without it.

No IPv6 for VRRPv2, which is not a gap: RFC 3768 has no IPv6.

Untriaged test-coverage gaps. tools/inert reports sites where breaking the code on purpose fails no test — roughly a third of the candidate sites in internal/check. Each is either dead code or code no test covers, and the tool cannot tell which. They have not been worked through.

What "production" would take

The warning at the top is not a formality, and it is not because the project is abandoned. Concretely, before it comes off:

• External review. Nobody but the author has read this code. For a daemon that parses hostile input and decides which machine owns an address, that is the largest single risk.
• A real deployment. It has never run outside a test namespace. Nothing here has met a switch that does something unexpected, a driver that reorders packets, or six months of uptime.
• Sustained load and soak testing. The measurements are minutes long. Slow leaks, fragmentation and clock drift do not show up in minutes.
• The tools/inert survivors triaged, so "tested" means the same thing everywhere in the tree.
• The three thin subsystems finished, or documented as permanently out of scope.

There is no schedule. It is a personal project.

Contributing

See CONTRIBUTING.md. The rule that is not obvious: parity with keepalived beats correctness, so a patch fixing a bug this port copies on purpose will usually be declined.

Security reports: SECURITY.md.

License

GPL-2.0-or-later — see LICENSE. Every source file carries an SPDX-License-Identifier.

This is a derivative work of keepalived, Copyright (C) 2001-2017 Alexandre Cassen acassen@gmail.com and contributors, licensed GPL-2.0-or-later. It is a reimplementation written against the keepalived source, with that source cited by file and line throughout, and is covered by the same license.

keepalived is at https://github.com/acassen/keepalived. This project is not affiliated with or endorsed by its maintainers.