Documentation
¶
Overview ¶
Package keyload reloads TLS certificates, keys and CA bundles while a server or client keeps running, so certificate rotation never needs a restart.
A Reloader loads TLS material from a Source (usually Files), validates it and keeps it in an immutable Snapshot. When the files change, it validates the new material and swaps it in atomically. If the new material is bad (an expired certificate, a key that doesn't match, a half-written file), the last good snapshot stays in service and the failure is reported through OnReload and the metrics.
r, err := keyload.New(keyload.Files{
Cert: "/tls/tls.crt", Key: "/tls/tls.key",
Roots: "/tls/ca.crt", ClientCAs: "/tls/ca.crt",
})
// Servers: certificate and client CAs follow reloads.
srv := &http.Server{TLSConfig: r.ServerConfig(nil)}
// Clients: every connection uses the current roots and certificate.
tr := &http.Transport{}
err = r.ConfigureTransport(tr, nil)
The grpccreds module does the same for grpc-go, and Reloader.ClientConfig covers libraries that only accept a *tls.Config. Every mode except ClientConfig leaves certificate verification to crypto/tls.
Example ¶
package main
import (
"context"
"errors"
"fmt"
"log"
"os"
"github.com/rossedman/keyload"
"github.com/rossedman/keyload/internal/testpki"
)
// setup writes tls.crt, tls.key and ca.crt for localhost to a temporary
// directory, the way cert-manager writes a Secret.
func setup() *testpki.Dir {
dir, err := os.MkdirTemp("", "keyload-example")
if err != nil {
log.Fatal(err)
}
return testpki.NewDir(testpki.NoTest, dir)
}
func main() {
dir := setup()
defer os.RemoveAll(dir.Path)
r, err := keyload.New(keyload.Files{
Cert: dir.File("tls.crt"),
Key: dir.File("tls.key"),
Roots: dir.File("ca.crt"),
})
if err != nil {
log.Fatal(err) // the first load must be valid
}
defer r.Close()
fmt.Println("serving generation", r.Current().Generation)
// cert-manager writes a new certificate. The file watcher picks it up
// within moments; Reload just makes it immediate.
dir.Rotate(testpki.NoTest, testpki.LeafOptions{})
_ = r.Reload(context.Background())
fmt.Println("serving generation", r.Current().Generation)
// Someone writes a broken key. It is rejected, and the last good
// certificate keeps serving.
testpki.WriteFile(testpki.NoTest, dir.File("tls.key"), []byte("oops"))
err = r.Reload(context.Background())
var re *keyload.ReloadError
if errors.As(err, &re) {
fmt.Println("rejected:", re.Result, re.Role)
}
fmt.Println("serving generation", r.Current().Generation)
}
Output: serving generation 1 serving generation 2 rejected: invalid key serving generation 2
Index ¶
- Variables
- func MetricsHandler(rs ...*Reloader) http.Handler
- type CertInfo
- type Event
- type Files
- type Material
- type Option
- type ReloadError
- type Reloader
- func (r *Reloader) ClientConfig(base *tls.Config) (*tls.Config, error)
- func (r *Reloader) Close() error
- func (r *Reloader) ConfigureTransport(t *http.Transport, base *tls.Config) error
- func (r *Reloader) Current() *Snapshot
- func (r *Reloader) DialTLSContext(base *tls.Config) func(ctx context.Context, network, addr string) (net.Conn, error)
- func (r *Reloader) Reload(ctx context.Context) error
- func (r *Reloader) ServerConfig(base *tls.Config) *tls.Config
- func (r *Reloader) Stats() Stats
- func (r *Reloader) WriteMetrics(w io.Writer) error
- type Result
- type Role
- type Snapshot
- type Source
- type SourceFunc
- type Stats
- type Trigger
- type Watcher
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrKeyMismatch = errors.New("private key does not match the leaf certificate") ErrNotYetValid = errors.New("certificate is not yet valid") ErrExpired = errors.New("certificate has expired") ErrExpiresTooSoon = errors.New("certificate expires sooner than the minimum remaining lifetime") ErrEmptyBundle = errors.New("no certificates found") ErrTruncatedPEM = errors.New("truncated or malformed PEM block") ErrChainVerification = errors.New("leaf does not chain to the configured roots") )
Validation errors, matched with errors.Is.
var ErrNoServerName = errors.New("keyload: ClientConfig requires base.ServerName; without it the server's hostname would not be checked")
ErrNoServerName is returned by Reloader.ClientConfig when base has no ServerName. Without one there would be no hostname check.
var ErrProxyConfigured = errors.New("keyload: transport uses an HTTPS proxy, which bypasses DialTLSContext; use Reloader.ClientConfig as TLSClientConfig instead")
ErrProxyConfigured is returned by Reloader.ConfigureTransport for a transport that sends HTTPS through a proxy. http.Transport does not use DialTLSContext for proxied HTTPS requests, so the reloaded roots would be silently ignored. Use Reloader.ClientConfig for such transports.
Functions ¶
Types ¶
type Event ¶
type Event struct {
Time time.Time
Trigger Trigger
Result Result
Generation uint64 // the new generation on success, the one still in service on failure
Err error // a *ReloadError on failure
// Certificates added to or removed from the snapshot, on success.
Added, Removed []CertInfo
}
Event reports a reload that changed the snapshot or failed.
type Files ¶
type Files struct {
Cert, Key string // leaf certificate (optionally followed by intermediates) and key; leave empty for trust-only clients
Roots string // CA bundle for verifying servers
ClientCAs string // CA bundle for verifying clients
}
Files is a Source that reads PEM files. Every field is optional, but Cert and Key go together.
Files is also a Watcher. On Linux it watches each file's directory, and the directory of its symlink target, with inotify, which is what makes Kubernetes Secret and ConfigMap volume updates visible. On filesystems where inotify misses changes (NFS, FUSE, SMB), when inotify limits are hit, and on other operating systems, it checks every 30 seconds instead.
type Material ¶
type Material struct {
Cert []byte // leaf certificate, optionally followed by intermediates
Key []byte // private key for the leaf
Roots []byte // CA bundle for verifying servers
ClientCAs []byte // CA bundle for verifying clients
// Origins say where each field came from, such as a file path. They are
// used only in errors and events.
CertOrigin, KeyOrigin, RootsOrigin, ClientCAsOrigin string
}
Material is the raw PEM that a Source returns.
A nil field means the source does not provide that piece of material. A non-nil but empty field means it is configured and empty, which fails validation.
type Option ¶
type Option func(*options)
Option configures a Reloader.
func DisableWatch ¶
func DisableWatch() Option
DisableWatch stops the Reloader from using the source's Watcher, leaving reloads to signals, timers and manual calls.
func MinRemaining ¶
MinRemaining rejects a leaf certificate, or any certificate in its chain, that expires sooner than d from the time of the reload.
func Name ¶
Name labels this Reloader's metrics with reloader="name", for processes that run more than one.
func OnReload ¶
OnReload calls fn after every reload attempt that changed the snapshot or failed. Reloads of unchanged material do not produce events. fn is called synchronously, in order, and must not call Reloader.Reload. It can be given more than once.
Example ¶
package main
import (
"context"
"fmt"
"log"
"os"
"github.com/rossedman/keyload"
"github.com/rossedman/keyload/internal/testpki"
)
// setup writes tls.crt, tls.key and ca.crt for localhost to a temporary
// directory, the way cert-manager writes a Secret.
func setup() *testpki.Dir {
dir, err := os.MkdirTemp("", "keyload-example")
if err != nil {
log.Fatal(err)
}
return testpki.NewDir(testpki.NoTest, dir)
}
func files(dir *testpki.Dir) keyload.Files {
return keyload.Files{
Cert: dir.File("tls.crt"),
Key: dir.File("tls.key"),
Roots: dir.File("ca.crt"),
ClientCAs: dir.File("ca.crt"),
}
}
func main() {
dir := setup()
defer os.RemoveAll(dir.Path)
r, err := keyload.New(files(dir), keyload.DisableWatch(), keyload.OnReload(func(ev keyload.Event) {
if ev.Err != nil {
fmt.Printf("%s reload rejected (%s), still serving generation %d\n", ev.Trigger, ev.Result, ev.Generation)
return
}
fmt.Printf("%s reload: generation %d, %d certificates added\n", ev.Trigger, ev.Generation, len(ev.Added))
}))
if err != nil {
log.Fatal(err)
}
defer r.Close()
testpki.WriteFile(testpki.NoTest, dir.File("ca.crt"), nil) // an empty CA bundle
_ = r.Reload(context.Background())
}
Output: initial reload: generation 1, 3 certificates added manual reload rejected (invalid), still serving generation 1
func OnSignal ¶
OnSignal reloads when the process receives sig, such as syscall.SIGHUP. It can be given more than once.
func OverlapWindow ¶
OverlapWindow keeps CA certificates that disappear from a bundle in the pool for d after they were removed, so peers with certificates issued by the outgoing CA keep working while they rotate. It applies to both roots and client CAs. The default is no overlap.
func RequireChain ¶
func RequireChain() Option
RequireChain rejects a leaf that does not chain to the newly loaded roots (or to the client CAs when no roots are configured). It catches a mismatched certificate and CA bundle before peers do.
type ReloadError ¶
type ReloadError struct {
Result Result // ResultInvalid or ResultReadError
Role Role // which material was at fault, if known
Origin string // where it came from, such as a file path, if known
Err error
}
ReloadError describes why a reload was rejected. The previous snapshot stays in service.
func (*ReloadError) Error ¶
func (e *ReloadError) Error() string
func (*ReloadError) Unwrap ¶
func (e *ReloadError) Unwrap() error
type Reloader ¶
type Reloader struct {
// contains filtered or unexported fields
}
Reloader keeps the current Snapshot and replaces it when the source changes. It is safe for concurrent use.
func New ¶
New loads material from src, validates it and starts watching for changes, until Reloader.Close. It returns an error if the first load is invalid, so a service with bad material fails at startup rather than serving it.
r, err := keyload.New(keyload.Files{Cert: "tls.crt", Key: "tls.key", Roots: "ca.crt"})
func (*Reloader) ClientConfig ¶
ClientConfig returns a client config for libraries that only accept a *tls.Config and never let you build one per connection ("Config mode"). It is the only place keyload verifies certificates itself; prefer Reloader.DialTLSContext or the grpccreds module where you can.
The returned config sets InsecureSkipVerify and verifies the server in VerifyConnection, on every handshake including resumed ones, with x509.Certificate.Verify and the same options crypto/tls uses: the current snapshot's roots (system roots if none are configured), the peer's intermediates, DNSName set to base.ServerName, ServerAuth key usage and base.Time as the clock. base.VerifyConnection, if set, runs after a successful check.
Because crypto/tls does not verify in this mode, tls.ConnectionState.VerifiedChains is empty and base.VerifyPeerCertificate receives nil verifiedChains.
ClientConfig returns ErrNoServerName if base.ServerName is empty, and an error if base already sets InsecureSkipVerify. When base has no client certificate the snapshot's leaf is presented, following reloads.
Example ¶
package main
import (
"crypto/tls"
"fmt"
"log"
"net/http"
"net/http/httptest"
"os"
"github.com/rossedman/keyload"
"github.com/rossedman/keyload/internal/testpki"
)
// setup writes tls.crt, tls.key and ca.crt for localhost to a temporary
// directory, the way cert-manager writes a Secret.
func setup() *testpki.Dir {
dir, err := os.MkdirTemp("", "keyload-example")
if err != nil {
log.Fatal(err)
}
return testpki.NewDir(testpki.NoTest, dir)
}
func main() {
dir := setup()
defer os.RemoveAll(dir.Path)
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {}))
srv.TLS = &tls.Config{Certificates: []tls.Certificate{dir.Leaf.TLS(testpki.NoTest)}}
srv.StartTLS()
defer srv.Close()
r, err := keyload.New(keyload.Files{Roots: dir.File("ca.crt")})
if err != nil {
log.Fatal(err)
}
defer r.Close()
// For libraries that take a *tls.Config and nothing else. ServerName is
// required: it is the hostname the server's certificate is checked for.
cfg, err := r.ClientConfig(&tls.Config{ServerName: "localhost"})
if err != nil {
log.Fatal(err)
}
conn, err := tls.Dial("tcp", srv.Listener.Addr().String(), cfg)
if err != nil {
log.Fatal(err)
}
defer conn.Close()
fmt.Println("verified", conn.ConnectionState().PeerCertificates[0].DNSNames[0])
}
Output: verified localhost
func (*Reloader) ConfigureTransport ¶
ConfigureTransport sets t.DialTLSContext to Reloader.DialTLSContext with base. It returns ErrProxyConfigured if t.Proxy would send HTTPS requests through a proxy, since http.Transport then does its own TLS with TLSClientConfig. When t.ForceAttemptHTTP2 is set and base.NextProtos is empty, it offers "h2" and "http/1.1".
Example ¶
package main
import (
"crypto/tls"
"fmt"
"log"
"net/http"
"net/http/httptest"
"os"
"github.com/rossedman/keyload"
"github.com/rossedman/keyload/internal/testpki"
)
// setup writes tls.crt, tls.key and ca.crt for localhost to a temporary
// directory, the way cert-manager writes a Secret.
func setup() *testpki.Dir {
dir, err := os.MkdirTemp("", "keyload-example")
if err != nil {
log.Fatal(err)
}
return testpki.NewDir(testpki.NoTest, dir)
}
func main() {
dir := setup()
defer os.RemoveAll(dir.Path)
// A server whose certificate comes from dir's CA.
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {}))
srv.TLS = &tls.Config{Certificates: []tls.Certificate{dir.Leaf.TLS(testpki.NoTest)}}
srv.StartTLS()
defer srv.Close()
// A client that only needs the CA bundle.
r, err := keyload.New(keyload.Files{Roots: dir.File("ca.crt")})
if err != nil {
log.Fatal(err)
}
defer r.Close()
tr := &http.Transport{}
if err := r.ConfigureTransport(tr, nil); err != nil {
log.Fatal(err) // keyload.ErrProxyConfigured if tr would use an HTTPS proxy
}
resp, err := (&http.Client{Transport: tr}).Get(srv.URL)
if err != nil {
log.Fatal(err)
}
resp.Body.Close()
fmt.Println(resp.Status)
}
Output: 200 OK
func (*Reloader) DialTLSContext ¶
func (r *Reloader) DialTLSContext(base *tls.Config) func(ctx context.Context, network, addr string) (net.Conn, error)
DialTLSContext returns a dial function that builds a fresh client config from the current snapshot for every connection: a clone of base with RootCAs from the snapshot and, when base has no client certificate, the snapshot's leaf. When base.ServerName is empty it is taken from the dialed address. Certificate verification is left to crypto/tls.
Use it as http.Transport.DialTLSContext, or prefer Reloader.ConfigureTransport, which also refuses transports that use a proxy. For HTTP/2 set base.NextProtos to []string{"h2", "http/1.1"}.
func (*Reloader) Reload ¶
Reload loads, validates and swaps in new material now. On failure the current snapshot stays in service and the returned error is a *ReloadError. Reloading unchanged material is a no-op.
func (*Reloader) ServerConfig ¶
ServerConfig returns a server config that follows reloads. Every handshake gets a config built from the current snapshot: a clone of base with Certificates and ClientCAs from the snapshot. Certificate verification is left to crypto/tls.
One config is built per snapshot generation and cached. The returned config keeps base's session ticket keys, cipher suites, ClientAuth policy and other settings, and session resumption keeps working across reloads. Resumed sessions have their client certificate re-verified against the current client CAs, so removing a CA also stops resumption by its clients.
Handshakes use configs built from base, not from the returned config, so changes made to the returned config afterwards are not seen. In particular, set NextProtos in base: http.Server and httptest add "h2" to a copy of the config they are given, which the configs built here do not inherit, so HTTP/2 needs base.NextProtos = []string{"h2", "http/1.1"}.
base.GetConfigForClient is replaced; base.GetCertificate is ignored while the snapshot holds a leaf.
Example ¶
package main
import (
"crypto/tls"
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"os"
"github.com/rossedman/keyload"
"github.com/rossedman/keyload/internal/testpki"
)
// setup writes tls.crt, tls.key and ca.crt for localhost to a temporary
// directory, the way cert-manager writes a Secret.
func setup() *testpki.Dir {
dir, err := os.MkdirTemp("", "keyload-example")
if err != nil {
log.Fatal(err)
}
return testpki.NewDir(testpki.NoTest, dir)
}
func files(dir *testpki.Dir) keyload.Files {
return keyload.Files{
Cert: dir.File("tls.crt"),
Key: dir.File("tls.key"),
Roots: dir.File("ca.crt"),
ClientCAs: dir.File("ca.crt"),
}
}
func main() {
dir := setup()
defer os.RemoveAll(dir.Path)
r, err := keyload.New(files(dir))
if err != nil {
log.Fatal(err)
}
defer r.Close()
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "hello over %s", req.Proto)
}))
srv.EnableHTTP2 = true
srv.TLS = r.ServerConfig(&tls.Config{
ClientAuth: tls.RequireAndVerifyClientCert,
NextProtos: []string{"h2", "http/1.1"}, // for HTTP/2, set NextProtos here
})
srv.StartTLS()
defer srv.Close()
// A client using the same reloader for its own certificate and roots.
tr := &http.Transport{ForceAttemptHTTP2: true}
if err := r.ConfigureTransport(tr, nil); err != nil {
log.Fatal(err)
}
resp, err := (&http.Client{Transport: tr}).Get(srv.URL)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
fmt.Println(string(body))
}
Output: hello over HTTP/2.0
func (*Reloader) Stats ¶
Stats returns the Reloader's counters and the expiry times of the material in service.
func (*Reloader) WriteMetrics ¶
WriteMetrics writes the Reloader's metrics to w in the Prometheus text exposition format:
keyload_generation current snapshot generation
keyload_reload_total{result} success, invalid, read_error
keyload_cert_not_after_seconds{role} expiry of the leaf and earliest CA per bundle
keyload_last_success_timestamp_seconds time of the last successful reload
With the Name option every series also carries reloader="name". To serve several Reloaders from one endpoint, use MetricsHandler.
type Result ¶
type Result string
Result is the outcome of a reload attempt, as reported in events and the keyload_reload_total metric.
type Snapshot ¶
type Snapshot struct {
Certificate *tls.Certificate // leaf + chain + key; nil for trust-only clients
Roots *x509.CertPool // verifies servers (client side); nil means system roots
ClientCAs *x509.CertPool // verifies clients (server side)
// RootCerts and ClientCACerts list the certificates in Roots and
// ClientCAs, including any outgoing CAs kept by [OverlapWindow].
RootCerts []*x509.Certificate
ClientCACerts []*x509.Certificate
Generation uint64
LoadedAt time.Time
NotAfter time.Time // earliest expiry in the leaf chain; zero without a leaf
// contains filtered or unexported fields
}
Snapshot is one immutable, validated set of TLS material. A Snapshot is never modified after it is published; treat every field as read-only.
func (*Snapshot) ClientConfig ¶
ClientConfig returns a static client config built from this snapshot: a clone of base with RootCAs taken from the snapshot and, when base has no client certificate of its own, the snapshot's leaf. Verification is left to crypto/tls. Most callers want Reloader.DialTLSContext.
func (*Snapshot) ServerConfig ¶
ServerConfig returns a static server config built from this snapshot: a clone of base with Certificates and ClientCAs taken from the snapshot. Most callers want Reloader.ServerConfig, which follows reloads.
type Source ¶
Source loads TLS material. Implementations return raw PEM and leave parsing and validation to the Reloader.
type SourceFunc ¶
SourceFunc adapts a function to the Source interface.
type Stats ¶
type Stats struct {
Generation uint64
Success uint64 // reloads that swapped in a new snapshot, including the first load
Invalid uint64 // reloads rejected by validation
ReadError uint64 // reloads that could not read their source
LastSuccess time.Time
LeafNotAfter time.Time // earliest expiry in the leaf chain; zero without a leaf
RootsNotAfter time.Time // earliest expiry among the roots; zero without roots
ClientCAsNotAfter time.Time // earliest expiry among the client CAs; zero without client CAs
}
Stats is a point-in-time view of a Reloader's counters.
type Watcher ¶
type Watcher interface {
// Watch calls changed whenever the material may have changed, until ctx
// is done.
Watch(ctx context.Context, changed func())
}
Watcher is implemented by sources that can tell when their material may have changed. The Reloader calls Watch once, in its own goroutine, and reloads whenever changed is called. Spurious calls are cheap: a reload of unchanged material is a no-op.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
client
command
Command client calls an HTTPS URL on an interval, presenting a client certificate and trusting a CA bundle that both reload from disk.
|
Command client calls an HTTPS URL on an interval, presenting a client certificate and trusting a CA bundle that both reload from disk. |
|
gencerts
command
Command gencerts writes a throwaway CA and a certificate for localhost to a directory, laid out like a cert-manager Secret (ca.crt, tls.crt, tls.key), so you can try the other examples.
|
Command gencerts writes a throwaway CA and a certificate for localhost to a directory, laid out like a cert-manager Secret (ca.crt, tls.crt, tls.key), so you can try the other examples. |
|
metrics
command
Command metrics serves Prometheus metrics for a reloader and logs every reload as structured JSON.
|
Command metrics serves Prometheus metrics for a reloader and logs every reload as structured JSON. |
|
proxy
command
Command proxy calls an HTTPS URL through an HTTP proxy, trusting a CA bundle that reloads from disk.
|
Command proxy calls an HTTPS URL through an HTTP proxy, trusting a CA bundle that reloads from disk. |
|
server
command
Command server is an HTTPS server with mutual TLS.
|
Command server is an HTTPS server with mutual TLS. |
|
grpccreds
module
|
|
|
internal
|
|
|
testpki
Package testpki generates certificates for tests.
|
Package testpki generates certificates for tests. |