kel

README

Kubernetes knowledge, written once.

Functions that know Kubernetes, for Go templates and any engine that calls Go functions.

{{ if hasCrashingPod .pods }}
crashing: {{ join (podContainerReasons .pods) ", " }}
{{ end }}

KEL is a library, not a language. There is no parser, grammar or runtime to learn: Go's text/template already composes functions with pipes. KEL adds functions that understand Kubernetes objects (Deployments, Pods, conditions, Events, …), and general helpers for the values around them: strings, numbers, time, cron, semver, Prometheus results and more.

Why

Every team that runs Kubernetes ends up writing the same functions. hasCrashingPod. rolloutComplete. allReplicasReady. isTerminating. The names change a little and the logic doesn't. Each copy lives in one controller, and when that project ends the knowledge goes with it.

The shape of the API hardly moves: a Deployment's status looks the same in every cluster. What takes knowing is how to read it. A crash-looping container doesn't always say CrashLoopBackOff. A Deployment can report Available while its rollout is stuck. A pod that restarted last week still shows the restarts today. Each team learns these one incident at a time, and writes the lesson into its own copy.

KEL collects that knowledge in one place: each lesson written once, as a function, with a test that keeps it true. Use it wherever the question comes up: a controller, a CLI, a test assertion, an alert template.

What KEL knows

A few of the things KEL's functions account for, so their callers don't have to:

• Crash loops don't always say so. Between back-offs a crash-looping container shows terminated with reason Error; only while it waits does it show CrashLoopBackOff. podCrashLoopDetected sees both.
• The reason moves. Once an OOM-killed container is waiting to restart, its state says CrashLoopBackOff and OOMKilled is in its last termination. podOOMKilledDetected reads both, but only while the container isn't running: a past kill on a container that recovered is history.
• Restart counts never reset while the pod lives. hasCrashingPod asks whether a container is failing now; podRestartedWithin .pods "10m" asks whether one restarted recently.
• Init containers are reported separately, in initContainerStatuses. A pod stuck in Init:Error has nothing wrong in its containerStatuses. The pod functions read both, and skip init containers that finished.
• Status can be about an older spec. Until status.observedGeneration reaches metadata.generation, the controller hasn't acted on the latest change. rolloutComplete and conditionsHealthy treat that as not done.
• Available isn't finished. A Deployment stays Available on its old pods while the new ones fail. rolloutComplete follows the rules kubectl rollout status uses; conditionsHealthy also reads Progressing.
• Most operators agree on conditions. Ready, Stalled, Reconciling and Synced mean the same thing across many projects' resources, so conditionsHealthy answers for kinds KEL has never seen.
• Typed objects arrive without their kind. Objects from a typed client-go list call have an empty TypeMeta. KEL takes the kind from the Go type.

Install

go get github.com/intentd/kel

Use it in a template

A complete program is in examples/main.go: a Deployment mid-rollout with a crash-looping pod, asked about from a template and from Go. Run it with go run ./examples.

kel.FuncMap() returns every function as a template.FuncMap:

data := map[string]any{
	"deployment": deployment, // *appsv1.Deployment, *unstructured.Unstructured or a map
	"pods":       pods,       // []corev1.Pod, *corev1.PodList, a list of maps, …
}

tmpl := template.Must(template.New("status").Funcs(kel.FuncMap()).Parse(
	`{{ if rolloutComplete .deployment }}rolled out{{ else }}rolling out{{ end }}`,
))
err := tmpl.Execute(os.Stdout, data)

Pass objects in whatever shape you have them:

• typed client-go objects: *corev1.Pod, []corev1.Pod, *corev1.PodList
• unstructured objects: *unstructured.Unstructured, []unstructured.Unstructured, *unstructured.UnstructuredList
• maps, as a dynamic client returns them or as decoded from JSON or YAML

Every function gives the same answer for each. Functions do no I/O: a function about pods takes the pods as an argument, so you decide what to fetch and pass it in. A function that takes a list wants a list; one object is not a list of one.

Use it from Go

Every function is also an exported Go function, so the same logic works without templates:

import "github.com/intentd/kel/kube"

pods, err := clientset.CoreV1().Pods(ns).List(ctx, opts)
if !kube.RolloutComplete(deployment) && kube.HasCrashingPod(pods) {
	log.Printf("rollout stuck: %v", kube.PodContainerReasons(pods))
}

// Labels and annotations as the map[string]string Go APIs take:
selector.Matches(labels.Set(kube.Labels(pod).StringMap()))

What's inside

276 functions in three packages, grouped by what they read:

Package	Reads	Domains
core	plain values	strings, math, conditional, types, collections, safe access, data, time, cron, semver, random, net, domain, validation
kube	Kubernetes objects	kubernetes, fields, replica, pods, warnings, container, job, service, ingress, hpa, pvc, node, statefulset, replicaset, quantity, Kubernetes validation
ecosystem	what tools around Kubernetes produce	Prometheus query results, Git repository URLs, image references

A few examples:

Expression	Answers
{{ rolloutComplete .deployment }}	Has the rollout finished, by the rules kubectl rollout status uses for Deployments, StatefulSets and DaemonSets?
{{ conditionsHealthy .obj }}	Do the object's standard conditions say it's healthy, for any kind that reports them?
{{ conditionReason .obj "Ready" }}	Why is the Ready condition in its current state?
{{ hasCrashingPod .pods }}	Is any container failing right now?
{{ podOOMKilledDetected .pods }}	Is any container failing because it runs out of memory?
{{ warningReasons .events }}	Which Warning events were recorded?
{{ timeSince (creationTimestamp .obj) }}	How many seconds old is the object?
{{ semverConstraint .spec.version "^1.4" }}	Does the version satisfy the range?
{{ promAboveThreshold .response 1000 }}	Is the Prometheus query result above 1000?

How functions behave

The functions follow the same rules, so you can predict one you haven't used:

• Missing data gives zero values. An absent field, an empty list or a nil object returns "", 0, false or an empty list, not a panic.
• Questions return booleans. is…, has…, all…, any… and …Detected functions return bool.
• Lists return lists. podNames, warningReasons, ingressRules and other plural names return []string (or []int64 for ports). Use join to turn one into text.
• Bad input is an error. A function that cannot answer, such as semverMajor on "not-a-version", returns an error. In a template the error stops execution and is reported, instead of rendering a wrong value.
• Answers are about now. Health functions describe the current state, not the object's history, and don't read the clock. The few that do, such as podRestartedWithin and timeSince, say so.

Add your own functions

Register adds functions beside KEL's and panics if a name is already taken. Override replaces one of KEL's on purpose and panics if the name doesn't exist, so a typo can't add a new function by mistake.

funcs := kel.FuncMap() // a copy; changing it does not affect other callers
kel.Register(funcs, template.FuncMap{"teamOwner": teamOwner})

Find a function

Each function's documentation comes from its Go doc comment, and the registry is generated from those comments. kel.Builtin() returns it, which is useful for CLI help, editor completion or your own reference pages:

reg := kel.Builtin()
f, ok := reg.Get("rolloutComplete") // domain, description, example, keywords
matches := reg.Search("crash")      // hasCrashingPod, podCrashLoopDetected, …
pods := reg.ByDomain("pods")

If you add functions of your own, document them the same way and merge them with reg.With(...). The gendoc package reads doc comments in KEL's format from any Go package.

Status

KEL is before 1.0. Function names and signatures may still change between minor versions; release notes list every breaking change. It started inside Inrun, which uses it in every Blueprint expression, and is now a separate module with no dependency on Inrun.

Contributing

KEL grows by what people have learned the hard way. If you've debugged a state that a function here reads wrong, or written the same check into a second project, that's a contribution: a fix with a test that shows the case, or a new function.

A good KEL function:

• answers a question that comes up across projects, not one project's own question. Functions about a widely used tool's output, such as a Prometheus query result, belong in ecosystem
• takes its input in the shape its source produces: Kubernetes objects as the API returns them, a Prometheus result as the HTTP API returns it
• returns zero values for missing data, and an error only when it cannot answer
• has a doc comment in the standard shape, with keywords and an example, and tests. A test built from a real object (kubectl get -o json) is the best evidence for a subtle case: testdata/live holds snapshots of fourteen states from a real cluster, and how to capture more

Document a new function like this:

// PodCount returns the number of pods.
//
// Keywords: pods, count, total
//
// Example:
//
//	{{ podCount .pods }}  → 3
func PodCount(pods interface{}) int {

Then run go generate ./... to update the registry. go test ./... fails if the registry is out of date, if a registered function has no documentation, if a signature breaks the rules above, or if a documented example gives a different answer for typed or unstructured objects than for maps.

License

Apache 2.0

Documentation

Overview

Package kel is KEL, the Kubernetes Expression Library: functions that know Kubernetes, for any Go template or expression engine. It is a library, not a language: no parser, no grammar, no runtime of its own.

FuncMap returns every function as a template.FuncMap, and Builtin describes them: domain, description, example and keywords, from their doc comments. Register adds your own functions beside KEL's, and Override replaces one on purpose.

The functions live in three packages, by what they read:

• core: plain values (strings, numbers, time, cron, collections, …)
• kube: Kubernetes objects, in the shape the API returns them
• ecosystem: what the tools around Kubernetes produce, such as a Prometheus query result or a container image reference

Every function takes its inputs as arguments and does no I/O: a function about pods is given the pods.

Index

• Variables
• func FuncMap() template.FuncMap
• func Override(dst, src template.FuncMap)
• func Register(dst, src template.FuncMap)
• type DomainInfo
• type FunctionInfo
• type Registry
  • func Builtin() Registry
  • func (r Registry) ByDomain(domain string) []FunctionInfo
  • func (r Registry) Domain(name string) (DomainInfo, bool)
  • func (r Registry) Get(name string) (FunctionInfo, bool)
  • func (r Registry) Search(term string) []FunctionInfo
  • func (r Registry) With(other Registry) Registry

Variables

var BuiltinDomains = []DomainInfo{
	{Name: "collections", Title: "Collections", Intro: "Convert a value that may arrive in several shapes (a native list or map, or the same as a JSON or YAML string) into a list, a map or a string, so the functions after it see one shape.\n\nRead lists and maps without guarding every step: get an element by index or a value by key, test membership, take keys and values, or keep and drop keys. Out-of-range and missing keys give zero values, not errors."},
	{Name: "conditional", Title: "Conditional", Intro: "Choose between values inside an expression: defaults for empty values, the first non-empty of several, and ternaries, so a template does not need an if block for a one-line choice."},
	{Name: "container", Title: "Container", Intro: "Read the containers of a workload (a Deployment, StatefulSet, Job, Pod, …, through its pod template): images, environment variables, ports and counts, by container index or name."},
	{Name: "cron", Title: "Cron", Intro: "Work with five-field cron expressions (minute hour day-of-month month day-of-week): read single fields, build and normalise expressions, convert between the string form and a map with one key per field, describe a schedule in words, and check that an expression is valid."},
	{Name: "data", Title: "Data", Intro: "Encode, hash and shape values: base64, a short SHA-256 for content-addressed names, slugs and truncation that keep a value valid as a Kubernetes name, and lookup tables written inline in an expression."},
	{Name: "domain", Title: "Domain", Intro: "Take domain names and URLs apart: the host of a URL, and a domain without its subdomain."},
	{Name: "fields", Title: "Fields", Intro: "Read the fields every Kubernetes object has: name, namespace, UID, resource version, creation timestamp, labels and annotations."},
	{Name: "hpa", Title: "HPA", Intro: "Read a HorizontalPodAutoscaler: current, desired, minimum and maximum replicas, whether it is scaling or at its limit, its conditions, its scale target and the types of metric it reports."},
	{Name: "ingress", Title: "Ingress", Intro: "Read an Ingress: whether it has a load balancer address, its host and IP, its class, the hosts in its rules and TLS section, and every load balancer address."},
	{Name: "job", Title: "Job", Intro: "Read Jobs and CronJobs: whether a Job succeeded, failed or is running, and when a CronJob last ran, last succeeded and runs next. The functions that look inside a Job's pods take the pods as a list."},
	{Name: "kubernetes", Title: "Kubernetes", Intro: "Answer the questions about any Kubernetes object that every controller asks: its conditions and their reasons, whether it is ready, synced, terminating or exists, and the fields of its spec and status, read safely."},
	{Name: "kubevalidation", Title: "Kubernetes validation", Intro: "Check values against the rules the API server applies to names and metadata: label keys and values, annotation keys, and DNS-1123 subdomains, so a bad value is caught with a clear message before the API server rejects it."},
	{Name: "math", Title: "Math", Intro: "Arithmetic on values that may arrive as integers, floats or numeric strings: add, subtract, multiply, divide, modulo, absolute value, minimum, maximum and clamping. Division and modulo by zero are errors."},
	{Name: "net", Title: "Net", Intro: "Check IP addresses and CIDR ranges: whether an address is valid, private, or inside a range."},
	{Name: "node", Title: "Node", Intro: "Read a Node: readiness, allocatable CPU and memory, conditions, taints, and where it is (zone, region, instance type, from the well-known topology labels); and which node a pod is scheduled on."},
	{Name: "pods", Title: "Pods", Intro: "Answer questions about a set of Pods, as the API returns them, passed as a list or a PodList: names, IPs, phases, nodes, readiness, restarts, and why containers are waiting or failed (CrashLoopBackOff, OOMKilled, ImagePullBackOff, …)."},
	{Name: "prometheus", Title: "Prometheus", Intro: "Read the response of a Prometheus HTTP API instant query (/api/v1/query), the object with \"status\" and \"data\": the value, sums and maxima across series, thresholds, series counts and label values. No data gives zero values; a sample that does not parse is an error."},
	{Name: "pvc", Title: "PVC", Intro: "Read a PersistentVolumeClaim (whether it is bound, its phase, capacity, storage class and access modes), the PersistentVolume bound to it, and its StorageClass."},
	{Name: "quantity", Title: "Quantity", Intro: "Work with Kubernetes resource quantities (\"500m\", \"2Gi\"): parse, compare, add and subtract them, keeping the canonical form."},
	{Name: "random", Title: "Random", Intro: "Generate cryptographically secure random values: hex, base64 and alphanumeric strings, and UUIDs. They return a different value on every call, so evaluate them once, when a secret is first created, and keep the result; evaluated on every reconcile, a password changes every time."},
	{Name: "replica", Title: "Replica", Intro: "Read the replica counts of a workload (desired, ready, available and updated) and whether all replicas are ready or a rollout has completed."},
	{Name: "replicaset", Title: "ReplicaSet", Intro: "Read a Deployment's ReplicaSets, passed as a list: how many, their names, and which are older than the Deployment's current revision; and any object's controller, from its ownerReferences."},
	{Name: "safeaccess", Title: "Safe access", Intro: "Read a nested field and fall back to a default when it is absent or of the wrong type, typed as a string, an integer or a boolean."},
	{Name: "semver", Title: "Semver", Intro: "Work with semantic versions (\"1.2.3\", \"v1.2.3-rc.1\"): read their parts, compare them, bump them, and check them against constraints such as \">= 1.20\". A version that does not parse is an error."},
	{Name: "service", Title: "Service", Intro: "Read a Service (cluster IP, node port, load balancer IP and host) and its EndpointSlices, passed as one slice or a list: whether any endpoint is ready, and the endpoints as \"ip:port\"."},
	{Name: "statefulset", Title: "StatefulSet", Intro: "Read what is particular to a StatefulSet: its current and update revisions, and the partition of a staged rolling update. Replica counts, rollout completion and pods have their own functions."},
	{Name: "strings", Title: "Strings", Intro: "The string operations expressions need most: case, trimming, prefixes and suffixes, splitting and joining, concatenation, case conversion between camelCase and kebab-case, and shortening text for display."},
	{Name: "time", Title: "Time", Intro: "Reason about timestamps (RFC3339) and durations (Go units plus d, w, mo and y): elapsed and remaining time, formatting, time windows and weekdays, duration arithmetic, and the next run of a cron schedule."},
	{Name: "types", Title: "Types", Intro: "Inspect and convert types: what type a value has (map, slice, string, number, bool, null), whether it is empty, and conversion to strings, numbers, booleans, lists and JSON."},
	{Name: "validation", Title: "Validation", Intro: "Check that free text is the kind of value it claims to be: an email address, a URL, JSON, a port number. A form field is free text until something checks it; catching a malformed value early, with a clear message, beats finding out downstream.\n\nCheck references to artifacts the tools around Kubernetes consume: a git repository (Argo CD, Flux) and a container image reference (registries). Catching a malformed one early beats finding out when the tool fails to reconcile it."},
	{Name: "warnings", Title: "Warnings", Intro: "Answer questions about Kubernetes Events, as the API returns them (core/v1 or events.k8s.io/v1), passed as a list or an EventList. Only Warning events count, so a caller can pass all of an object's events."},
}

BuiltinDomains is every domain, from the builders' doc comments.

var BuiltinFunctions = []FunctionInfo{}/* 276 elements not displayed */

BuiltinFunctions is every function, from its doc comment.

Functions

func FuncMap

func FuncMap() template.FuncMap

FuncMap returns KEL's functions as a template.FuncMap: a copy, so a caller can add its own (with Register) without changing KEL's.

tmpl, err := template.New("f").
    Option("missingkey=zero").
    Funcs(kel.FuncMap()).
    Parse(value)

func Override

func Override(dst, src template.FuncMap)

Override replaces functions already in dst with src's, on purpose; a name not in dst panics.

func Register

func Register(dst, src template.FuncMap)

Register adds src's functions to dst; a name already in dst panics, since two functions would answer to one name. Use it to add your own functions beside KEL's.

Types

type DomainInfo

type DomainInfo struct {
	Name  string // e.g. "cron"
	Title string // e.g. "Cron"
	Intro string // what the functions are for
}

DomainInfo describes one group of functions, from its builder's doc comment.

type FunctionInfo

type FunctionInfo struct {
	Name        string   // template name, e.g. "cronValid"
	Domain      string   // e.g. "cron"
	Description string   // what it does
	Example     string   // a template using it
	SeeAlso     []string // related functions
	Keywords    []string // search terms
}

FunctionInfo describes one function, from its doc comment.

type Registry

type Registry struct {
	Functions []FunctionInfo // sorted by domain, then name
	Domains   []DomainInfo   // sorted by name
}

Registry is a set of documented functions and their domains. Builtin returns KEL's; a program that adds functions can merge its own with With.

func Builtin

func Builtin() Registry

Builtin returns the registry of KEL's functions.

func (Registry) ByDomain

func (r Registry) ByDomain(domain string) []FunctionInfo

ByDomain returns the functions in domain (case-insensitive).

func (Registry) Domain

func (r Registry) Domain(name string) (DomainInfo, bool)

Domain returns the domain called name (case-insensitive).

func (Registry) Get

func (r Registry) Get(name string) (FunctionInfo, bool)

Get returns the function called name (case-insensitive).

func (Registry) Search

func (r Registry) Search(term string) []FunctionInfo

Search returns the functions whose name, description, example or a keyword contains term (case-insensitive).

func (Registry) With

func (r Registry) With(other Registry) Registry

With returns r merged with other. A function in other replaces the one in r with the same name; a domain in both keeps r's title and both intros.