kloudlens

README

KloudLens

KloudLens is an eBPF-based runtime visibility platform for Kubernetes. It observes every container from birth to exit by pairing syscall enter/exit into semantic events, enriching them with pod and process metadata, and exposing the stream over gRPC.

Beyond raw syscall tracing, KloudLens offers five higher-level capabilities that operators can consume directly:

• Semantic Intent Aggregation — individual syscalls are folded into higher-level intents (FileRead, NetworkExchange, ProcessStart, …) so downstream consumers reason about behaviour, not opcodes.
• Causal Session Graph — process lineage and cross-container edges (fork/exec/file-touch/IPC) are materialised as a graph and queryable by session.
• Behavioural Baseline — a learner observes a workload for a window, snapshots a Profile, and a detector emits only the deviations from that baseline.
• Behavioural Contract IR + Gap Analysis — an enforcer-neutral YAML IR compiles to seccomp, AppArmor, KubeArmor, Cilium network policy, Kyverno, OPA/Rego, Kubernetes NetworkPolicy, and PodSecurity. Every rule carries the evidence that produced it — how often the behaviour was seen, when it was first and last seen, and a sample of the event IDs — so a reviewer can ask why any allowance exists. Gap analysis compares an observed profile against an existing policy to surface unused allowances and observed-but-denied actions.
• Loss Accounting — under pressure the agent sheds events rather than blocking the workload, and reports that in the same stream it shipped. OverflowSummary breaks a window's loss down by ring and by loss point (kernel ring overrun, in-kernel sampling, decoder), and FidelityTransition declares every change in sampling rate, so a consumer knows the fidelity of what it received instead of assuming completeness.

This repo builds the agent (kloudlens). The optional cluster fan-in (kloudlens-aggregator) lives in its own repo: boanlab/kloudlens-aggregator. The operator CLI, klctl, lives in boanlab/kloudlens-cli.

KloudLens supports three deployment modes:

Mode	Container source	Control plane
kubernetes (default)	containerd CRI + K8s API	4 CRDs (HookSubscription, BaselinePolicy, BehaviorContract, NodeCapabilities)
docker	containerd CRI	local YAML directory / docker-compose
host	containerd CRI (optional)	CLI flags only (no CRDs)

---

Documentation

New to KloudLens? Start here:

Doc	Purpose
getting-started/README.md	15-minute Kubernetes quickstart for the agent
getting-started/docker-mode.md	Host-binary and docker-compose deployment of the agent
getting-started/integrations.md	Prometheus scrape config and gRPC client reference
getting-started/troubleshooting.md	Symptom-indexed diagnostics for install & attach
contribution/README.md	Agent development environment & contribution guide

For CLI usage, detection recipes, integration pipelines, and the BehaviorContract PR workflow, see the CLI repo's own getting-started/: boanlab/kloudlens-cli.

Components

Component	Location
kloudlens (agent)	cmd/kloudlens/ — node DaemonSet; eBPF tracing + intent/graph/baseline/contract pipeline
kloudlens-aggregator	https://github.com/boanlab/kloudlens-aggregator — multi-node fan-in + re-export
klctl (CLI)	https://github.com/boanlab/kloudlens-cli
gRPC wire format	protobuf/ — EventService / AdminService / IntentExporter .proto files and the generated Go stubs
Manifests	deployments/ — CRDs, DaemonSet, docker-compose

Requirements

• Linux kernel 5.4+ with eBPF support (BTF recommended; BPF-LSM optional — KloudLens falls back to kprobe/security_* when absent)
• Go 1.24+ (build only)
• Clang / LLVM 14+ (only when rebuilding BPF sources)
• CAP_BPF + CAP_PERFMON (DaemonSet) or root (host binary)

---

Quick Deploy (Kubernetes)

Prerequisites

• Kubernetes cluster with containerd runtime
• Linux kernel 5.4+ on every node with BTF at /sys/kernel/btf/vmlinux
• kubectl with cluster-admin privileges

Install

git clone https://github.com/boanlab/KloudLens.git
cd KloudLens

kubectl apply -f deployments/crds/ # 4 CRDs
kubectl apply -f deployments/manifests/daemonset.yaml # DaemonSet + RBAC + ConfigMap

For the docker-compose / host-binary install paths, see getting-started/docker-mode.md.

Verify

kubectl get pods -n kloudlens -o wide
kubectl -n kloudlens port-forward ds/kloudlens 9090:9090
curl -s 127.0.0.1:9090/healthz
curl -s 127.0.0.1:9090/metrics | head

If you have klctl installed (kloudlens-cli), klctl status --agent=127.0.0.1:8180 and klctl caps give a higher-level view.

---

Configuration

Every setting is a command-line flag; kloudlens --help prints the same list. The shipped DaemonSet sets the core flags below.

Core flags

Flag	Default	Purpose
--node	(empty)	Node name stamped on every ContainerMeta. Required for --publish-caps.
--cluster	(empty)	Cluster name stamped on every ContainerMeta.
--mode	monitor	monitor emits intents and deviations; learn builds a baseline profile.
--enrich	off	Pod/container metadata source: off, proc, cri, docker.
--cri-endpoint	(crictl default)	CRI socket URI, e.g. unix:///run/containerd/containerd.sock.
--wal-dir	(empty)	Intent WAL directory. Required for --subscribe-addr.
--subscribe-addr	(empty)	host:port for the event subscribe API.
--admin-addr	(empty)	host:port for the AdminService that klctl drives.
--metrics-addr	(empty)	host:port for Prometheus /metrics and the health endpoints.
--profile-in	(empty)	Profile JSON to detect against in monitor mode.
--profile-out	(empty)	Where learn mode writes the promoted profile on shutdown.
--auto-downgrade	true	Adaptive sampling controller that reacts to ring-buffer pressure.
--advertise-listeners	true	Advertise this node's listening sockets so the aggregator can resolve cross-node peers.
--graph	on	Causal session graph. off skips graph writes on the hot path.

Other flags

Flag	Default	Purpose
--overflow-window	1s	Loss-accounting window; one OverflowSummary per window in which loss advanced, carrying a per-ring breakdown.
--coalesce-reap-interval	1s	Close-out cadence for expired coalesce windows. 0 leaves totals to the next same-key event.
--auto-downgrade-interval	1s	How often the downgrade controller samples ring usage.
--advertise-interval	10s	Listener advertise cadence.
--enrich-interval	30s	How often the enricher rebuilds its NS/CRI caches.
--proc-root	/proc	procfs the agent resolves PIDs and descriptors through. The default is correct under hostPID: true; set it when procfs is mounted elsewhere.
--docker-endpoint	unix:///var/run/docker.sock	Docker Engine API for --enrich=docker.
--wal-max-bytes	2 GiB	WAL size cap; oldest segments are trimmed past it.
--wal-segment-size	32 MiB	WAL segment rotation size.
--wal-ttl	2h	WAL retention time.
--wal-gc-every	30s	WAL janitor cadence. 0 disables it.
--graph-db	(memory only)	bbolt path for session-graph persistence.
--graph-db-timeout	2s	bbolt file-lock wait.
--min-samples	50	learn: minimum samples before a profile may be promoted.
--image-ref	kloudlens/node	learn: selects the workload's learner by image and seeds the profile ID. An image the agent never observed is an error, not a node-wide fallback.
--label-hash	(empty)	learn: workload label hash seeding the profile ID.
--deviations-out	-	monitor: deviation JSONL path. - is stderr, empty discards.
--target-ns	(empty)	Monitor only these pidNS:mntNS keys.
--except-ns	(empty)	Skip these pidNS:mntNS keys. Ignored when --target-ns is set.
--export-grpc	(empty)	host:port of an IntentExporter collector to ship intents to.
--export-queue	1024	In-flight queue depth for --export-grpc; oldest is dropped on overflow.
--publish-caps	false	Publish the NodeCapability CR. Requires --node.
--publish-caps-interval	5m	Re-publish cadence for --publish-caps.
--publish-caps-apiserver	(in-cluster)	Override the apiserver URL for --publish-caps.
--enable-raw-syscalls	false	Attach raw_syscalls tracepoints. High volume; duplicates the per-syscall hooks.
--enable-raw-stream	false	Append raw SyscallEvents to the WAL raw stream and SubscribeRaw. High volume.
--enable-lsm-audit	false	Attach the security_* kprobes. Per-event dentry walks; high overhead.
--enable-correlation	false	Cross-event correlation heuristics on the hot path. Not yet surfaced downstream.
--duration	0	Stop after this long. 0 runs until SIGINT.
--output	-	Intent JSONL path. - is stdout.
--stats-every	5s	Periodic stats line on stderr. 0 disables it.
--no-ebpf	false	Skip the eBPF loader; pipeline wire-check only.
--skip-bpf-programs	(empty)	Drop named BPF programs before load. Escape hatch for verifier rejections.
--version-tag	(build tag)	Version label on kloudlens_build_info and AgentStatus.

---

Streaming Events

kloudlens ships two gRPC listeners — --admin-addr (klctl control plane) and --subscribe-addr (event subscribe). The shipped DaemonSet binds them on :8180 and :8181; the host-binary defaults are empty (you supply the address explicitly). The aggregator fan-ins every node onto a single merged stream.

kubectl port-forward -n kloudlens daemonset/kloudlens 8180:8180 8181:8181

Any gRPC client speaking the services defined in protobuf/ can subscribe. For an out-of-the-box consumer with intent / deviation / baseline / contract workflows, use klctl:

klctl stream intents --agent=127.0.0.1:8181
klctl stream deviations --agent=127.0.0.1:8181
klctl get events --follow --agent=127.0.0.1:8181

See getting-started/integrations.md for Prometheus scrape config, and protobuf/README.md for the raw message definitions.

Record kinds on the intent stream

Kind	Meaning
ProcessStart	exec, with binary and argv
FileRead / FileWrite / FileReadWrite / FileAccess	file I/O and metadata access, with resolved path
NetworkExchange	socket activity, with peer where resolvable
DNSAnswer	A record parsed from a DNS response
ListenerAdvertise	a local listener, feeding cross-node peer attribution
ListenerWithdraw	that listener retiring when its process exits
OverflowSummary	events lost in a window, broken down by loss point and ring
CoalesceSummary	suppressed events from a coalesce window whose key saw no follow-up event
FidelityTransition	a change in capture fidelity, with the sampling rate and enrichment tier it installed

ListenerAdvertise / ListenerWithdraw are a pair: the aggregator adds a listener to its cluster registry on the first and retires it on the second, so a connect to a reused address cannot resolve to a dead process.

The last three are accounting records rather than observations — they describe what the stream could not carry, or under what fidelity it was captured. Because they ride the same stream, a consumer reading only observations still learns its own fidelity instead of assuming completeness.

OverflowSummary carries read and dropped for the window, plus kernel_dropped and sampled_dropped with a _<ring> breakdown for the rings that actually lost events, and adaptive_level when a downgrade supervisor is running. loss_ratio covers ring overruns and decode drops; sampled records are reported beside it rather than folded in, since shedding is a reduction the operator configured rather than a failure.

OverflowSummary and CoalesceSummary have /metrics counterparts — kloudlens_overflow_summary_total and kloudlens_coalesce_suppressed_total — so loss is visible whether you read the stream or scrape the agent.

---

Observability

kloudlens exposes /metrics (Prometheus), /healthz, and /readyz. Key series include ring-buffer usage and drops, adaptive-downgrade level, WAL overflow, subscriber backlog, per-sink export counters, and graph/session gauges.

See getting-started/integrations.md for scrape config and the alert shortlist.

---

Architecture

syscall tracepoints + security_* kprobes + fentry/filp_close + cgroup_skb
 → eight category ring buffers
 (crit | bulk_file | bulk_net | bulk_proc | bulk_file_meta | dns | proc_lc | sock_lc)
 → user-space pair / decode / enrich / aggregate
 → intent + graph + deviation streams
 → WAL + gRPC (EventService + AdminService) + Prometheus

Per-package map and feature matrix: contribution/README.md.

---

Development

See contribution/README.md for the development environment, build instructions, and contribution guidelines.

make build test # gofmt + golangci-lint + gosec + build + test (from repo root)

Live eBPF smoke tests (require root + a supported kernel):

KLOUDLENS_LIVE_SENSOR=1 go test -race ./internal/sensor/...

License

The user-space agent (everything under the repo root except bpf/kloudlens.bpf.c) is licensed under the Apache License 2.0 — see LICENSE.

The kernel-space BPF program at bpf/kloudlens.bpf.c is licensed under GPL-2.0 because it links against GPL-only kernel helpers (the standard BPF licensing constraint). The BPF headers it includes (*.bpf.h) are dual-licensed as (GPL-2.0-only OR Apache-2.0) so they remain reusable from the user-space side as well.

---

Copyright 2026 BoanLab @ Dankook University