ouroboros

README

ouroboros

Go reimplementation of compumike/hairpin-proxy — fixes the hairpin-NAT problem for Kubernetes Ingress controllers configured with PROXY-protocol.

Why

When an external load balancer in front of an ingress-controller (typically ingress-nginx with use-proxy-protocol: true) prepends the PROXY-protocol header, internal traffic from in-cluster pods bypasses the LB and reaches the ingress-controller without the header. The connection is then rejected. Common offenders: cert-manager HTTP-01 challenges, internal https:// calls to your own public hostnames, healthchecks.

Do you need ouroboros at all? Since KEP-1860 (beta in Kubernetes 1.30, GA in 1.32) the kube-proxy can stop short-circuiting LoadBalancer IPs to the local Service when the cloud-controller-manager (CCM) sets status.loadBalancer.ingress[].ipMode: Proxy. With that flag set the LB always processes the connection — including its PROXY-protocol injection — and the hairpin path simply does not exist. A CNI that overrides kube-proxy must also honour the ipMode field — check your CNI's release notes for KEP-1860 support before relying on this fix. If your CCM and CNI both implement the contract you can remove ouroboros entirely; if only your CCM does, deploy on Kubernetes 1.30+ and check that the CNI agrees. ouroboros remains a workaround for the cluster topologies where that machinery is not available.

ouroboros fixes this with two cooperating components:

1. A controller that watches Ingress (and optionally Gateway-API Gateway + HTTPRoute) and rewrites the kube-system/coredns ConfigMap so internal lookups for those hostnames resolve to a small in-cluster proxy.
2. A TCP proxy that listens on 8080/8443, prepends the PROXY-protocol v1 header, and forwards to the real ingress-controller.

Both components ship as one binary, dispatched by subcommand.

Architecture

                        ┌────────────────────┐
                        │ Ingress / Gateway  │
                        │   (k8s API)        │
                        └────────┬───────────┘
                                 │ informers
                                 ▼
   ┌───────────────────────────────────────────┐
   │ ouroboros controller                      │
   │ - extracts hostnames                      │
   │ - reconciles CoreDNS ConfigMap (or hosts) │
   └───────────────────┬───────────────────────┘
                       │
                       ▼
       ┌───────────────────────────┐
       │ kube-system/coredns       │
       │  rewrite name foo.example │
       │    ouroboros-proxy....    │
       └───────────┬───────────────┘
                   │ DNS lookup from a pod
                   ▼
   ┌───────────────────────────────────────────┐
   │ ouroboros proxy   (Service ClusterIP)     │
   │ - accepts TCP                             │
   │ - prepends PROXY-protocol v1 header       │
   └───────────────────┬───────────────────────┘
                       │
                       ▼
              ingress-nginx-controller

Install

helm install ouroboros oci://ghcr.io/lexfrei/charts/ouroboros \
  --version 0.3.0 \
  --namespace ouroboros --create-namespace

Override the upstream backend if you don't run ingress-nginx:

helm install ouroboros oci://ghcr.io/lexfrei/charts/ouroboros \
  --namespace ouroboros --create-namespace \
  --set proxy.target.host=my-ingress.my-ns.svc.cluster.local

Enable Gateway-API support:

--set controller.gatewayApi.enabled=true

Modes

Mode	Reconciler	Use when
coredns	mutates kube-system/coredns ConfigMap	default — works for any pod that uses CoreDNS for DNS
etc-hosts	writes /etc/hosts on each node (DaemonSet)	for kubelet, container runtime, or anything bypassing CoreDNS
external-dns	emits externaldns.k8s.io/v1alpha1.DNSEndpoint CRs	managed clusters that block writes to kube-system/coredns (EKS Auto, GKE Autopilot, AKS); clusters with node-local-dns (the per-node cache bypasses CoreDNS for non-cluster.local queries — see caveat below); split-horizon DNS; multi-cluster published DNS

Switch via --set controller.mode=external-dns (or --set etcHosts.enabled=true for etc-hosts).

external-dns mode

Set controller.mode=external-dns and ouroboros stops mutating CoreDNS. Instead it writes a DNSEndpoint per hostname per address family (one A record holding all IPv4 ClusterIPs of the proxy Service, one AAAA holding all IPv6 — dual-stack Services produce both) into the controller's namespace. An external-dns deployment configured with --source=crd picks them up and publishes to whichever DNS provider it manages.

helm install ouroboros oci://ghcr.io/lexfrei/charts/ouroboros \
  --namespace ouroboros --create-namespace \
  --set controller.mode=external-dns

externalDns.proxyService (default: the chart-rendered proxy Service) is auto-resolved to a ClusterIP at startup. Use externalDns.proxyIP to override; in that case the controller does not need a get on Services. Add provider-specific annotations such as external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" via externalDns.annotations.

externalDns.cleanupOnUninstall (default true) renders a Helm post-delete hook that runs kubectl delete dnsendpoints filtered by ouroboros's ownership labels after the chart is uninstalled. It runs post-delete (not pre-delete) on purpose: at post-delete time the controller Deployment is already gone, so it cannot race-recreate any DNSEndpoint we delete. external-dns then sees the records vanish via watch and drops upstream DNS without waiting for its TXT-registry sweep.

Why DNSEndpoint and not annotated Services / DNSRecordSet

Two alternatives surface during design discussions:

• Annotated headless Services (the lexfrei/kuberture pattern): produces a Service object per hostname, which pollutes the Service catalog and races with kube-proxy.
• DNSRecordSet: an evolving proposal in external-dns; not yet ratified, so adopting it would tie the chart to a moving target.

DNSEndpoint is external-dns's documented stable contract — every shipping provider (Cloudflare, Route53, AzureDNS, GCloud, etc.) supports it through the CRD source.

Coverage caveat (both modes). Hostname extraction is asymmetric by design:

• Ingress: only spec.tls[].hosts is read; plain HTTP-only Ingresses are ignored. The hairpin-NAT problem manifests for TLS-terminated PROXY-protocol traffic.
• Gateway-API: Gateway.spec.listeners[].hostname and HTTPRoute.spec.hostnames are read regardless of protocol. Listeners are commonly paired (HTTP + HTTPS for redirect-to-TLS), and operators expect both to be hairpinned.

etc-hosts caveat. Each DaemonSet pod runs a full controller — Ingress/Gateway informers per node. On large clusters that is N replicated kube-apiserver watches producing identical results. Prefer coredns mode unless your nodes genuinely bypass cluster DNS.

node-local-dns caveat. coredns mode does NOT cover pods that resolve through node-local-dns. Pods on node-local-dns-equipped clusters query the per-node cache first; for non-cluster.local queries (which is exactly the hairpin case) node-local-dns forwards UPSTREAM and never sees the rewrite block ouroboros writes into CoreDNS. Hairpin silently fails for those pods. ouroboros logs a Warn at startup when it detects the kube-system/node-local-dns ConfigMap. Override the lookup target via OUROBOROS_NODE_LOCAL_DNS_NAMESPACE / OUROBOROS_NODE_LOCAL_DNS_CONFIGMAP if your deployment uses a non-default location. Two reliable workarounds:

• Switch controller.mode=external-dns — DNSEndpoint records flow through whatever provider/CCM the cluster uses, independent of the node-local cache.
• Manually add the same rewrite name directives to the node-local-dns Corefile block(s) that handle external queries. ouroboros does not auto-mutate node-local-dns because its Corefile uses pillar templates and zone scopes that are gnarly to safely transform without per-cluster knowledge.

Multi-ingress-controller caveat. Clusters running two ingress controllers (one with PROXY-protocol, one without) need to scope ouroboros to the PROXY-protocol one — otherwise every hostname is rewritten and traffic for the other controller's hosts hits a 404. Set controller.ingressClass=<class> (and controller.gatewayClass=<class> for Gateway-API) to filter sources. Ingresses without an explicit spec.ingressClassName are dropped under the filter — they are ambiguous, and silently hairpinning them via the wrong controller is the failure mode this knob exists to prevent.

RBAC matrix (operator-facing)

The chart suppresses the Role belonging to the other modes — operators running external-dns mode never see kube-system manifests, which is the frequent ask from managed-cluster users.

Mode	Cluster-scope reads	Namespaced writes
coredns	networking.k8s.io/ingresses (+ gateway.networking.k8s.io opt-in)	kube-system: configmaps/coredns get,update,patch
external-dns	same	externalDns.namespace (default release-ns): externaldns.k8s.io/dnsendpoints full CRUD; release-ns: named-Service get for ClusterIP auto-discovery; release-ns post-delete hook: SA + Role[dnsendpoints delete,deletecollection] + RoleBinding for cleanup-on-uninstall
etc-hosts	same	(no extra Role; node-local file write via DaemonSet hostPath)

CoreDNS reload caveat. coredns mode assumes CoreDNS' reload plugin is enabled (the default in kubeadm). If your Corefile lacks it, ouroboros logs a warning and the rewrite block is written but not picked up until CoreDNS pods are restarted manually. Verify with:

kubectl --namespace kube-system get configmap coredns --output jsonpath='{.data.Corefile}' | grep -w reload

Verification

After install:

kubectl --namespace kube-system get configmap coredns --output jsonpath='{.data.Corefile}' | grep -A20 BEGIN.ouroboros

You should see a block like:

# === BEGIN ouroboros (do not edit by hand) ===
rewrite name foo.example.com ouroboros-proxy.ouroboros.svc.cluster.local.
# === END ouroboros ===

From any pod:

getent hosts foo.example.com
# returns the ouroboros-proxy ClusterIP, not the LoadBalancer IP

A curl https://foo.example.com/ from a pod will then see X-Forwarded-For populated correctly by the ingress-controller because the PROXY-protocol header reached it.

Configuration

Both subcommands accept flags and env vars (flags override env, env overrides defaults). Run with --help for the full list — the table below is the source-of-truth alphabetical reference.

ouroboros controller

Flag	Env var	Default	Notes
--mode	OUROBOROS_CONTROLLER_MODE	coredns	One of coredns, etc-hosts, external-dns.
--kubeconfig	OUROBOROS_CONTROLLER_KUBECONFIG	(empty = in-cluster)	Path to a kubeconfig file.
--gateway-api	OUROBOROS_CONTROLLER_GATEWAY_API	false	Watch Gateway/HTTPRoute in addition to Ingress.
--resync	OUROBOROS_CONTROLLER_RESYNC	10m	Informer resync period.
--coredns-namespace	OUROBOROS_CONTROLLER_COREDNS_NAMESPACE	kube-system	CoreDNS ConfigMap namespace.
--coredns-configmap	OUROBOROS_CONTROLLER_COREDNS_CONFIGMAP	coredns	CoreDNS ConfigMap name.
--coredns-key	OUROBOROS_CONTROLLER_COREDNS_KEY	Corefile	Data key holding the Corefile.
--proxy-fqdn	OUROBOROS_CONTROLLER_PROXY_FQDN	ouroboros-proxy.ouroboros.svc.cluster.local.	Required for coredns mode. Must end with a trailing dot.
--etc-hosts	OUROBOROS_CONTROLLER_ETC_HOSTS	/host/etc/hosts	Path to host-mounted hosts file (etc-hosts mode).
--proxy-ip	OUROBOROS_CONTROLLER_PROXY_IP	(empty)	Required for etc-hosts mode.
--external-dns-namespace	OUROBOROS_CONTROLLER_EXTERNAL_DNS_NAMESPACE	(release namespace)	Where DNSEndpoint CRs are written (external-dns mode). Validated as RFC 1123 label only when explicitly set; the release-namespace fallback is already valid by definition.
--external-dns-record-ttl	OUROBOROS_CONTROLLER_EXTERNAL_DNS_RECORD_TTL	60	Record TTL on each emitted DNSEndpoint, [1, 86400] seconds.
--external-dns-proxy-ip	OUROBOROS_CONTROLLER_EXTERNAL_DNS_PROXY_IP	(empty)	Override target IP. Empty = discover via the named Service.
--external-dns-proxy-service	OUROBOROS_CONTROLLER_EXTERNAL_DNS_PROXY_SERVICE	ouroboros-proxy	Service name resolved at startup to ClusterIP.
--external-dns-annotation	(no env mapping; chart only)	(none)	Repeatable key=value annotations copied onto every emitted DNSEndpoint. Reserved keys are rejected at runtime.
--external-dns-label	(no env mapping; chart only)	(none)	Repeatable key=value labels copied onto every emitted DNSEndpoint. Use case: multi-instance external-dns with --label-filter (e.g. dedicated internal-DNS instance). Reserved keys (app.kubernetes.io/managed-by, ouroboros.lexfrei.tech/instance) are rejected.
--ingress-class	OUROBOROS_CONTROLLER_INGRESS_CLASS	(empty)	Filter Ingresses by spec.ingressClassName. Empty = all. Ingresses without an explicit class are dropped under the filter.
--gateway-class	OUROBOROS_CONTROLLER_GATEWAY_CLASS	(empty)	Filter Gateways by spec.gatewayClassName (and HTTPRoutes attached to surviving Gateways). Empty = all.

ouroboros proxy

Flag	Env var	Default
--listen-http	OUROBOROS_PROXY_LISTEN_HTTP	:8080
--listen-https	OUROBOROS_PROXY_LISTEN_HTTPS	:8443
--listen-health	OUROBOROS_PROXY_LISTEN_HEALTH	:8081
--target-host	OUROBOROS_PROXY_TARGET_HOST	ingress-nginx-controller.ingress-nginx.svc.cluster.local
--target-http-port	OUROBOROS_PROXY_TARGET_HTTP_PORT	80
--target-https-port	OUROBOROS_PROXY_TARGET_HTTPS_PORT	443
--dial-timeout	OUROBOROS_PROXY_DIAL_TIMEOUT	5s
--ready-timeout	OUROBOROS_PROXY_READY_TIMEOUT	2s
--shutdown-grace	OUROBOROS_PROXY_SHUTDOWN_GRACE	30s

Build

go build ./cmd/ouroboros
docker build --file Containerfile --tag ouroboros:dev .

Develop

go test -race -count=1 ./...
golangci-lint run ./...
helm lint   charts/ouroboros
helm unittest charts/ouroboros

License

BSD 3-Clause — see LICENSE.