k8router

README

k8router

This is a software project by SOSETH and VIS to manage several external load-balancers which forward traffic to several Kubernetes clusters.

Externally expose kubernetes ingresses using HAProxy and terminate TLS for them.

Problem Description

Given a set of n Kubernetes clusters and m pairs of load-balancers to each of them, the goal of the project is to transport information about available domains in each cluster to all load-balancers such that any router is able to forward traffic any cluster.

Open Tasks

• Figure out why update/patch on the fake client set doesn't get propagated. This prevents us from writing more interesting unit tests...
• systemd integration (instead of doing sudo systemctl ...)

Prerequisites

• HAProxy, installed and configured to use a conf.d-style configuration format. Have a look here for more information.
• Kubernetes clusters and client configurations for a k8router user.
  • The user must be able to watch Ingresses in all namespaces in all clusters. See k8s-rbac.yml for more information.
• Certificates for all your domains.
• sysctl net.ipv4.vs.conntrack = 1
• Source-NAT rule for the service IP subnet

Each Kubernetes cluster has to expose its API to all the routers. Every kubelet node has to be accessible by all routers.

Configuration

An example configuration might look like this:

haproxyDropinPath: /etc/haproxy/conf.d/90-k8router.conf
haproxyTemplatePath: /root/template
clusters:
  - name: local
    kubeconfig: /etc/k8router/k8s/kubeconfig.yml
certificates:
  - cert: /foo
    name: realcert
    domains:
      - example.org
  - cert: /bar
    name: dummycert
    domains:
      - '*.org'
      - '*.com'
ips:
  - 1.2.3.4

This will generate a configuration at /etc/haproxy/conf.d/90-k8router.conf from /root/template for one cluster (/etc/k8router/k8s/kubeconfig.yml), using two certificates and one external IP. An example template file is included here, note that the certificates are specified as directories (see the HAProxy docs on this one).

Running

Execute ./k8router -verbose -config <path/to/config> in a terminal, the log output should tell you if something goes wrong. Due to missing systemd integration we still require passwordless sudo for the service user.

License