cluster-api-provider-kind

README

cluster-api-provider-kind

Description

This is a controller/operator for KINDCluster custom resource. It assumes the cluster-api (capi) to be present in the cluster. It also watches for events coming from capi Cluster cr (in case of pause or unpause).

Very Quick Start

Add the provider:

$ cat ~/.cluster-api/clusterctl.yaml
providers:
  - name: "kind"
    url: "https://github.com/jkremser/cluster-api-provider-kind/releases/latest/infrastructure-components.yaml"
    type: "InfrastructureProvider"

clusterctl init --core cluster-api  --infrastructure kind
clusterctl generate cluster foo --kubernetes-version=1.24.0 -i kind | kubectl apply -f -

Getting Started (dev)

clusterctl init  --core cluster-api:v1.2.0  --infrastructure kind:v0.1.0  --config ~/.cluster-api/dev-repository/config.yaml
# create the config.yaml with overrides by following https://cluster-api.sigs.k8s.io/clusterctl/configuration.html#clusterctl-configuration-file

k logs -f -lcontrol-plane=controller-manager -n capi-system

k scale deployment cluster-api-provider-kind-controller-manager --replicas=0 -n cluster-api-provider-kind-system

make run

clusterctl generate cluster foo --kubernetes-version 1.24.0 --config ~/.cluster-api/dev-repository/config.yaml -i kind | kubectl apply -f -

clusterctl generate cluster foo --kubernetes-version 1.24.0 --config ~/.cluster-api/dev-repository/config.yaml -i kind | kubectl delete -f -

Advanced setup

By default, socket for docker is in /run/docker.socket however, mounting the whole /run directory from host to container isn't good idea. So let's use a socket that's isolated in its own directory.

# to be able to run capi-kind in capi-kind in capi-kind ... in capi-kind, you have to do:
# if you are ok with a "flat" structure, skip this and use the images w/o -socket suffix
mkdir -p /mnt/host
sed -i'' 's;/run/docker.sock;/mnt/host/docker.sock;g' /etc/systemd/system/sockets.target.wants/docker.socket
sudo systemctl daemon-reload
sudo systemctl restart docker.socket
docker context create mnt-ctx --description "description" --docker host=unix:///mnt/host/docker.sock
docker context use mnt-ctx

nice to have

• k8s version -> image for kind mapping
• KINDMachine + controller combo and letting the bootstrap for another provider (craeteoption CreateWithStopBeforeSettingUpKubernetes)
• somehow propagate the logs from KIND to cluster operator/user

Troubleshooting

Unable to create Kind multi-node cluster

Check if the kube-proxy is starting on the control plane and its logs, if you encounter this message:

E0718 11:31:46.677362       1 run.go:74] "command failed" err="failed complete: too many open files"

run

sudo sysctl fs.inotify.max_user_watches=524288
sudo sysctl fs.inotify.max_user_instances=256

# for persistency put them to /etc/sysctl.d/99-sysctl.conf

When hitting the rate limit for github api has been reached during clusterctl calls, just export the GITHUB_TOKEN with your token.