kubekey

command module
v3.1.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 15, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

README ΒΆ

CI

English | δΈ­ζ–‡

πŸ‘‹ Welcome to KubeKey!

KubeKey is an open-source lightweight tool for deploying Kubernetes clusters. It provides a flexible, rapid, and convenient way to install Kubernetes/K3s only, both Kubernetes/K3s and KubeSphere, and related cloud-native add-ons. It is also an efficient tool to scale and upgrade your cluster.

In addition, KubeKey also supports customized Air-Gap package, which is convenient for users to quickly deploy clusters in offline environments.

KubeKey has passed CNCF kubernetes conformance verification.

Use KubeKey in the following three scenarios.

  • Install Kubernetes/K3s only
  • Install Kubernetes/K3s and KubeSphere together in one command
  • Install Kubernetes/K3s first, then deploy KubeSphere on it using ks-installer

Important: If you have existing Kubernetes clusters, please refer to ks-installer (Install KubeSphere on existing Kubernetes cluster).

Supported Environment

Linux Distributions

  • Ubuntu 16.04, 18.04, 20.04, 22.04
  • Debian Bullseye, Buster, Stretch
  • CentOS/RHEL 7
  • AlmaLinux 9.0
  • SUSE Linux Enterprise Server 15

Recommended Linux Kernel Version: 4.15 or later You can run the uname -srm command to check the Linux Kernel Version.

Kubernetes Versions

  • v1.19:   v1.19.15
  • v1.20:   v1.20.10
  • v1.21:   v1.21.14
  • v1.22:   v1.22.15
  • v1.23:   v1.23.10 (default)
  • v1.24:   v1.24.7
  • v1.25:   v1.25.3

Looking for more supported versions:
Kubernetes Versions
K3s Versions

Container Manager

  • Docker / containerd / CRI-O / iSula

Kata Containers can be set to automatically install and configure runtime class for it when the container manager is containerd or CRI-O.

Network Plugins

  • Calico / Flannel / Cilium / Kube-OVN / Multus-CNI

Kubekey also supports users to set the network plugin to none if there is a requirement for custom network plugin.

Requirements and Recommendations

  • Minimum resource requirements (For Minimal Installation of KubeSphere only):
    • 2 vCPUs
    • 4 GB RAM
    • 20 GB Storage

/var/lib/docker is mainly used to store the container data, and will gradually increase in size during use and operation. In the case of a production environment, it is recommended that /var/lib/docker mounts a drive separately.

  • OS requirements:
    • SSH can access to all nodes.
    • Time synchronization for all nodes.
    • sudo/curl/openssl should be used in all nodes.
    • docker can be installed by yourself or by KubeKey.
    • Red Hat includes SELinux in its Linux release. It is recommended to close SELinux or switch the mode of SELinux to Permissive
  • It's recommended that Your OS is clean (without any other software installed), otherwise there may be conflicts.
  • A container image mirror (accelerator) is recommended to be prepared if you have trouble downloading images from dockerhub.io. Configure registry-mirrors for the Docker daemon.
  • KubeKey will install OpenEBS to provision LocalPV for development and testing environment by default, this is convenient for new users. For production, please use NFS / Ceph / GlusterFS or commercial products as persistent storage, and install the relevant client in all nodes.
  • If you encounter Permission denied when copying, it is recommended to check SELinux and turn off it first
  • Dependency requirements:

KubeKey can install Kubernetes and KubeSphere together. Some dependencies need to be installed before installing kubernetes after version 1.18. You can refer to the list below to check and install the relevant dependencies on your node in advance.

Kubernetes Version β‰₯ 1.18
socat Required
conntrack Required
ebtables Optional but recommended
ipset Optional but recommended
ipvsadm Optional but recommended
  • Networking and DNS requirements:
    • Make sure the DNS address in /etc/resolv.conf is available. Otherwise, it may cause some issues of DNS in cluster.
    • If your network configuration uses Firewall or Security Group,you must ensure infrastructure components can communicate with each other through specific ports. It's recommended that you turn off the firewall or follow the link configuriation: NetworkAccess.

Usage

Get the KubeKey Executable File

  • The fastest way to get KubeKey is to use the script:

    curl -sfL https://get-kk.kubesphere.io | sh -
    
  • Binary downloads of the KubeKey also can be found on the Releases page. Unpack the binary and you are good to go!

  • Build Binary from Source Code

    git clone https://github.com/kubesphere/kubekey.git
    cd kubekey
    make kk
    

Create a Cluster

Quick Start

Quick Start is for all-in-one installation which is a good start to get familiar with Kubernetes and KubeSphere.

Note: Since Kubernetes temporarily does not support uppercase NodeName, contains uppercase letters in the hostname will lead to subsequent installation error

Command

If you have problem to access https://storage.googleapis.com, execute first export KKZONE=cn.

./kk create cluster [--with-kubernetes version] [--with-kubesphere version]
Examples
  • Create a pure Kubernetes cluster with default version (Kubernetes v1.23.10).

    ./kk create cluster
    
  • Create a Kubernetes cluster with a specified version.

    ./kk create cluster --with-kubernetes v1.24.1 --container-manager containerd
    
  • Create a Kubernetes cluster with KubeSphere installed.

    ./kk create cluster --with-kubesphere v3.2.1
    

Advanced

You have more control to customize parameters or create a multi-node cluster using the advanced installation. Specifically, create a cluster by specifying a configuration file.

If you have problem to access https://storage.googleapis.com, execute first export KKZONE=cn.

  1. First, create an example configuration file

    ./kk create config [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path]
    

    examples:

    • create an example config file with default configurations. You also can specify the file that could be a different filename, or in different folder.
    ./kk create config [-f ~/myfolder/abc.yaml]
    
    • with KubeSphere
    ./kk create config --with-kubesphere v3.2.1
    
  2. Modify the file config-sample.yaml according to your environment

Note: Since Kubernetes temporarily does not support uppercase NodeName, contains uppercase letters in workerNode`s name will lead to subsequent installation error

A persistent storage is required in the cluster, when kubesphere will be installed. The local volume is used default. If you want to use other persistent storage, please refer to addons.

  1. Create a cluster using the configuration file

    ./kk create cluster -f config-sample.yaml
    

Enable Multi-cluster Management

By default, KubeKey will only install a solo cluster without Kubernetes federation. If you want to set up a multi-cluster control plane to centrally manage multiple clusters using KubeSphere, you need to set the ClusterRole in config-example.yaml. For multi-cluster user guide, please refer to How to Enable the Multi-cluster Feature.

Enable Pluggable Components

KubeSphere has decoupled some core feature components since v2.1.0. These components are designed to be pluggable which means you can enable them either before or after installation. By default, KubeSphere will be started with a minimal installation if you do not enable them.

You can enable any of them according to your demands. It is highly recommended that you install these pluggable components to discover the full-stack features and capabilities provided by KubeSphere. Please ensure your machines have sufficient CPU and memory before enabling them. See Enable Pluggable Components for the details.

Add Nodes

Add new node's information to the cluster config file, then apply the changes.

./kk add nodes -f config-sample.yaml

Delete Nodes

You can delete the node by the following command,the nodeName that needs to be removed.

./kk delete node <nodeName> -f config-sample.yaml

Delete Cluster

You can delete the cluster by the following command:

  • If you started with the quick start (all-in-one):
./kk delete cluster
  • If you started with the advanced (created with a configuration file):
./kk delete cluster [-f config-sample.yaml]

Upgrade Cluster

Allinone

Upgrading cluster with a specified version.

./kk upgrade [--with-kubernetes version] [--with-kubesphere version] 
  • Support upgrading Kubernetes only.
  • Support upgrading KubeSphere only.
  • Support upgrading Kubernetes and KubeSphere.

Multi-nodes

Upgrading cluster with a specified configuration file.

./kk upgrade [--with-kubernetes version] [--with-kubesphere version] [(-f | --filename) path]
  • If --with-kubernetes or --with-kubesphere is specified, the configuration file will be also updated.
  • Use -f to specify the configuration file which was generated for cluster creation.

Note: Upgrading multi-nodes cluster need a specified configuration file. If the cluster was installed without kubekey or the configuration file for installation was not found, the configuration file needs to be created by yourself or following command.

Getting cluster info and generating kubekey's configuration file (optional).

./kk create config [--from-cluster] [(-f | --filename) path] [--kubeconfig path]
  • --from-cluster means fetching cluster's information from an existing cluster.
  • -f refers to the path where the configuration file is generated.
  • --kubeconfig refers to the path where the kubeconfig.
  • After generating the configuration file, some parameters need to be filled in, such as the ssh information of the nodes.

Documents

Contributors ✨

Thanks goes to these wonderful people (emoji key):

pixiake
pixiake

πŸ’» πŸ“–
Forest
Forest

πŸ’» πŸ“–
rayzhou2017
rayzhou2017

πŸ’» πŸ“–
shaowenchen
shaowenchen

πŸ’» πŸ“–
Zhao Xiaojie
Zhao Xiaojie

πŸ’» πŸ“–
Zack Zhang
Zack Zhang

πŸ’»
Akhil Mohan
Akhil Mohan

πŸ’»
pengfei
pengfei

πŸ“–
min zhang
min zhang

πŸ’» πŸ“–
zgldh
zgldh

πŸ’»
xrjk
xrjk

πŸ’»
yonghongshi
yonghongshi

πŸ’»
Honglei
Honglei

πŸ“–
liucy1983
liucy1983

πŸ’»
Lien
Lien

πŸ“–
Tony Wang
Tony Wang

πŸ“–
Hongliang Wang
Hongliang Wang

πŸ’»
dawn
dawn

πŸ’»
Duan Jiong
Duan Jiong

πŸ’»
calvinyv
calvinyv

πŸ“–
Benjamin Huo
Benjamin Huo

πŸ“–
Sherlock113
Sherlock113

πŸ“–
fu_changjie
fu_changjie

πŸ“–
yuswift
yuswift

πŸ’»
ruiyaoOps
ruiyaoOps

πŸ“–
LXM
LXM

πŸ“–
sbhnet
sbhnet

πŸ’»
misteruly
misteruly

πŸ’»
John Niang
John Niang

πŸ“–
Michael Li
Michael Li

πŸ’»
η‹¬ε­€ζ˜Šε€©
η‹¬ε­€ζ˜Šε€©

πŸ’»
Liu Shaohui
Liu Shaohui

πŸ’»
Leo Li
Leo Li

πŸ’»
Roland
Roland

πŸ’»
Vinson Zou
Vinson Zou

πŸ“–
tag_gee_y
tag_gee_y

πŸ’»
codebee
codebee

πŸ’»
Daniel Owen van Dommelen
Daniel Owen van Dommelen

πŸ€”
Naidile P N
Naidile P N

πŸ’»
Haiker Sun
Haiker Sun

πŸ’»
Jing Yu
Jing Yu

πŸ’»
Chauncey
Chauncey

πŸ’»
Tan Guofu
Tan Guofu

πŸ’»
lvillis
lvillis

πŸ“–
Vincent He
Vincent He

πŸ’»
laminar
laminar

πŸ’»
tongjin
tongjin

πŸ’»
Reimu
Reimu

πŸ’»
Ikko Ashimine
Ikko Ashimine

πŸ“–
Ben Ye
Ben Ye

πŸ’»
yinheli
yinheli

πŸ’»
hellocn9
hellocn9

πŸ’»
Brandan Schmitz
Brandan Schmitz

πŸ’»
yjqg6666
yjqg6666

πŸ“– πŸ’»
ε€±ηœ ζ˜―ηœŸζ»΄ιšΎε—
ε€±ηœ ζ˜―ηœŸζ»΄ιšΎε—

πŸ’»
mango
mango

πŸ‘€
wenwutang
wenwutang

πŸ’»
Shiny Hou
Shiny Hou

πŸ’»
zhouqiu0103
zhouqiu0103

πŸ’»
77yu77
77yu77

πŸ’»
hzhhong
hzhhong

πŸ’»
zhang-wei
zhang-wei

πŸ’»
Deshi Xiao
Deshi Xiao

πŸ’» πŸ“–
besscroft
besscroft

πŸ“–
εΌ εΏ—εΌΊ
εΌ εΏ—εΌΊ

πŸ’»
lwabish
lwabish

πŸ’» πŸ“–
qyz87
qyz87

πŸ’»
ZhengJin Fang
ZhengJin Fang

πŸ’»
Eric_Lian
Eric_Lian

πŸ’»
nicognaw
nicognaw

πŸ’»
吕德庆
吕德庆

πŸ’»
littleplus
littleplus

πŸ’»
Konstantin
Konstantin

πŸ€”
kiragoo
kiragoo

πŸ’»
jojotong
jojotong

πŸ’»
littleBlackHouse
littleBlackHouse

πŸ’» πŸ“–
guangwu
guangwu

πŸ’» πŸ“–
wongearl
wongearl

πŸ’»
wenwenxiong
wenwenxiong

πŸ’»
ζŸε–΅Sakura
ζŸε–΅Sakura

πŸ’»
cui fliter
cui fliter

πŸ“–
εˆ˜ζ—­
εˆ˜ζ—­

πŸ’»
yuyu
yuyu

πŸ’»
chilianyi
chilianyi

πŸ’»
Ronald Fletcher
Ronald Fletcher

πŸ’»

This project follows the all-contributors specification. Contributions of any kind welcome!

Documentation ΒΆ

Overview ΒΆ

Package main

Directories ΒΆ

Path Synopsis
api
v1beta1
Package v1beta1 contains the v1beta1 API implementation.
Package v1beta1 contains the v1beta1 API implementation.
bootstrap
k3s
Package main
Package main
k3s/api/v1beta1
Package v1beta1 contains API Schema definitions for the bootstrap v1beta1 API group +kubebuilder:object:generate=true +groupName=bootstrap.cluster.x-k8s.io
Package v1beta1 contains API Schema definitions for the bootstrap v1beta1 API group +kubebuilder:object:generate=true +groupName=bootstrap.cluster.x-k8s.io
k3s/controllers
Package controllers contains k3s config controllers.
Package controllers contains k3s config controllers.
k3s/pkg/cloudinit
Package cloudinit implements kubeadm cloudinit functionality.
Package cloudinit implements kubeadm cloudinit functionality.
k3s/pkg/locking
Package locking implements locking functionality.
Package locking implements locking functionality.
k3s/pkg/types
Package types contains k3s config types.
Package types contains k3s config types.
cmd
kk
Package controllers implements controllers.
Package controllers implements controllers.
kkcluster
Package kkcluster implements kkcluster controllers.
Package kkcluster implements kkcluster controllers.
kkinstance
Package kkinstance implements kkinstance controllers.
Package kkinstance implements kkinstance controllers.
kkmachine
Package kkmachine implements kkmachine controllers.
Package kkmachine implements kkmachine controllers.
controlplane
k3s
Package main
Package main
k3s/api/v1beta1
Package v1beta1 contains API Schema definitions for the controlplane v1beta1 API group +kubebuilder:object:generate=true +groupName=controlplane.cluster.x-k8s.io
Package v1beta1 contains API Schema definitions for the controlplane v1beta1 API group +kubebuilder:object:generate=true +groupName=controlplane.cluster.x-k8s.io
k3s/controllers
Package controllers contains k3s control plane controller.
Package controllers contains k3s control plane controller.
k3s/pkg/cluster
Package cluster contains internal implementation details for the k3s Control Plane.
Package cluster contains internal implementation details for the k3s Control Plane.
k3s/pkg/kubeconfig
Package kubeconfig implements the kubeconfig generation logic.
Package kubeconfig implements the kubeconfig generation logic.
pkg
Package pkg defines the interface for a cluster scope.
Package pkg defines the interface for a cluster scope.
clients/ssh
Package ssh implements the ssh client.
Package ssh implements the ssh client.
rootfs
Package rootfs defines the CAPKK rootfs.
Package rootfs defines the CAPKK rootfs.
scope
Package scope defines the basic context for an actuator to operate upon.
Package scope defines the basic context for an actuator to operate upon.
service
Package service implements various services.
Package service implements various services.
service/binary
Package binary define the binaries operations on the remote instance.
Package binary define the binaries operations on the remote instance.
service/binary/k3s
Package k3s define the binaries operations on the remote instance.
Package k3s define the binaries operations on the remote instance.
service/binary/kubernetes
Package kubernetes define the binaries operations on the remote instance.
Package kubernetes define the binaries operations on the remote instance.
service/bootstrap
Package bootstrap defines the CAPKK bootstrap operations on the remote instance.
Package bootstrap defines the CAPKK bootstrap operations on the remote instance.
service/containermanager
Package containermanager defines the operations on the remote instance's CRI.
Package containermanager defines the operations on the remote instance's CRI.
service/operation
Package operation define the remote instance operations interface.
Package operation define the remote instance operations interface.
service/operation/directory
Package directory defines the operations on remote instance directory.
Package directory defines the operations on remote instance directory.
service/operation/file
Package file defines the operations on the binaries.
Package file defines the operations on the binaries.
service/operation/file/checksum
Package checksum implements methods for obtaining and verifying the checksum of binary files.
Package checksum implements methods for obtaining and verifying the checksum of binary files.
service/operation/repository
Package repository defines the operations on the remote instance linux software packages repository.
Package repository defines the operations on the remote instance linux software packages repository.
service/operation/user
Package user defines the user operations on the remote instance.
Package user defines the user operations on the remote instance.
service/provisioning
Package provisioning defines the provisioning operations on the remote instance generated by cloudinit or ignition.
Package provisioning defines the provisioning operations on the remote instance generated by cloudinit or ignition.
service/provisioning/cloudinit
Package cloudinit defines cloud init adapter for existing nodes.
Package cloudinit defines cloud init adapter for existing nodes.
service/provisioning/commands
Package commands deals with various machine initialization methods viz.
Package commands deals with various machine initialization methods viz.
service/repository
Package repository contains the repository service.
Package repository contains the repository service.
service/util
Package util contains utility functions for the service.
Package util contains utility functions for the service.
util
Package util implements utilities.
Package util implements utilities.
util/filesystem
Package filesystem implements the operation of the local file system.
Package filesystem implements the operation of the local file system.
util/hash
Package hash implements hash utilities
Package hash implements hash utilities
test
e2e
Package e2e implements end to end testing.
Package e2e implements end to end testing.
e2e/framework
Package framework implements end to end testing.
Package framework implements end to end testing.
e2e/framework/bootstrap
Package bootstrap implements bootstrap functionality for e2e testing.
Package bootstrap implements bootstrap functionality for e2e testing.
e2e/framework/clusterctl
Package clusterctl implements clusterctl interaction.
Package clusterctl implements clusterctl interaction.
e2e/framework/clusterctl/logger
Package logger implements clusterctl logging functionality.
Package logger implements clusterctl logging functionality.
e2e/framework/internal/log
Package log implements test framework logging.
Package log implements test framework logging.
e2e/suites/capkk
Package capkk implements end to end testing.
Package capkk implements end to end testing.
e2e/suites/k3s
Package k3s implements k3s end to end testing.
Package k3s implements k3s end to end testing.
Package util contains utility functions
Package util contains utility functions
collections
Package collections implements collection utilities.
Package collections implements collection utilities.
log
Package log provides log utils.
Package log provides log utils.
osrelease
Package osrelease is to parse a os release file content.
Package osrelease is to parse a os release file content.
secret
Package secret provides a secret store for storing secrets.
Package secret provides a secret store for storing secrets.
Package version implements version handling code.
Package version implements version handling code.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL