README

Istio Node Readiness Controller

This package is used to add readiness taint to prevent race condition during critical daemonset pod installation

How it works

It will load the configmap defined by the user which contains the information of critical labels and their namespaces Controller will monitoring on all nodes and pods with critical labels and namespaces defined in the configuration map it will taint the node if some of the critical labels are not setup in current node and when all critical labels in given node is set up. it will untaint the node to allow non-critical pods to register

What it will do and not do

It is a complementary package to repair controller because repair controller itself cannot prevent daemonset failure and when istio-cni daemonset becomes unready, it is not able to install iptable rules to pods and introduce race condition thus it must work together with istio-cni-repair controller

It support much more generalized setting in node readiness checking, thus user can define their own configuration maps for more complicated readiness Check

How to use

create a configmap to define critical labels and their namespace

Configmap defines the namespace and label selector for critical pod, and in default it should be located in kube-system namespace with name node.readiness to let controller find them automatically An example of configmap Layout

./
configs/
   config

config file

- name: istio-cni
  selector: app=istio
  namespace: kube-system

command to create the configmap sample output of configmap

apiVersion: v1
kind: ConfigMap
metadata:
  name: "istio-cni-taint"
  namespace: "kube-system"
  labels:
    app: istio-cni
data:
  config: |
    - name: istio-cni
      selector: app=istio
      namespace: kube-system

supports multi label in one

apiVersion: v1
kind: ConfigMap
metadata:
  name: "istio-cni-taint"
  namespace: "kube-system"
  labels:
    app: istio-cni
data:
  config: |
    - name: istio-cni
      selector: app=istio, app=istio-cni
      namespace: kube-system
config the critical pods and add node readiness tolerations to it
Kind: Daemonset
Metadata:
    Name: istio-critical-pod
    Labels:
        app: istio
Spec:
  ...more...
    Toleration:
        Key: NodeReadiness
        Operator: Exists
        Effect: NoSchedule
build it as binary

the command line interface is in cni/cmd/istio-cni-taint/main.go using command

make istioctl

it will generate the binary version of command-line interface controller

run command line interface for debugging and tests

find the istio-cni-taint binary in your output directory run the following command to start controller

istio-cni-taint

If you want to customize nodes' readiness taint you should taint them by yourself

kubectl taint nodes <node-name> NodeReadiness:NoSchedule

and you need to set --register-with-taints option in kubelet to set readiness taint to newly added node

kubelet --register-with-taints=NodeReadiness:NoSchedule
Expand ▾ Collapse ▴

Documentation

Index

Constants

View Source 
const (
	TaintName = "NodeReadiness"
)

Variables

This section is empty.

Functions

func GetNodeLatestReadiness 

func GetNodeLatestReadiness(node v1.Node) bool

    node readiness validation by checking the last heartbeat status

    Types

    type ConfigSettings 

    type ConfigSettings struct {
	Name          string `yaml:"name"`
	Namespace     string `yaml:"namespace"`
	LabelSelector string `yaml:"selector"`
}

    func (ConfigSettings) String 

    func (config ConfigSettings) String() string

    type Controller 

    type Controller struct {
	// contains filtered or unexported fields
}

    func NewTaintSetterController 

    func NewTaintSetterController(ts *Setter) (*Controller, error)

    func (Controller) CheckNodeReadiness 

    func (tc Controller) CheckNodeReadiness(node v1.Node) bool

    func (Controller) ListAllNode 

    func (tc Controller) ListAllNode() []*v1.Node

    func (Controller) ProcessNode 

    func (tc Controller) ProcessNode(node *v1.Node) error

      if node is ready, check all of its critical labels and if all of them are ready , remove readiness taint else taint it

      func (Controller) RegisterTaints 

      func (tc Controller) RegisterTaints()

      func (*Controller) Run 

      func (tc *Controller) Run(stopCh <-chan struct{})

        controller will run all of the critical pod controllers and node controllers, process node and pod in every second

        type Options 

        type Options struct {
	ConfigmapName      string
	ConfigmapNamespace string
}

        type Setter 

        type Setter struct {
	Client client.Interface
	// contains filtered or unexported fields
}

        func NewTaintSetter 

        func NewTaintSetter(clientset client.Interface, options *Options) (ts *Setter, err error)

        func (*Setter) AddReadinessTaint 

        func (ts *Setter) AddReadinessTaint(node *v1.Node) error

          taint node with specific taint name with effect of no schedule do nothing if it already have the readiness taint

          func (*Setter) Configs 

          func (ts *Setter) Configs() []ConfigSettings

          func (*Setter) HasReadinessTaint 

          func (ts *Setter) HasReadinessTaint(node *v1.Node) bool

            check whether current node have readiness

            func (*Setter) LoadConfig 

            func (ts *Setter) LoadConfig(config v1.ConfigMap)

              load corresponding configmap's critical labels and their namespace

              func (*Setter) RemoveReadinessTaint 

              func (ts *Setter) RemoveReadinessTaint(node *v1.Node) error

                assumption: order of taint is not important

                Source Files

                View all