Istio Node Readiness Controller

This package is used to add readiness taint to prevent race condition during critical daemonset pod installation

How it works

It will load the configmap defined by the user which contains the information of critical labels and their namespaces Controller will monitoring on all nodes and pods with critical labels and namespaces defined in the configuration map it will taint the node if some of the critical labels are not setup in current node and when all critical labels in given node is set up. it will untaint the node to allow non-critical pods to register

What it will do and not do

It is a complementary package to repair controller because repair controller itself cannot prevent daemonset failure and when istio-cni daemonset becomes unready, it is not able to install iptable rules to pods and introduce race condition thus it must work together with istio-cni-repair controller

It support much more generalized setting in node readiness checking, thus user can define their own configuration maps for more complicated readiness Check

How to use

create a configmap to define critical labels and their namespace

Configmap defines the namespace and label selector for critical pod, and in default it should be located in kube-system namespace with name node.readiness to let controller find them automatically An example of configmap Layout

config file

- name: istio-cni
  selector: app=istio
  namespace: kube-system

command to create the configmap sample output of configmap

apiVersion: v1
kind: ConfigMap
  name: "istio-cni-taint"
  namespace: "kube-system"
    app: istio-cni
  config: |
    - name: istio-cni
      selector: app=istio
      namespace: kube-system

supports multi label in one

apiVersion: v1
kind: ConfigMap
  name: "istio-cni-taint"
  namespace: "kube-system"
    app: istio-cni
  config: |
    - name: istio-cni
      selector: app=istio, app=istio-cni
      namespace: kube-system
config the critical pods and add node readiness tolerations to it
Kind: Daemonset
    Name: istio-critical-pod
        app: istio
        Key: NodeReadiness
        Operator: Exists
        Effect: NoSchedule
build it as binary

the command line interface is in cni/cmd/istio-cni-taint/main.go using command

make istioctl

it will generate the binary version of command-line interface controller

run command line interface for debugging and tests

find the istio-cni-taint binary in your output directory run the following command to start controller


If you want to customize nodes' readiness taint you should taint them by yourself

kubectl taint nodes <node-name> NodeReadiness:NoSchedule

and you need to set --register-with-taints option in kubelet to set readiness taint to newly added node

kubelet --register-with-taints=NodeReadiness:NoSchedule
Expand ▾ Collapse ▴




View Source
const (
	TaintName = "NodeReadiness"


This section is empty.


func GetNodeLatestReadiness

func GetNodeLatestReadiness(node v1.Node) bool

    node readiness validation by checking the last heartbeat status


    type ConfigSettings

    type ConfigSettings struct {
    	Name          string `yaml:"name"`
    	Namespace     string `yaml:"namespace"`
    	LabelSelector string `yaml:"selector"`

    func (ConfigSettings) String

    func (config ConfigSettings) String() string

    type Controller

    type Controller struct {
    	// contains filtered or unexported fields

    func NewTaintSetterController

    func NewTaintSetterController(ts *Setter) (*Controller, error)

    func (Controller) CheckNodeReadiness

    func (tc Controller) CheckNodeReadiness(node v1.Node) bool

    func (Controller) ListAllNode

    func (tc Controller) ListAllNode() []*v1.Node

    func (Controller) ProcessNode

    func (tc Controller) ProcessNode(node *v1.Node) error

      if node is ready, check all of its critical labels and if all of them are ready , remove readiness taint else taint it

      func (Controller) RegisterTaints

      func (tc Controller) RegisterTaints()

      func (*Controller) Run

      func (tc *Controller) Run(stopCh <-chan struct{})

        controller will run all of the critical pod controllers and node controllers, process node and pod in every second

        type Options

        type Options struct {
        	ConfigmapName      string
        	ConfigmapNamespace string

        type Setter

        type Setter struct {
        	Client client.Interface
        	// contains filtered or unexported fields

        func NewTaintSetter

        func NewTaintSetter(clientset client.Interface, options *Options) (ts *Setter, err error)

        func (*Setter) AddReadinessTaint

        func (ts *Setter) AddReadinessTaint(node *v1.Node) error

          taint node with specific taint name with effect of no schedule do nothing if it already have the readiness taint

          func (*Setter) Configs

          func (ts *Setter) Configs() []ConfigSettings

          func (*Setter) HasReadinessTaint

          func (ts *Setter) HasReadinessTaint(node *v1.Node) bool

            check whether current node have readiness

            func (*Setter) LoadConfig

            func (ts *Setter) LoadConfig(config v1.ConfigMap)

              load corresponding configmap's critical labels and their namespace

              func (*Setter) RemoveReadinessTaint

              func (ts *Setter) RemoveReadinessTaint(node *v1.Node) error

                assumption: order of taint is not important