controllers

Documentation

Index

• Constants
• Variables
• type ClickHouseController
  • func (r *ClickHouseController) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)
  • func (r *ClickHouseController) SetupWithManager(mgr ctrl.Manager) error
• type KubernetesResource
• type Reconciler
  • func (r *Reconciler) CreateOrUpdate(ctx context.Context, kr *KubernetesResource) (controllerutil.OperationResult, error)
  • func (r *Reconciler) ReconcileStatefulSets(ctx context.Context, req ctrl.Request, ch *clickHouseCluster) (*ctrl.Result, error)
  • func (r *Reconciler) ReconcileStorageSize(ctx context.Context, req ctrl.Request, ch *clickHouseCluster) (*ctrl.Result, error)

Constants

const (
	StatefulSetOwnerKey = ".metadata.controller"
	CHDataVolumeName    = "data"
)

Variables

var ErrDataPVCNotFound = fmt.Errorf("`data` PVC has not been found for given STS")

Types

type ClickHouseController

type ClickHouseController struct {
	client.Client
	// APIReader reads directly from the API server, bypassing the client read cache.
	APIReader client.Reader
	Log       logr.Logger
	Scheme    *runtime.Scheme
	Recorder  record.EventRecorder
}

ClickHouseController runs reconciliation of a ClickHouse object.

func (*ClickHouseController) Reconcile

func (r *ClickHouseController) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

For more details, check Reconcile and its Result here: - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.11.0/pkg/reconcile

func (*ClickHouseController) SetupWithManager

func (r *ClickHouseController) SetupWithManager(mgr ctrl.Manager) error

SetupWithManager sets up the controller with the Manager.

type KubernetesResource

type KubernetesResource struct {
	// contains filtered or unexported fields
}

KubernetesResource abstracts a K8s resource for reconciliation. The mutator is used in CreateOrUpdate to maintain desired state.

type Reconciler

type Reconciler struct {
	// contains filtered or unexported fields
}

Reconciler is used to reconcile any number of resources.

func (*Reconciler) CreateOrUpdate

func (r *Reconciler) CreateOrUpdate(
	ctx context.Context, kr *KubernetesResource) (controllerutil.OperationResult, error)

CreateOrUpdate will create or update the resource and set the controller ownership.

func (*Reconciler) ReconcileStatefulSets

func (r *Reconciler) ReconcileStatefulSets(
	ctx context.Context, req ctrl.Request, ch *clickHouseCluster) (*ctrl.Result, error)

ReconcileStatefulSets reconciles the cluster StatefulSets. This uses a rollout strategy of one-by-one if the StatefulSets are being updated. For new clusters, it will create the StatefulSets all at once.

func (*Reconciler) ReconcileStorageSize

func (r *Reconciler) ReconcileStorageSize(
	ctx context.Context,
	req ctrl.Request,
	ch *clickHouseCluster,
) (*ctrl.Result, error)

ReconcileStorageSize is the place where scaling CH cluster data volumes takes place.

The way we do resize is in line with what Altinity operator does:

https://github.com/Altinity/clickhouse-operator/blob/d7e654a341e4102038f6226febc338b8060c64dd/pkg/controller/chi/worker.go#L2074-L2102

and how k8s community does it in general:

https://itnext.io/resizing-statefulset-persistent-volumes-with-zero-downtime-916ebc65b1d4

No pods gets restarted so DB should keep working undisturbed during the whole process. The steps can be summarized as: * make sure that stateful sets are in steady state/Ready * remove stateful sets but without removing the pods themselves - orphan them * increase PVCs storage capacity * re-add stateful sets, allow k8s to adopt orphaned pods

Both STSes re-creation and PVC scaling is done by the same controller - clickhouse operator. Reconciliation for the given resource is a single threaded operation. I added one more stage in the pipeline that does the storage scalling in front of the previous reconciliation stages.

So it looks like this ATM: |-> reconciliation event starts -> STSes are removed (new code) -> PVCs are scaled (new code) -> STSes are re-created with the new PVC templates that match scaled PVCs (old code) -> reconciliation ends ->|

Decreasing size of storage is not supported. Scaling storage on Kind clusters is not supported:

https://github.com/rancher/local-path-provisioner/issues/18

GKE not only supports scaling storage but it also does it LIVE - without the need of restarting pods:

https://kubernetes.io/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim

StorageClass for the given PVC must have `allowVolumeExpansion: yes` for expansion to work. We do not check it in the controller and leave it to the operator. Controller will print appropriate message in the logs in such case.