Apache Pinot Kuberentes Operator
Project status: alpha Not all planned features are completed. The API, spec, status and other user facing objects may change, and not in a backward compatible way.
Documentation can be found here
Overview
The Pinot Operator provides Kubernetes native deployment and management of
Apache Prometheus and related components. The purpose of this project is to
simplify and automate the configuration of a Apache Pinot stack for Kubernetes clusters.
The Pinot operator includes, but is not limited to, the following features:
-
Kubernetes Custom Resources: Use Kubernetes custom resources to deploy and manage Apache Pnot and related components.
-
Simplified Deployment Configuration: Configure the fundamentals of Brokers/Controller/Server like versions, persistence,
retention policies, and replicas from a native Kubernetes resource.
The operator has been largely inspired by the BanzaiCloud istio-operator and the RabbitMQ cluster-operator. They are great resources to learn how to create operators.
Quickstart
If you have a running Kubernetes cluster and kubectl
configured to access it, run the following command to install the operator:
kubectl apply -f https://github.com/spaghettifunk/pinot-operator/releases/latest/download/pinot-cluster-operator.yaml
Then you can deploy a Pinot cluster:
kubectl apply -f https://raw.githubusercontent.com/spaghettifunk/pinot-operator/main/docs/examples/hello-world/pinot.yaml
How to develop
The operator is based on the kubebuilder
project and it has being scaffolded with it. To make it run, you need to do a few steps:
make generate
to generate the deepcopy
files
make manifests
to generate the correct CRDs
- Initiate KinD with
kind create cluster
. If you do not have KinD, check this page to install it
make install
to deploy the CRDs to your cluster
WATCH_NAMESPACE=pinot-system POD_NAMESPACE=pinot-system make run
to run the pinot-controller locally
If you want to stop the controller, press CTRL-C
and wait 30 seconds for the stop handler
to complete.
Local debugging
Sometimes you want to test things locally and not in an actual k8s cluster (either local or in the cloud). To avoid connectivity issues among services, you need to enable the do a couple of things:
- Do a
port-forward
towards the service you need (typically the pinot-controller
for accessing the APIs). For example, the command would look like the following :kubectl port-forward svc/pinot-controller-headless -n pinot-system 9000:9000
- Set up the
LOCAL_DEBUG
env variable to true
- Run the the local development command as follow
WATCH_NAMESPACE=pinot-system POD_NAMESPACE=pinot-system LOCAL_DEBUG=true make run
Versioning
Apache Pinot Kubernetes Operator follows non-strict semver.
The versioning guidelines document contains guidelines
on how we implement non-strict semver. The version number MAY or MAY NOT follow the semver rules. Hence, we highly recommend to read
the release notes to understand the changes and their potential impact for any release.
Contributing
This project follows the typical GitHub pull request model. Before starting any work, please either comment on an existing issue, or file a new one.
Please read contribution guidelines if you are interested in contributing to this project.
Gotchas
In case the CRD is stuck in delete phase, follow the steps in this workaround