wildfly-operator

README

image:https://github.com/wildfly/wildfly-operator/actions/workflows/go.yml/badge.svg?branch=main["Go",link="https://github.com/wildfly/wildfly-operator/actions/workflows/go.yml"]
image:https://quay.io/repository/wildfly/wildfly-operator/status?token=c568da69-fcdb-4b58-8821-5e98f6522290["Docker Repository on Quay", link="https://quay.io/repository/wildfly/wildfly-operator"]
image:https://goreportcard.com/badge/github.com/wildfly/wildfly-operator["Go Report Card", link="https://goreportcard.com/report/github.com/wildfly/wildfly-operator"]
image:https://img.shields.io/badge/zulip-join_chat-brightgreen.svg["Join Chat", link="https://wildfly.zulipchat.com/"]

# WildFly Operator for Kubernetes/OpenShift

The WildFly Operator for Kubernetes provides easy monitoring and configuration for Java applications deployed on http://wildfly.org[WildFly application server] using the https://github.com/wildfly/wildfly-s2i[Source-to-Image (S2I) template for WildFly].

Once installed, the WildFly Operator provides the following features:

* Create/Destroy: Easily launch an application deployed on WildFly

* Simple Configuration: Configure the fundamentals of WildFly-based application including number of nodes, application image, etc.

## Custom Resource Definitions

The operator acts on the following Custom Resource Definitions (CRDs):

* `WildFlyServer`, which defines a WildFly deployment. The `Spec` and `Status` of this resources are defined in the https://github.com/wildfly/wildfly-operator/blob/main/doc/apis.adoc[API documentation].

## Quickstart

### Install the Operator and associate resources:

The examples require that https://kubernetes.io/docs/setup/minikube/[Minikube] is installed and running.

[source,shell]
----
# install WildFlyServer CRD
$ make install
# Install all resources for the WildFly Operator
$ make deploy
----

### Install a custom resource

An example of a custom resource of `WildFlyServer` is described in https://github.com/wildfly/wildfly-operator/blob/main/config/samples/quickstart-cr.yaml[quickstart-cr.yaml]:

[source,yaml]
----
apiVersion: wildfly.org/v1alpha1
kind: WildFlyServer
metadata:
  name: quickstart
spec:
  applicationImage: "quay.io/wildfly-quickstarts/wildfly-operator-quickstart:18.0"
  replicas: 2
  storage:
    volumeClaimTemplate:
      spec:
        resources:
          requests:
            storage: 3Gi
----

[NOTE]
=====
It is based on the S2I application image https://quay.io/repository/wildfly-quickstarts/wildfly-operator-quickstart[jmesnil/wildfly-operator-quickstart:18.0] that provides a simple Java Web application https://github.com/jmesnil/wildfly-operator-quickstart[wildfly-operator-quickstart] on top of WildFly 18.0.0.Final which returns the IP address of its host:

[source,shell]
----
$ curl http://localhost:8080/
{"ip":"172.17.0.3"}
----

This simple application illustrates that successive calls will be load balanced across the various pods that runs the application.
=====

[source,shell]
----
$ kubectl create -f config/samples/quickstart-cr.yaml
wildflyserver.wildfly.org/quickstart created
----

Once the application is deployed, it can be accessed through a load balancer:

[source,shell]
----
$ curl $(minikube service quickstart-loadbalancer --url)
{"ip":"172.17.0.7"}
$ curl $(minikube service quickstart-loadbalancer --url)
{"ip":"172.17.0.8"}
$ curl $(minikube service quickstart-loadbalancer --url)
{"ip":"172.17.0.7"}
----

As illustrated above, calls to the application are load balanced across the pods that runs the application image (as we can see from the different IP addresses).

The WildFly operator describes the deployed application with `$ kubectl describe wildflyserver quickstart`:

[source,yaml]
----
Name:         quickstart
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  wildfly.org/v1alpha1
Kind:         WildFlyServer
Metadata:
  Creation Timestamp:  2019-04-09T08:49:24Z
  Generation:          1
  Resource Version:    7954
  Self Link:           /apis/wildfly.org/v1alpha1/namespaces/default/wildflyservers/quickstart
  UID:                 5feb0fd3-5aa4-11e9-af00-7a65e1e4ff53
Spec:
  Application Image:  quay.io/wildfly-quickstarts/wildfly-operator-quickstart:18.0
  Bootable Jar:       false
  Replicas:           2
  Storage:
    Volume Claim Template:
      Spec:
        Resources:
          Requests:
            Storage:  3Gi
Status:
  Pods:
    Name:    quickstart-0
    Pod IP:  172.17.0.7
    Name:    quickstart-1
    Pod IP:  172.17.0.8
Events:      <none>
----

The `Status` section is updated with the 2 pods names containing the application image.

You can modify this custom resource spec to scale up its replicas from `2` to `3`:

[source,shell]
----
$ kubectl edit wildflyserver quickstart
# Change the `replicas: 2` spec to `replicas: 3` and save

wildflyserver.wildfly.org/quickstart edited
----

The deployment will be updated to scale up to 3 Pods and the resource `Status` will be updated accordingly:

[source,shell]
----
$ kubectl describe wildflyserver quickstart
----

[source,yaml]
----
Name:         quickstart
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  wildfly.org/v1alpha1
Kind:         WildFlyServer
Metadata:
  Creation Timestamp:  2019-04-09T08:49:24Z
  Generation:          2
  Resource Version:    8137
  Self Link:           /apis/wildfly.org/v1alpha1/namespaces/default/wildflyservers/quickstart
  UID:                 5feb0fd3-5aa4-11e9-af00-7a65e1e4ff53
Spec:
  Application Image:  quay.io/wildfly-quickstarts/wildfly-operator-quickstart:18.0
  Bootable Jar:       false
  Replicas:           3
  Storage:
    Volume Claim Template:
      Spec:
        Resources:
          Requests:
            Storage:  3Gi
Status:
  Pods:
    Name:    quickstart-0
    Pod IP:  172.17.0.7
    Name:    quickstart-1
    Pod IP:  172.17.0.8
    Name:    quickstart-2
    Pod IP:  172.17.0.9
Events:      <none>
----

You can then remove this custom resource and its associated resources:

[source,shell]
----
$ kubectl delete wildflyserver quickstart

wildflyserver.wildfly.org "quickstart" deleted
----

You can remove the WildFly Operator resources:
[source,shell]
----
$ make undeploy
customresourcedefinition.apiextensions.k8s.io "wildflyservers.wildfly.org" deleted
serviceaccount "wildfly-operator" deleted
role.rbac.authorization.k8s.io "wildfly-operator" deleted
rolebinding.rbac.authorization.k8s.io "wildfly-operator" deleted
deployment.apps "wildfly-operator" deleted
----

#### OpenShift

The examples can also be installed in OpenShift and requires a few additional steps.

The instructions require that https://github.com/minishift/minishift[Minishift] is installed and running.

Deploying the operator and its resources by executing the following commands:

[source,shell]
----
$ oc login -u system:admin
$ oc adm policy add-cluster-role-to-user cluster-admin developer
$ make install
$ make deploy

$ oc login -u developer
----

When a `WildFlyServer` resource is installed from `config/samples/quickstart-cr.yaml`, a `route` is automatically created to expose the application. To know the URL of the exposed service, run:

[source,shell]
----
$ oc get route quickstart-loadbalancer --template='{{ .spec.host }}'
----

This will display the host of the route (on my local machine, it displays `quickstart-loadbalancer-myproject.192.168.64.16.nip.io`).

The application can then be accessed by running:

[source,shell]
----
$ curl "http://$(oc get route quickstart-loadbalancer --template='{{ .spec.host }}')"
{"ip":"172.17.0.9"}
----

# Developer Instructions

## System Requirements

* https://github.com/golang/go[go] with `$GOPATH` set to `$HOME/go`
* Docker
* Either https://github.com/minishift/minishift[Minishift] or https://kubernetes.io/docs/setup/minikube/[Minikube]    

### Building the WildFly Operator

1. Add the source under `$GOPATH`:
+  
```
$ git clone https://github.com/wildfly/wildfly-operator.git $GOPATH/src/github.com/wildfly/wildfly-operator
```
2. Change to the source directory.
+
```
$ cd $GOPATH/src/github.com/wildfly/wildfly-operator
```
3. Review the available build targets.
+
```
$ make
```
4. Run any build target. For example, compile and build the WildFly Operator with:
+
```
$ make build
```

### Running the Operator

The Operator can run in two modes:

 * *Local mode*: The Operator is deployed as a local application running on your local computer. When using this mode, you don’t
need to build an Operator image. The operator becomes a local application that will be monitoring resources of your
kubernetes/OpenShift cluster.
 * *Deploy mode*: The Operator is deployed and runs in your cluster and not in your local computer. To use this mode
you need to build the Operator, push its image in a public container registry, for example, https://quay.io/[quay.io], and deploy it
as a regular resource on your cluster.

#### Local Mode

The following commands run the Operator as a local application and deploy the quickstart on minikube:

[source,shell]
----
$ make install
$ make run WATCH_NAMESPACE="$(kubectl get sa default -o jsonpath='{.metadata.namespace}')"
$ kubectl create -f config/samples/quickstart-cr.yaml
----

`make install` builds the CRD by using kustomize. +
`make run` will build the Operator and run it as a local application. +
You have to define the namespace you want to watch for changes by specifying the `WATCH_NAMESPACE` environment variable.

The following command removes the quickstart custom resource:

[source,shell]
----
$ kubectl delete -f config/samples/quickstart-cr.yaml
----

You can stop the Operator by using CTL-C.

If you want to Debug the Operator code in Local Mode, use `make debug` instead of `make run`:

[source,shell]
----
$ make debug WATCH_NAMESPACE="$(kubectl get sa default -o jsonpath='{.metadata.namespace}')"
----

This target will download https://github.com/go-delve/delve[Delve] and will start the Operator listening at 2345 port. You can later attach a debugger from your IDE.

#### Deploy Mode

In this mode you need to build the Operator, push its image in a public container registry, and deploy it as a regular resource.

To build the Operator image and push it to https://quay.io/[quay.io], execute the following command:

[source,shell]
----
$ QUAYIO_USERNAME="my-quay-user"
$ make manifests docker-build docker-push IMG="quay.io/${QUAYIO_USERNAME}/wildfly-operator-dev:latest"
----

To deploy this image in your cluster and deploy the quickstart example, execute the following:

[source,shell]
----
$ make deploy IMG="quay.io/${QUAYIO_USERNAME}/wildfly-operator-dev:latest"
$ kubectl create -f config/samples/quickstart-cr.yaml
----

To remove the quickstart custom resource, execute the following:

[source,shell]
----
$ kubectl delete -f config/samples/quickstart-cr.yaml
----

To remove the Operator from your cluster, execute the following:

[source,shell]
----
$ make undeploy
----

## Test suite

### Run unit tests

[source,shell]
----
$ make unit-test
----

### Run end-to-end (e2e) tests

To run the e2e tests you need to have a cluster accessible from your local machine and have logged in as kubeadmin.

#### Run end-to-end (e2e) tests in local mode
This is useful for development since you don’t need to build and push the Operator image to a docker registry.
The Operator will be deployed as a local application. Execute the following to run the test suite deploying the Operator
as a local application:

[source,shell]
----
$ make local-test
----

The test suite creates the resources for each test under the `wildfly-op-test-ns` namespace.
You can monitor the resources created by the test suite in a different terminal window by issuing:

[source,shell]
----
$ kubectl get pods -w -n wildfly-op-test-ns
----

If you stop the execution or if a test fails, you have to clean any resource left in your cluster before running the tests again:

[source,shell]
----
$ make clean
----

Note: Transaction recovery tests will be skipped under this mode since they cannot run outside the cluster.

#### Run end-to-end (e2e) tests in Deploy Mode

In this mode, the Operator will run in Deploy Mode, so you need to have the latest Operator image available somewhere.
Before running the e2e tests. Execute the following to build and push the Operator you want to test to a public container
registry, for example to https://quay.io/[quay.io]

[source,shell]
----
$ QUAYIO_USERNAME="my-quay-user"
$ make manifests docker-build docker-push IMG="quay.io/${QUAYIO_USERNAME}/wildfly-operator-dev:latest"
----

Once you have your Operator image publicly accessible, run the tests specifying the location of the Operator image under test:

[source,shell]
----
$ make test IMG="quay.io/${QUAYIO_USERNAME}/wildfly-operator-dev:latest"
----

The test suite creates the resources for each test under the `wildfly-op-test-ns` namespace.
You can monitor the resources created by the test suite in a different terminal window by issuing:

[source,shell]
----
$ oc get pods -w -n wildfly-op-test-ns
----

If a test fails or if you stop the testsuite before it finish normally, clean up the resources before starting it again

[source,shell]
----
$ make clean
----

### OLM installation

You can also install the Operator by using OLM. This could be useful to verify how changes in the CSV will be
applied. The following instructions describes how to prepare the Operator image, bundle and catalog to deploy the
Operator in a cluster that uses OML. The example commands use https://quay.io/[quay.io] as container registry and OpenShift as kubernetes cluster:

Execute the following command to build the Operator image, bundle and catalog:

[source,shell]
----
$ QUAYIO_USERNAME="my-quay-user"
$ make manifests docker-build docker-push bundle bundle-build bundle-push catalog-build catalog-push \
IMG="quay.io/${QUAYIO_USERNAME}/wildfly-operator-dev:latest" \
BUNDLE_IMG="quay.io/${QUAYIO_USERNAME}/wildfly-operator-bundle:1.0.0" \
CATALOG_IMG="quay.io/${QUAYIO_USERNAME}/wildfly-operator-catalog:1.0.0"
----

* `manifests docker-build docker-push`: Creates the Operator image and push it to your container registry.
* `bundle bundle-build bundle-push`: Builds a bundle with the resources needed by the operator. You can
modify the autogenerated CSV by looking at `bundle/manifests/` directory.
* `catalog-build catalog-push`: Creates a catalog containing the bundled Operator.

Then deploy a `CatalogSource` which contains information for accessing a repository of Operators.
When using OpenShift, the catalog source needs to be created on the `openshift-marketplace` namespace:

[source,yaml]
----
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: cs-wildfly-operator
  namespace: openshift-marketplace
spec:
  displayName: WildFly Operator Dev
  publisher: Company-Name
  sourceType: grpc
  image: quay.io/your-username/wildfly-operator-catalog:1.0.0
  updateStrategy:
    registryPoll:
      interval: 10m
----

Wait until your Operator is recognized by OLM:

[source,shell]
----
$ oc get packagemanifests | grep wildfly-operator
wildfly-operator                                   WildFly Operator Dev   48s
----

Once the operator is recognised by OML, you can install the Operator from the OpenShift web console or
by the command line creating a Subscription and an OperatorGroup resource:

[source,yaml]
----
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: subscription-wildfly-operator
spec:
  channel: alpha
  installPlanApproval: Automatic
  name: wildfly-operator
  source: cs-wildfly-operator
  sourceNamespace: openshift-marketplace
  startingCSV: wildfly-operator.v1.0.0
----

[source,yaml]
----
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: op-group-wildfly-operator
----

The Operator will be installed on the current namespace.