kube-applier
Table of Contents
Created by gh-md-toc
Forked from: https://github.com/box/kube-applier
kube-applier is Kubernetes deployment tool strongly following
gitOps
principals. It enables continuous deployment of Kubernetes objects by applying
declarative configuration files from a Git repository to a Kubernetes cluster.
kube-applier runs as a Deployment in your cluster and watches the Git
repo to ensure that the cluster objects are
up-to-date with their associated spec files (JSON or YAML) in the repo.
Whenever a new commit to the repo occurs, or at a specified
interval, kube-applier performs a "run", issuing kubectl
apply commands at
namespace level. The convention is that level 1 subdirs of REPO_PATH represent
k8s namespaces: the name of the dir is the same as the namespace and the dir
contains manifests for the given namespace.
kube-applier serves a status page and provides
metrics for monitoring.
Usage
Environment variables
Required:
REPO_PATH
- (string) Absolute path to the directory containing
configuration files to be applied. It must be a Git repository or a path within
one. Level 1 subdirs of this directory represent kubernetes namespaces.
Optional:
-
DIFF_URL_FORMAT
should be a URL for a hosted remote repo that supports
linking to a commit hash. Replace the commit hash portion with "%s" so it can
be filled in by kube-applier (e.g.
https://github.com/kubernetes/kubernetes/commit/%s
).
-
LISTEN_PORT
- (int) Port for the container. This should be the same port
specified in the container spec. Default is 8080.
-
REPO_PATH_FILTERS
- (string) A comma separated list of sub directories to
be applied. Supports shell file name
patterns.
-
SERVER
- (string) Address of the Kubernetes API server. By default,
kube-applier uses in-cluster configuration targetting local internal endpoint.
Address must be specified with this environment variable (which is then written
into a kubeconfig
file). Authentication
to the API server is handled by service account tokens. See Accessing the
Cluster
for more info.
-
POLL_INTERVAL_SECONDS
- (int) Number of seconds to wait between each check
for new commits to the repo (default is 5).
-
FULL_RUN_INTERVAL_SECONDS
- (int) Number of
seconds between automatic full runs (default is 3600). Set to 0 to disable.
-
DRY_RUN
- (bool) If true, kubectl command will be run with --server-dry-run
flag. This means live configuration of the cluster is not changed.
-
LOG_LEVEL
- (string) trace|debug|info|warn|error case insensitive
Annotations
kube-applier behaviour is controlled through annotations on the Namespace
resource.
kind: Namespace
apiVersion: v1
metadata:
name: team
annotations:
kube-applier.io/enabled: 'true'
kube-applier.io/dry-run: 'false'
kube-applier.io/prune: 'true'
Mounting the Git Repository
Git-sync keeps a local directory up to date with a remote repo. The local
directory resides in a shared emptyDir volume that is mounted in both the
git-sync and kube-applier containers.
Reference the git-sync repo for setup
and usage.
What happens if the contents of the local Git repo change in the middle of a kube-applier run?
If there are changes to files in the $REPO_PATH
directory during a
kube-applier run, those changes may or may not be reflected in that run,
depending on the timing of the changes.
Given that the $REPO_PATH
directory is a Git repo or located within one, it
is likely that the majority of changes will be associated with a Git commit.
Thus, a change in the middle of a run will likely update the HEAD commit hash,
which will immediately trigger another run upon completion of the current run
(regardless of whether or not any of the changes were effective in the current
run). However, changes that are not associated with a new Git commit will not
trigger a run.
If I remove a configuration file, will kube-applier remove the associated Kubernetes object?
This is dependent on the kube-applier.io/prune
value (default true). If true,
then the
pruneWhitelist
is passed to the apply
command.
Deploying
Included is a Kustomize (https://kustomize.io/) base you can reference in your
namespace:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- github.com/utilitywarehouse/kube-applier//manifests/base?ref=v2.3.1
and patch as per example:
manifests/example/
Monitoring
Status UI
kube-applier hosts a status page on a webserver, served at the service endpoint
URL. The status page displays information about the most recent apply run,
including:
- Start and end times
- Latency
- Most recent commit
- Blacklisted files
- Errors
- Files applied successfully
The HTML template for the status page lives in templates/status.html
, and static/
holds additional assets.
Metrics
kube-applier uses Prometheus for
metrics. Metrics are hosted on the webserver at /metrics (status UI is the
index page). In addition to the Prometheus default metrics, the following
custom metrics are included:
-
run_latency_seconds - A
Summary
that keeps track of the durations of each apply run, tagged with a boolean for
whether or not the run was a success (i.e. no failed apply attempts).
-
namespace_apply_count - A
Counter
for each namespace that has had an apply attempt over the lifetime of the
container, incremented with each apply attempt and tagged by the namespace and
the result of the attempt.
-
result_summary - A
Gauge
for each deployment, labelled with the namespace, action, status and type of
object applied
-
kubectl_exit_code_count - A
Counter
for each exit code returned by executions of kubectl
, labelled with the
namespace and exit code.
The Prometheus HTTP API (also see
the Go
library)
can be used for querying the metrics server.
Running locally
# manifests git repository
export LOCAL_REPO_PATH="${HOME}/dev/work/kubernetes-manifests"
# directory within the manifests repository that contains namespace directories
export CLUSTER_DIR="exp-1-aws"
export DIFF_URL_FORMAT="https://github.com/utilitywarehouse/kubernetes-manifests/commit/%s"
export REPO_PATH_FILTERS="sys-*,kube-system,labs"
export LOG_LEVEL="info"
export DRY_RUN="true"
export SERVER="https://api.server"
The ca.crt and token to use against the remote server. This can be the
ServiceAccount that KA runs with, or any SA that contains enough privileges
sa=$(kubectl --context=exp-1-aws -n sys-kube-applier get secret | grep kube-applier | awk '{print $1}')
kubectl --context=exp-1-aws -n sys-kube-applier get secret ${sa} -o json | jq -r '."data"."ca.crt"' | base64 -d > /tmp/ka-ca.crt
kubectl --context=exp-1-aws -n sys-kube-applier get secret ${sa} -o json | jq -r '."data"."token"' | base64 -d > /tmp/ka-token
make build
make run
Copyright and License
Copyright 2016 Box, Inc. All rights reserved.
Copyright (c) 2017-2019 Utility Warehouse Ltd.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.