flinkk8soperator

module
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 27, 2020 License: Apache-2.0

README

Current Release Build Status GoDoc License CodeCoverage Go Report Card Commit activity Commit since last release Slack

Flinkk8soperator

FlinkK8sOperator is a Kubernetes operator that manages Flink applications on Kubernetes. The operator acts as control plane to manage the complete deployment lifecycle of the application.

Project Status

Beta

The operator is in use for some less-critical jobs at Lyft. At this point the focus is on testing and stability. While in Beta, we will attempt to limit the number of backwards-incompatible changes, but they may still occur as necessary.

Prerequisites

  • Version >= 1.10 of Kubernetes (versions < 1.13 require --feature-gates=CustomResourceSubresources=true)
  • Version >= 1.7 of Apache Flink.

Overview

Flink operator overview

The goal of running Flink on Kubernetes is to enable more flexible, lighter-weight deployment of streaming applications, without needing to manage infrastructure. The Flink operator aims to abstract out the complexity of hosting, configuring, managing and operating Flink clusters from application developers. It achieves this by extending any kubernetes cluster using a custom resources.

The Operator creates flink clusters dynamically using the specified custom resource. Flink clusters in kubernetes consist of the following:

Deploying and managing Flink applications in Kubernetes involves two steps:

  • Building Flink application packaged as a docker image: A docker image is built containing the application source code with the necessary dependencies built in. This is required to bootstrap the Jobmanager and Taskmanager pods. At Lyft we use Source-To-Image S2I as the image build tool that provides a common builder image with Apache Flink pre-installed. The docker image could be built using any pre-existing workflows at an organization.

  • Creating the Flink application custom resource: The custom resource for Flink application provides the spec for configuring and managing flink clusters in Kubernetes. The FlinkK8sOperator, deployed on Kubernetes, continuously monitors the resource and the corresponding flink cluster, and performs action based on the diff.

Documentation

Community

We welcome you to contribute and make the operator better! For questions, and changes please create an issue or submit a pull request.

Directories

Path Synopsis
cmd
integ
pkg
apis/app/v1alpha1
+k8s:deepcopy-gen=package +groupName=flink.k8s.io
+k8s:deepcopy-gen=package +groupName=flink.k8s.io
apis/app/v1beta1
+k8s:deepcopy-gen=package +groupName=flink.k8s.io
+k8s:deepcopy-gen=package +groupName=flink.k8s.io
client/clientset/versioned
This package has the automatically generated clientset.
This package has the automatically generated clientset.
client/clientset/versioned/fake
This package has the automatically generated fake clientset.
This package has the automatically generated fake clientset.
client/clientset/versioned/scheme
This package contains the scheme of the automatically generated clientset.
This package contains the scheme of the automatically generated clientset.
client/clientset/versioned/typed/app/v1beta1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/app/v1beta1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
client/clientset/versioned/typed/app/v1beta2
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/app/v1beta2/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL