extension

package module
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 28, 2021 License: Apache-2.0 Imports: 39 Imported by: 4

README

Eirinix

godoc Build Status go report card codecov

Extensions Library for Cloud Foundry Eirini

How to use

Install
go get -u code.cloudfoundry.org/eirinix
Write your extension

An Eirini extension is a structure which satisfies the eirinix.Extension interface.

The interface is defined as follows:

type Extension interface {
	Handle(context.Context, Manager, *corev1.Pod, admission.Request) admission.Response
}

For example, a dummy extension (which does nothing) would be:


type MyExtension struct {}

func (e *MyExtension) Handle(context.Context, eirinix.Manager, *corev1.Pod, admission.Request) admission.Response {
	return admission.Response{}
}
Start the extension with eirinix

import "code.cloudfoundry.org/eirinix"

func main() {
    x := eirinix.NewManager(
            eirinix.ManagerOptions{
                Namespace:  "kubernetes-namespace",
                Host:       "listening.eirini-x.org",
                Port:       8889,
                // KubeConfig can be ommitted for in-cluster connections
                KubeConfig: kubeConfig,
        })

    x.AddExtension(&MyExtension{})
    log.Fatal(x.Start())
}
Issues

Kubernetes fails to contact the eirini-extensions mutating webhook if they are set in mandatory mode. This will make any pod fail that is meant to be patched by eirini. An indication that this is happening is that any app being publishesd using cf push is creating timeouts. When running kubectl get events -n eirini lines of log containing

Job Warning FailedCreate job-controller Error creating: Internal error occured

are shown.

Services

When you expose the webhook server through a service, you can advertize the webhook to kubernetes with a service reference.

You can do that by specifying a ServiceName instead:


import "code.cloudfoundry.org/eirinix"

func main() {
    x := eirinix.NewManager(
            eirinix.ManagerOptions{
                Namespace:  "eirini",
                Host:       "0.0.0.0",
                // KubeConfig can be ommitted for in-cluster connections
                KubeConfig: kubeConfig,
                ServiceName: "listening-extension",
                Port: 8889,
                // WebhookNamespace, when ServiceName is supplied, a WebhookNamespace is required to indicate in which namespace the webhook service runs on
                WebhookNamespace: "cf",
        })

    x.AddExtension(&MyExtension{})
    log.Fatal(x.Start())
}

The host will be the listening ip, and the service name refer to the kubernetes service. You need also to specify WebhookNamespace which is the namespace where the extension pod is running.

If you specify Port that will be both the port on which the webhook service will listen and the internal port (the container port). If you don't specify it, the default is 443 (https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#service-reference).

Split Extension registration into two binaries

You can split your extension into two binaries, one which registers the MutatingWebhook to kubernetes, and one which actually runs the MutatingWebhook http server.

To register only the extension, you can run the manager with the same option, but instead of Start(), you can call RegisterExtensions(). Note, the process will exit and there is no loop, an error is returned in case of failure.


import "code.cloudfoundry.org/eirinix"

func main() {
    x := eirinix.NewManager(
            eirinix.ManagerOptions{
                Namespace:  "eirini",
                Host:       "0.0.0.0",
                ServiceName: "listening-extension",
                WebhookNamespace: "cf",
        })

    x.AddExtension(&MyExtension{})
    err := x.RegisterExtensions()

    ...
}

Now we can run the Extension in a separate binary, by disabling the registration of the webhooks by setting RegisterWebHook to *false inside the eirinix.ManagerOptions:


import "code.cloudfoundry.org/eirinix"

func main() {
    RegisterWebhooks := false
    x := eirinix.NewManager(
            eirinix.ManagerOptions{
                Namespace:  "eirini",
                Host:       "0.0.0.0",
                ServiceName: "listening-extension",
                WebhookNamespace: "cf",
                RegisterWebHook: &RegisterWebhooks,
        })

    x.AddExtension(&MyExtension{})
    log.Fatal(x.Start())
}
Fix for a running cluster

In order to trigger re-generation of the mutating webhook certificate, we have to delete the secrets and the associated mutating webhook:

  • run kubectl delete secret eirini-extensions-webhook-server-cert -n eirini
  • run kubectl delete mutatingwebhookconfiguration eirini-x-mutating-hook-default
  • connect to the eirini-0 pod (kubectl exec -it eirini-0 -n eirini /bin/bash)
  • run monit restart eirini-extensions
Fix on redeploy on an existing k8s (which had a scf deployed before):

  • In case of multiple re-deployments on the same cluster it can happen that old secrets are still present on the cluster. The eirinix library then tries to reuse those, resulting in a failed connection since the service will have a different IP-address.

  • Before redeploying run kubectl get secrets -n eirini, if there is an eirini-x-setupcertificate (the name may vary depending on the operator fingerprint set on the extension, see https://godoc.org/code.cloudfoundry.org/eirinix#ManagerOptions for details) present, delete it using kubectl delete secret eirini-x-setupcertificate -n eirini

    We need also to remove the older mutatingwebhook:

    $> kubectl get mutatingwebhookconfiguration
NAME                                     CREATED AT
eirini-x-mutating-hook-default   2019-06-10T08:55:30Z

$> kubectl delete mutatingwebhookconfiguration eirini-x-mutating-hook-default

Documentation

Index

Constants

View Source 
const (
	LabelGUID        = "cloudfoundry.org/guid"
	LabelVersion     = "cloudfoundry.org/version"
	LabelAppGUID     = "cloudfoundry.org/app_guid"
	LabelProcessType = "cloudfoundry.org/process_type"
	LabelSourceType  = "cloudfoundry.org/source_type"
)

Variables

This section is empty.

Functions

func AddToScheme 

func AddToScheme(s *runtime.Scheme) error

AddToScheme adds all Resources to the Scheme

Types

type Config 

type Config struct {
	CtxTimeOut time.Duration

	// Namespace that is being watched by controllers
	Namespace         string
	WebhookServerHost string
	WebhookServerPort int32
	Fs                afero.Fs
}

Config controls the behaviour of different controllers

type DefaultExtensionManager 

type DefaultExtensionManager struct {
	// Extensions is the list of the Extensions that will be registered by the Manager
	Extensions []Extension

	// Watchers is the list of Eirini watchers handlers
	Watchers []Watcher

	// Reconcilers is the list of Eirini Reconcilers
	Reconcilers []Reconciler

	// KubeManager is the kubernetes manager object which is setted up by the Manager
	KubeManager manager.Manager

	// Logger is the logger used internally and accessible to the Extensions
	Logger *zap.SugaredLogger

	// Context is the context structure used by internal components
	Context context.Context

	// WebhookConfig is the webhook configuration used to generate certificates
	WebhookConfig *WebhookConfig

	// WebhookServer is the webhook server where the Manager registers the Extensions to.
	WebhookServer *webhook.Server

	// Credsgen is the credential generator implementation used for generating certificates
	Credsgen credsgen.Generator

	// Options are the manager options
	Options ManagerOptions
	// contains filtered or unexported fields
}

DefaultExtensionManager represent an implementation of Manager

func (*DefaultExtensionManager) AddExtension 

func (m *DefaultExtensionManager) AddExtension(v interface{}) error

AddExtension adds an Eirini extension to the manager. It accepts Eirinix.Watcher, Eirinix.Reconciler and Eirinix.Extension types.

func (*DefaultExtensionManager) AddReconciler 

func (m *DefaultExtensionManager) AddReconciler(r Reconciler)

AddReconciler adds an Erini reconciler Extension to the manager

func (*DefaultExtensionManager) AddWatcher 

func (m *DefaultExtensionManager) AddWatcher(w Watcher)

AddWatcher adds an Erini watcher Extension to the manager

func (*DefaultExtensionManager) GenWatcher 

func (m *DefaultExtensionManager) GenWatcher(client corev1client.CoreV1Interface) (watch.Interface, error)

GenWatcher generates a watcher from a corev1client interface

func (*DefaultExtensionManager) GenWebHookServer 

func (m *DefaultExtensionManager) GenWebHookServer()

GenWebHookServer prepares the webhook server structures

func (*DefaultExtensionManager) GetContext 

func (m *DefaultExtensionManager) GetContext() context.Context

GetContext returns the context which can be used by Extensions and Reconcilers to perform background requests

func (*DefaultExtensionManager) GetKubeClient 

func (m *DefaultExtensionManager) GetKubeClient() (corev1client.CoreV1Interface, error)

GetKubeClient returns a kubernetes Corev1 client interface from the rest config used.

func (*DefaultExtensionManager) GetKubeConnection 

func (m *DefaultExtensionManager) GetKubeConnection() (*rest.Config, error)

GetKubeConnection sets up a connection to a Kubernetes cluster if not existing.

func (*DefaultExtensionManager) GetKubeManager 

func (m *DefaultExtensionManager) GetKubeManager() manager.Manager

GetKubeManager returns the kubernetes manager which can be used by Reconcilers to perform direct requests

func (*DefaultExtensionManager) GetLogger 

func (m *DefaultExtensionManager) GetLogger() *zap.SugaredLogger

GetLogger returns the Manager injected logger

func (*DefaultExtensionManager) GetManagerOptions 

func (m *DefaultExtensionManager) GetManagerOptions() ManagerOptions

GetManagerOptions returns the Manager options

func (*DefaultExtensionManager) HandleEvent 

func (m *DefaultExtensionManager) HandleEvent(e watch.Event)

HandleEvent handles a watcher event. It propagates the event to all the registered watchers.

func (*DefaultExtensionManager) ListExtensions 

func (m *DefaultExtensionManager) ListExtensions() []Extension

ListExtensions returns the list of the Extensions added to the Manager

func (*DefaultExtensionManager) ListReconcilers 

func (m *DefaultExtensionManager) ListReconcilers() []Reconciler

ListReconcilers returns the list of the Extensions added to the Manager

func (*DefaultExtensionManager) ListWatchers 

func (m *DefaultExtensionManager) ListWatchers() []Watcher

ListWatchers returns the list of the Extensions added to the Manager

func (*DefaultExtensionManager) LoadExtensions 

func (m *DefaultExtensionManager) LoadExtensions() error

LoadExtensions generates and register webhooks from the Extensions added to the Manager

func (*DefaultExtensionManager) OperatorSetup 

func (m *DefaultExtensionManager) OperatorSetup() error

OperatorSetup prepares the webhook server, generates certificates and configuration. It also setups the namespace label for the operator

func (*DefaultExtensionManager) PatchFromPod 

func (m *DefaultExtensionManager) PatchFromPod(req admission.Request, pod *corev1.Pod) admission.Response

func (*DefaultExtensionManager) ReadWatcherEvent 

func (m *DefaultExtensionManager) ReadWatcherEvent(w watch.Interface)

ReadWatcherEvent tries to read events from the watcher channel. It should be run in a loop.

func (*DefaultExtensionManager) RegisterExtensions 

func (m *DefaultExtensionManager) RegisterExtensions() error

RegisterExtensions generates the manager and the operator setup, and loads the extensions to the webhook server

func (*DefaultExtensionManager) SetKubeClient 

func (m *DefaultExtensionManager) SetKubeClient(c corev1client.CoreV1Interface)

SetKubeClient sets a kube client corev1 from a given one

func (*DefaultExtensionManager) SetKubeConnection 

func (m *DefaultExtensionManager) SetKubeConnection(c *rest.Config)

SetKubeConnection sets a rest config from a given one

func (*DefaultExtensionManager) SetManagerOptions 

func (m *DefaultExtensionManager) SetManagerOptions(o ManagerOptions)

SetManagerOptions sets the ManagerOptions with the provided one

func (*DefaultExtensionManager) Start 

func (m *DefaultExtensionManager) Start() error

Start starts the Manager infinite loop, and returns an error on failure

func (*DefaultExtensionManager) Stop 

func (m *DefaultExtensionManager) Stop()

func (*DefaultExtensionManager) Watch 

func (m *DefaultExtensionManager) Watch() error

Watch starts the Watchers Manager infinite loop, and returns an error on failure

type DefaultMutatingWebhook 

type DefaultMutatingWebhook struct {

	// EiriniExtension is the Eirini extension associated with the webhook
	EiriniExtension Extension

	// EiriniExtensionManager is the Manager which will be injected into the Handle.
	EiriniExtensionManager Manager

	// FilterEiriniApps indicates if the webhook will filter Eirini apps or not.
	FilterEiriniApps bool

	// Name is the name of the webhook
	Name string
	// Path is the path this webhook will serve.
	Path string
	// Rules maps to the Rules field in admissionregistrationv1beta1.Webhook
	Rules []admissionregistrationv1beta1.RuleWithOperations
	// FailurePolicy maps to the FailurePolicy field in admissionregistrationv1beta1.Webhook
	// This optional. If not set, will be defaulted to Ignore (fail-open) by the server.
	// More details: https://github.com/kubernetes/api/blob/f5c295feaba2cbc946f0bbb8b535fc5f6a0345ee/admissionregistration/v1beta1/types.go#L144-L147
	FailurePolicy admissionregistrationv1beta1.FailurePolicyType
	// NamespaceSelector maps to the NamespaceSelector field in admissionregistrationv1beta1.Webhook
	// This optional.
	NamespaceSelector *metav1.LabelSelector
	// Handlers contains a list of handlers. Each handler may only contains the business logic for its own feature.
	// For example, feature foo and bar can be in the same webhook if all the other configurations are the same.
	// The handler will be invoked sequentially as the order in the list.
	// Note: if you are using mutating webhook with multiple handlers, it's your responsibility to
	// ensure the handlers are not generating conflicting JSON patches.
	Handler admission.Handler
	// Webhook contains the Admission webhook information that we register with the controller runtime.
	Webhook *webhook.Admission
	// contains filtered or unexported fields
}

DefaultMutatingWebhook is the implementation of the Webhook generated out of the Eirini Extension

func (*DefaultMutatingWebhook) GetFailurePolicy 

func (w *DefaultMutatingWebhook) GetFailurePolicy() admissionregistrationv1beta1.FailurePolicyType

func (*DefaultMutatingWebhook) GetHandler 

func (w *DefaultMutatingWebhook) GetHandler() admission.Handler

func (*DefaultMutatingWebhook) GetLabelSelector 

func (w *DefaultMutatingWebhook) GetLabelSelector() *metav1.LabelSelector

func (*DefaultMutatingWebhook) GetName 

func (w *DefaultMutatingWebhook) GetName() string

func (*DefaultMutatingWebhook) GetNamespaceSelector 

func (w *DefaultMutatingWebhook) GetNamespaceSelector() *metav1.LabelSelector

func (*DefaultMutatingWebhook) GetPath 

func (w *DefaultMutatingWebhook) GetPath() string

func (*DefaultMutatingWebhook) GetPod 

func (w *DefaultMutatingWebhook) GetPod(req admission.Request) (*corev1.Pod, error)

GetPod retrieves a pod from a types.Request

func (*DefaultMutatingWebhook) GetRules 

func (w *DefaultMutatingWebhook) GetRules() []admissionregistrationv1beta1.RuleWithOperations

func (*DefaultMutatingWebhook) GetWebhook 

func (w *DefaultMutatingWebhook) GetWebhook() *webhook.Admission

func (*DefaultMutatingWebhook) Handle 

func (w *DefaultMutatingWebhook) Handle(ctx context.Context, req admission.Request) admission.Response

Handle delegates the Handle function to the Eirini Extension

func (*DefaultMutatingWebhook) InjectClient 

func (w *DefaultMutatingWebhook) InjectClient(c client.Client) error

InjectClient injects the client.

func (*DefaultMutatingWebhook) InjectDecoder 

func (w *DefaultMutatingWebhook) InjectDecoder(d *admission.Decoder) error

InjectDecoder injects the decoder.

func (*DefaultMutatingWebhook) RegisterAdmissionWebHook 

func (w *DefaultMutatingWebhook) RegisterAdmissionWebHook(server *webhook.Server, opts WebhookOptions) error

RegisterAdmissionWebHook registers the Mutating WebHook to the WebHook Server and returns the generated Admission Webhook

type Extension 

type Extension interface {
	// Handle handles a kubernetes request.
	// It is the main entry point of the Eirini extensions and the arguments are the
	// decoded payloads from the kubeapi server.
	//
	// The manager will attempt to decode a pod from the request if possible and passes it to the Manager.
	Handle(context.Context, Manager, *corev1.Pod, admission.Request) admission.Response
}

Extension is the Eirini Extension interface

An Eirini Extension must implement it by providing only an Handle method which will be used as a response to the kube api server.

The Extension typically returns a set of patches defining the difference between the pod received in the request and the wanted state from the Extension.

type Manager 

type Manager interface {

	// AddExtension adds an Extension to the manager
	//
	// The manager later on, will register the Extension when Start() is being called.
	AddExtension(v interface{}) error

	// AddReconciler adds a Reconciler Extension to the manager
	//
	// The manager later on, will register the Extension when Start() is being called.
	AddReconciler(r Reconciler)

	// Start starts the manager infinite loop.
	//
	// Registers all the Extensions and generates
	// the respective mutating webhooks.
	//
	// Returns error in case of failure.
	Start() error

	// ListExtensions returns a list of the current loaded Extension
	ListExtensions() []Extension

	// ListReconcilers returns a list of the current loaded Reconcilers
	ListReconcilers() []Reconciler

	// GetContext returns the context of the manager, which can be used in internall cals by extension
	GetContext() context.Context

	// GetKubeManager returns the kubernetes manager which can be used by Reconcilers to perform
	// direct requests
	GetKubeManager() manager.Manager

	// GetKubeConnection sets up a kube connection if not already present
	//
	// Returns the rest config used to establish a connection to the kubernetes cluster.
	GetKubeConnection() (*rest.Config, error)

	// GetKubeClient sets up a kube client if not already present
	//
	// Returns the kubernetes interface.
	GetKubeClient() (corev1client.CoreV1Interface, error)

	// GetLogger returns the logger of the application. It can be passed an already existing one
	// by using NewManager()
	GetLogger() *zap.SugaredLogger

	// Watch starts the main loop for the registered watchers
	Watch() error

	// AddWatcher register a watcher to EiriniX
	AddWatcher(w Watcher)

	// Helper to compute the patch from a pod update
	PatchFromPod(req admission.Request, pod *corev1.Pod) admission.Response

	// Register Extensions to the kubernetes cluster.
	RegisterExtensions() error

	// Stop stops the manager execution
	Stop()

	// SetManagerOptions it is a setter for the ManagerOptions
	SetManagerOptions(ManagerOptions)

	// GetManagerOptions returns current ManagerOptions
	GetManagerOptions() ManagerOptions
}

Manager is the interface of the manager for registering Eirini extensions

It will generate webhooks that will satisfy the MutatingWebhook interface from the defined Extensions.

func NewManager 

func NewManager(opts ManagerOptions) Manager

NewManager returns a manager for the kubernetes cluster. the kubeconfig file and the logger are optional

type ManagerOptions 

type ManagerOptions struct {

	// Namespace is the namespace where pods will trigger the extension. Use empty to trigger on all namespaces.
	Namespace string

	// Host is the listening host address for the Manager
	Host string

	// Port is the listening port
	Port int32

	// Context is the context to be used for Kube requests. Leave it empty for automatic generation
	Context *context.Context

	// KubeConfig is the kubeconfig path. Optional, omit for in-cluster connection
	KubeConfig string

	// Logger is the default logger. Optional, if omitted a new one will be created
	Logger *zap.SugaredLogger

	// FailurePolicy default failure policy for the webhook server.  Optional, defaults to fail
	FailurePolicy *admissionregistrationv1beta1.FailurePolicyType

	// FilterEiriniApps enables or disables Eirini apps filters.  Optional, defaults to true
	FilterEiriniApps *bool

	// OperatorFingerprint is a unique string identifiying the Manager.  Optional, defaults to eirini-x
	OperatorFingerprint string

	// SetupCertificateName is the name of the generated certificates.  Optional, defaults uses OperatorFingerprint to generate a new one
	SetupCertificateName string

	// RegisterWebHook enables or disables automatic registering of webhooks. Defaults to true
	RegisterWebHook *bool

	// SetupCertificate enables or disables automatic certificate generation. Defaults to true
	SetupCertificate *bool

	// ServiceName registers the Extension as a MutatingWebhook reachable by a service
	ServiceName string

	// WebhookNamespace, when ServiceName is supplied, a WebhookNamespace is required to indicate in which namespace the webhook service runs on
	WebhookNamespace string

	// WatcherStartRV is the starting ResourceVersion of the PodList which is being watched (see Kubernetes #74022).
	// If omitted, it will start watching from the current RV.
	WatcherStartRV string
}

ManagerOptions represent the Runtime manager options

type MutatingWebhook 

type MutatingWebhook interface {
	Handle(context.Context, admission.Request) admission.Response
	InjectClient(c client.Client) error
	InjectDecoder(d *admission.Decoder) error
	RegisterAdmissionWebHook(*webhook.Server, WebhookOptions) error

	GetName() string
	GetPath() string
	GetRules() []admissionregistrationv1beta1.RuleWithOperations
	GetFailurePolicy() admissionregistrationv1beta1.FailurePolicyType
	GetNamespaceSelector() *metav1.LabelSelector
	GetLabelSelector() *metav1.LabelSelector
	GetHandler() admission.Handler
	GetWebhook() *webhook.Admission
}

MutatingWebhook is the interface of the generated webhook from the Extension

It represent the minimal set of methods that the libraries used behind the scenes expect from a structure that implements a Mutating Webhook

func NewWebhook 

func NewWebhook(e Extension, m Manager) MutatingWebhook

NewWebhook returns a MutatingWebhook out of an Eirini Extension

type Reconciler 

type Reconciler interface {
	Reconcile(request reconcile.Request) (reconcile.Result, error)
	Register(Manager) error
}

Reconciler is the Eirini Reconciler Extension interface

An Eirini Reconciler must implement a Reconcile method which is called when a new request is being created.

type Watcher 

type Watcher interface {
	Handle(Manager, watch.Event)
}

Watcher is the Eirini Watcher Extension interface.

An Eirini Watcher must implement a Handle method, which is called with the event that occurred in the namespace.

type WatcherChannelClosedError 

type WatcherChannelClosedError struct {
	// contains filtered or unexported fields
}

WatcherChannelClosedError can be used to filter for "watcher channel closed" in a block like this: if err, ok := err.(*extension.WatcherChannelClosedError); ok { // Do things }

func (*WatcherChannelClosedError) Error 

func (e *WatcherChannelClosedError) Error() string

Error implements the error Interface for WatcherChannelClosedError

type WebhookConfig 

type WebhookConfig struct {
	ConfigName    string
	CertDir       string
	Certificate   []byte
	Key           []byte
	CaCertificate []byte
	CaKey         []byte
	// contains filtered or unexported fields
}

WebhookConfig generates certificates and the configuration for the webhook server

func NewWebhookConfig 

func NewWebhookConfig(c client.Client, config *Config, generator credsgen.Generator, configName string, setupCertificateName string, serviceName string, webhookNamespace string) *WebhookConfig

NewWebhookConfig returns a new WebhookConfig

func (*WebhookConfig) GenerateAdmissionWebhook 

func (f *WebhookConfig) GenerateAdmissionWebhook(webhooks []MutatingWebhook) []admissionregistrationv1beta1.MutatingWebhook

type WebhookOptions 

type WebhookOptions struct {
	ID             string // Webhook path will be generated out of that
	MatchLabels    map[string]string
	Manager        manager.Manager
	ManagerOptions ManagerOptions
}

WebhookOptions are the options required to register a WebHook to the WebHook server

Source Files

View all Source files

Directories

Path Synopsis
Package testing contains methods to create test data.
 Package testing contains methods to create test data.
fakes
Code generated by counterfeiter.
 Code generated by counterfeiter.
util
ctxlog

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.