package module
v0.0.0-...-9709769 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Mar 4, 2020 License: Apache-2.0 Imports: 33 Imported by: 17


CAPI e2e testing framework

This framework aims to define common end-to-end patterns that can be reused across Cluster API providers.


Use this framework as you see fit. If there are pieces that don't work for you, do not use them. If there are pieces that almost work for you, customize them. If this does not fit your e2e testing please file an issue and we can discuss your use case and find a nice solution for everyone.

Optionally override the images that get loaded onto the management cluster.

This feature allows you to obtain a CAPI management image locally and use that specific image in your kind cluster. If you do not have one locally then the latest :master image will be used on the management cluster.


This framework contains


  • You must use ginkgo for your testing framework.


To see this framework in use please take a look at the docker provider found in the cluster-api repository.




View Source
const (
	// DefaultManagementClusterName is the default name of the Kind cluster
	// used by the the e2e framework.
	DefaultManagementClusterName = "mgmt"

	// DefaultKubernetesVersion is the default version of Kubernetes to deploy
	// for testing.
	DefaultKubernetesVersion = "v1.16.2"


This section is empty.


func ApplyYAMLURL

func ApplyYAMLURL(ctx context.Context, input ApplyYAMLURLInput)

ApplyYAMLURL is essentially kubectl apply -f <url>. If the YAML in the URL contains Kinds not registered with the scheme this will fail.

func AssertAllClusterAPIResourcesAreGone

func AssertAllClusterAPIResourcesAreGone(ctx context.Context, input AssertAllClusterAPIResourcesAreGoneInput)

AssertAllClusterAPIResourcesAreGone ensures that all known Cluster API resources have been remvoed.

func AssertControlPlaneFailureDomains

func AssertControlPlaneFailureDomains(ctx context.Context, input AssertControlPlaneFailureDomainsInput)

AssertControlPlaneFailureDomains will look at all control plane machines and see what failure domains they were placed in. If machines were placed in unexpected or wrong failure domains the expectation will fail.

func CreateCluster

func CreateCluster(ctx context.Context, input CreateClusterInput, intervals ...interface{})

CreateCluster will create the Cluster and InfraCluster objects.

func CreateKubeadmControlPlane

func CreateKubeadmControlPlane(ctx context.Context, input CreateKubeadmControlPlaneInput, intervals ...interface{})

CreateKubeadmControlPlane creates the control plane object and necessary dependencies.

func CreateMachineDeployment

func CreateMachineDeployment(ctx context.Context, input CreateMachineDeploymentInput)

CreateMachineDeployment creates the machine deployment and dependencies.

func CreateRelatedResources

func CreateRelatedResources(ctx context.Context, input CreateRelatedResourcesInput, intervals ...interface{})

CreateRelatedResources is used to create runtime.Objects.

func DeleteCluster

func DeleteCluster(ctx context.Context, input DeleteClusterInput)

DeleteCluster deletes the cluster and waits for everything the cluster owned to actually be gone.

func DumpProviderResources

func DumpProviderResources(mgmt ManagementCluster, resources map[string]runtime.Object, resourcePath string, writer io.Writer) error

DumpProviderResources dump provider specific API related resources to YAML

func DumpResources

func DumpResources(mgmt ManagementCluster, resourcePath string, writer io.Writer) error

DumpResources dump cluster API related resources to YAML

func EnsureNamespace

func EnsureNamespace(ctx context.Context, mgmt client.Client, namespace string)

EnsureNamespace verifies if a namespaces exists. If it doesn't it will create the namespace.

func InstallComponents

func InstallComponents(ctx context.Context, mgmt Applier, components ...ComponentGenerator)

InstallComponents is a helper function that applies components, generally to a management cluster.

func TryAddDefaultSchemes

func TryAddDefaultSchemes(scheme *runtime.Scheme)

TryAddDefaultSchemes tries to add the following schemes:

  • Kubernetes corev1
  • Kubernetes appsv1
  • CAPI core
  • Kubeadm Bootstrapper
  • Kubeadm ControlPlane

Any error that occurs when trying to add the schemes is ignored.

func TypeToKind

func TypeToKind(i interface{}) string

TypeToKind returns the Kind without the package prefix. Pass in a pointer to a struct This will panic if used incorrectly.

func WaitForAPIServiceAvailable

func WaitForAPIServiceAvailable(ctx context.Context, mgmt Waiter, serviceName string)

WaitForAPIServiceAvailable will wait for an an APIService to be available. For example, kubectl wait --for=condition=Available --timeout=300s apiservice

func WaitForClusterDeleted

func WaitForClusterDeleted(ctx context.Context, input WaitForClusterDeletedInput, intervals ...interface{})

WaitForClusterDeleted waits until the cluster object has been deleted.

func WaitForClusterToProvision

func WaitForClusterToProvision(ctx context.Context, input WaitForClusterToProvisionInput, intervals ...interface{})

WaitForClusterToProvision will wait for a cluster to have a phase status of provisioned.

func WaitForControlPlaneToBeReady

func WaitForControlPlaneToBeReady(ctx context.Context, input WaitForControlPlaneToBeReadyInput, intervals ...interface{})

WaitForControlPlaneToBeReady will wait for a control plane to be ready. TODO(chuckha): Once we implement control plane Ready, then we should update this to wait actually wait for ready. TODO(chuckha): In the meantime this uses initialized as a placeholder for Ready.

func WaitForKubeadmControlPlaneMachinesToExist

func WaitForKubeadmControlPlaneMachinesToExist(ctx context.Context, input WaitForKubeadmControlPlaneMachinesToExistInput, intervals ...interface{})

WaitForKubeadmControlPlaneMachinesToExist will wait until all control plane machines have node refs.

func WaitForMachineDeploymentNodesToExist

func WaitForMachineDeploymentNodesToExist(ctx context.Context, input WaitForMachineDeploymentNodesToExistInput, intervals ...interface{})

WaitForMachineDeploymentNodesToExist waits until all nodes associated with a machine deployment exist.

func WaitForPodsReadyInNamespace

func WaitForPodsReadyInNamespace(ctx context.Context, cluster Waiter, namespace string)

WaitForPodsReadyInNamespace will wait for all pods to be Ready in the specified namespace. For example, kubectl wait --for=condition=Ready --timeout=300s --namespace capi-system pods --all

func YAMLForComponentSource

func YAMLForComponentSource(ctx context.Context, source ComponentSource) ([]byte, error)

YAMLForComponentSource returns the YAML for the provided component source.


type Applier

type Applier interface {
	// Apply allows us to apply YAML to the cluster, `kubectl apply`
	Apply(context.Context, []byte) error

Applier is an interface around applying YAML to a cluster

type ApplyYAMLURLInput

type ApplyYAMLURLInput struct {
	Client        client.Client
	HTTPGetter    HTTPGetter
	NetworkingURL string
	Scheme        *runtime.Scheme

ApplyYAMLURLInput is the input for ApplyYAMLURL.

type AssertAllClusterAPIResourcesAreGoneInput

type AssertAllClusterAPIResourcesAreGoneInput struct {
	Lister  Lister
	Cluster *clusterv1.Cluster

AssertAllClusterAPIResourcesAreGoneInput is the input for AssertAllClusterAPIResourcesAreGone.

type AssertControlPlaneFailureDomainsInput

type AssertControlPlaneFailureDomainsInput struct {
	GetLister  GetLister
	ClusterKey client.ObjectKey
	// ExpectedFailureDomains is required because this function cannot (easily) infer what success looks like.
	// In theory this field is not strictly necessary and could be replaced with enough clever logic/math.
	ExpectedFailureDomains map[string]int

AssertControlPlaneFailureDomainsInput is the input for AssertControlPlaneFailureDomains.

type ComponentConfig

type ComponentConfig struct {
	// Name is the name of the component.
	// This field is primarily used for logging.
	Name string `json:"name"`

	// Sources is an optional list of component YAML to apply to the management
	// cluster.
	// This field may be omitted when wanting only to block progress via one or
	// more Waiters.
	Sources []ComponentSource `json:"sources,omitempty"`

	// Waiters is an optional list of checks to perform in order to determine
	// whether or not the installed components are ready.
	Waiters []ComponentWaiter `json:"waiters,omitempty"`

ComponentConfig describes a component required by the e2e test environment.

type ComponentGenerator

type ComponentGenerator interface {
	// GetName returns the name of the component.
	GetName() string
	// Manifests return the YAML bundle.
	Manifests(context.Context) ([]byte, error)

ComponentGenerator is used to install components, generally any YAML bundle.

func ComponentGeneratorForComponentSource

func ComponentGeneratorForComponentSource(source ComponentSource) ComponentGenerator

ComponentGeneratorForComponentSource returns a ComponentGenerator for the provided ComponentSource.

type ComponentReplacement

type ComponentReplacement struct {
	// Old is the pattern to replace.
	// A regular expression may be used.
	Old string `json:"old"`
	// New is the string used to replace the old pattern.
	// An empty string is valid.
	New string `json:"new,omitempty"`

ComponentReplacement is used to replace some of the generated YAML prior to application.

type ComponentSource

type ComponentSource struct {
	// Name is used for logging when a component has multiple sources.
	Name string `json:"name,omitempty"`

	// Value is the source of the component's YAML.
	// May be a URL or a kustomization root (specified by Type).
	// If a Type=url then Value may begin with file://, http://, or https://.
	// If a Type=kustomize then Value may be any valid go-getter URL. For
	// more information please see
	Value string `json:"value"`

	// Type describes how to process the source of the component's YAML.
	// Defaults to "kustomize".
	Type ComponentSourceType `json:"type,omitempty"`

	// Replacements is a list of patterns to replace in the component YAML
	// prior to application.
	Replacements []ComponentReplacement `json:"replacements,omitempty"`

ComponentSource describes how to obtain a component's YAML.

type ComponentSourceType

type ComponentSourceType string

ComponentSourceType indicates how a component's source should be obtained.

const (
	// URLSource is component YAML available directly via a URL.
	// The URL may begin with file://, http://, or https://.
	URLSource ComponentSourceType = "url"

	// KustomizeSource is a valid kustomization root that can be used to produce
	// the component YAML.
	KustomizeSource ComponentSourceType = "kustomize"

type ComponentWaiter

type ComponentWaiter struct {
	// Value varies depending on the specified Type.
	// Please see the documentation for the different WaiterType constants to
	// understand the valid values for this field.
	Value string `json:"value"`

	// Type describes the type of check to perform.
	// Defaults to "pods".
	Type ComponentWaiterType `json:"type,omitempty"`

ComponentWaiter contains information to help determine whether installed components are ready.

type ComponentWaiterType

type ComponentWaiterType string

ComponentWaiterType indicates the type of check to use to determine if the installed components are ready.

const (
	// ServiceWaiter indicates to wait until a service's condition is Available.
	// When ComponentWaiter.Value is set to "service", the ComponentWaiter.Value
	// should be set to the name of a Service resource.
	ServiceWaiter ComponentWaiterType = "service"

	// PodsWaiter indicates to wait until all the pods in a namespace have a
	// condition of Ready.
	// When ComponentWaiter.Value is set to "pods", the ComponentWaiter.Value
	// should be set to the name of a Namespace resource.
	PodsWaiter ComponentWaiterType = "pods"

type Config

type Config struct {
	// Name is the name of the Kind management cluster.
	// Defaults to DefaultManagementClusterName.
	ManagementClusterName string `json:"managementClusterName,omitempty"`

	// KubernetesVersion is the version of Kubernetes to deploy when testing.
	// Defaults to DefaultKubernetesVersion.
	KubernetesVersion string `json:"kubernetesVersion,omitempty"`

	// Images is a list of container images to load into the Kind cluster.
	Images []ContainerImage `json:"images,omitempty"`

	// Components is a list of component configurations applied to the
	// Kind cluster.
	// The components are applied serially, in the listed order.
	Components []ComponentConfig `json:"components,omitempty"`

Config is the input used to configure the e2e test environment.

func DefaultConfig

func DefaultConfig() (Config, error)

DefaultConfig returns a default Config object that loads cert-manager, CAPI core, the Kubeadm Bootstrapper, and the Kubeadm ControlPlane.

Callers may append their own images to the returne Config.Images and their own components to Config.Components in order to stand up a management cluster for testing infrastructure providers.

func LoadConfig

func LoadConfig(data []byte) (*Config, error)

LoadConfig loads a Config from the provided YAML data.

func MustDefaultConfig

func MustDefaultConfig() Config

MustDefaultConfig panics if DefaultConfig returns an error.

func (*Config) Defaults

func (c *Config) Defaults()

Defaults assigns default values to the object.

func (*Config) Validate

func (c *Config) Validate() error

Validate validates the configuration.

type ContainerImage

type ContainerImage struct {
	// Name is the fully qualified name of the image.
	Name string

	// LoadBehavior may be used to dictate whether a failed load operation
	// should fail the test run. This is useful when wanting to load images
	// *if* they exist locally, but not wanting to fail if they don't.
	// Defaults to MustLoadImage.
	LoadBehavior LoadImageBehavior

ContainerImage describes an image to load into a cluster and the behavior when loading the image.

type ControlplaneClusterInput

type ControlplaneClusterInput struct {
	Management        ManagementCluster
	Cluster           *clusterv1.Cluster
	InfraCluster      runtime.Object
	Nodes             []Node
	MachineDeployment MachineDeployment
	RelatedResources  []runtime.Object
	CreateTimeout     time.Duration
	DeleteTimeout     time.Duration

	ControlPlane    *controlplanev1.KubeadmControlPlane
	MachineTemplate runtime.Object

ControlplaneClusterInput defines the necessary dependencies to run a multi-node control plane cluster. Deprecated.

func (*ControlplaneClusterInput) CleanUpCoreArtifacts

func (input *ControlplaneClusterInput) CleanUpCoreArtifacts()

CleanUpCoreArtifacts deletes the cluster and waits for everything to be gone. Assertions made on objects owned by the Cluster:

  • All Machines are removed
  • All MachineSets are removed
  • All MachineDeployments are removed
  • All KubeadmConfigs are removed
  • All Secrets are removed


func (*ControlplaneClusterInput) ControlPlaneCluster

func (input *ControlplaneClusterInput) ControlPlaneCluster()

ControlPlaneCluster creates an n node control plane cluster. Assertions:

  • The number of nodes in the created cluster will equal the number of control plane nodes plus the number of replicas in the machine deployment.

Deprecated. Please use the supplied functions below to get the exact behavior desired.

func (*ControlplaneClusterInput) SetDefaults

func (input *ControlplaneClusterInput) SetDefaults()

SetDefaults defaults the struct fields if necessary. Deprecated.

type CreateClusterInput

type CreateClusterInput struct {
	Creator      Creator
	Cluster      *clusterv1.Cluster
	InfraCluster runtime.Object

CreateClusterInput is the input for CreateCluster.

type CreateKubeadmControlPlaneInput

type CreateKubeadmControlPlaneInput struct {
	Creator         Creator
	ControlPlane    *controlplanev1.KubeadmControlPlane
	MachineTemplate runtime.Object

CreateKubeadmControlPlaneInput is the input for CreateKubeadmControlPlane.

type CreateMachineDeploymentInput

type CreateMachineDeploymentInput struct {
	Creator                 Creator
	MachineDeployment       *clusterv1.MachineDeployment
	BootstrapConfigTemplate runtime.Object
	InfraMachineTemplate    runtime.Object

CreateMachineDeploymentInput is the input for CreateMachineDeployment.

type CreateRelatedResourcesInput

type CreateRelatedResourcesInput struct {
	Creator          Creator
	RelatedResources []runtime.Object

CreateRelatedResourcesInput is the input type for CreateRelatedResources.

type Creator

type Creator interface {
	Create(ctx context.Context, obj runtime.Object, opts ...client.CreateOption) error

Creator can creates resources.

type DeleteClusterInput

type DeleteClusterInput struct {
	Deleter Deleter
	Cluster *clusterv1.Cluster

DeleteClusterInput is the input for DeleteCluster.

type Deleter

type Deleter interface {
	Delete(ctx context.Context, obj runtime.Object, opts ...client.DeleteOption) error

Deleter can delete resources.

type GetLister

type GetLister interface {

GetLister can get and list resources.

type Getter

type Getter interface {
	Get(ctx context.Context, key client.ObjectKey, obj runtime.Object) error

Getter can get resources.

type HTTPGetter

type HTTPGetter interface {
	Get(url string) (resp *http.Response, err error)

HTTPGetter wraps up the Get method exposed by the net/http.Client.

type ImageLoader

type ImageLoader interface {
	// LoadImage will put a local image onto the cluster.
	LoadImage(context.Context, string) error

ImageLoader is an interface around loading an image onto a cluster.

type InitManagementClusterInput

type InitManagementClusterInput struct {

	// Scheme is used to initialize the scheme for the management cluster
	// client.
	// Defaults to a new runtime.Scheme.
	Scheme *runtime.Scheme

	// ComponentGenerators is a list objects that supply additional component
	// YAML to apply to the management cluster.
	// Please note this is meant to be used at runtime to add YAML to the
	// management cluster outside of what is provided by the Components field.
	// For example, a caller could use this field to apply a Secret required by
	// some component from the Components field.
	ComponentGenerators []ComponentGenerator

	// NewManagementClusterFn may be used to provide a custom function for
	// returning a new management cluster. Otherwise kind.NewCluster is used.
	NewManagementClusterFn func() (ManagementCluster, error)

InitManagementClusterInput is the information required to initialize a new management cluster for e2e testing.

func (*InitManagementClusterInput) Defaults

func (c *InitManagementClusterInput) Defaults(ctx context.Context)

Defaults assigns default values to the object.

type Lister

type Lister interface {
	List(ctx context.Context, list runtime.Object, opts ...client.ListOption) error

Lister can lists resources.

type LoadImageBehavior

type LoadImageBehavior string

LoadImageBehavior indicates the behavior when loading an image.

const (
	// MustLoadImage causes a load operation to fail if the image cannot be
	// loaded.
	MustLoadImage LoadImageBehavior = "mustLoad"

	// TryLoadImage causes any errors that occur when loading an image to be
	// ignored.
	TryLoadImage LoadImageBehavior = "tryLoad"

type MachineDeployment

type MachineDeployment struct {
	MachineDeployment       *clusterv1.MachineDeployment
	BootstrapConfigTemplate runtime.Object
	InfraMachineTemplate    runtime.Object

MachineDeployment contains the objects needed to create a CAPI MachineDeployment resource and its associated template resources. Deprecated. Please use the individual create/assert methods.

type ManagementCluster

type ManagementCluster interface {
	// Teardown will completely clean up the ManagementCluster.
	// This should be implemented as a synchronous function.
	// Generally to be used in the AfterSuite function if a management cluster is shared between tests.
	// Should try to clean everything up and report any dangling artifacts that needs manual intervention.
	// GetName returns the name of the cluster.
	GetName() string
	// GetClient returns a client to the Management cluster.
	GetClient() (client.Client, error)
	// GetWorkdloadClient returns a client to the specified workload cluster.
	GetWorkloadClient(ctx context.Context, namespace, name string) (client.Client, error)

ManagementCluster are all the features we need out of a kubernetes cluster to qualify as a management cluster.

func InitManagementCluster

func InitManagementCluster(ctx context.Context, input *InitManagementClusterInput) ManagementCluster

InitManagementCluster returns a new cluster initialized as a CAPI management cluster.

type Node

type Node struct {
	Machine         *clusterv1.Machine
	InfraMachine    runtime.Object
	BootstrapConfig runtime.Object

Node contains all the pieces necessary to make a single node Deprecated.

type WaitForClusterDeletedInput

type WaitForClusterDeletedInput struct {
	Getter  Getter
	Cluster *clusterv1.Cluster

WaitForClusterDeletedInput is the input for WaitForClusterDeleted.

type WaitForClusterToProvisionInput

type WaitForClusterToProvisionInput struct {
	Getter  Getter
	Cluster *clusterv1.Cluster

WaitForClusterToProvisionInput is the input for WaitForClusterToProvision.

type WaitForControlPlaneToBeReadyInput

type WaitForControlPlaneToBeReadyInput struct {
	Getter       Getter
	ControlPlane *controlplanev1.KubeadmControlPlane

WaitForControlPlaneToBeReadyInput is the input for WaitForControlPlaneToBeReady.

type WaitForKubeadmControlPlaneMachinesToExistInput

type WaitForKubeadmControlPlaneMachinesToExistInput struct {
	Lister       Lister
	Cluster      *clusterv1.Cluster
	ControlPlane *controlplanev1.KubeadmControlPlane

WaitForKubeadmControlPlaneMachinesToExistInput is the input for WaitForKubeadmControlPlaneMachinesToExist.

type WaitForMachineDeploymentNodesToExistInput

type WaitForMachineDeploymentNodesToExistInput struct {
	Lister            Lister
	Cluster           *clusterv1.Cluster
	MachineDeployment *clusterv1.MachineDeployment

WaitForMachineDeploymentNodesToExistInput is the input for WaitForMachineDeploymentNodesToExist.

type Waiter

type Waiter interface {
	// Wait allows us to wait for something in the cluster, `kubectl wait`
	Wait(context.Context, ...string) error

Waiter is an interface around waiting for something on a kubernetes cluster.


Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL