framework

package
v1.26.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 15, 2023 License: Apache-2.0 Imports: 57 Imported by: 2,159

README

Overview

The Kubernetes E2E framework simplifies writing Ginkgo tests suites. It's main usage is for these tests suites in the Kubernetes repository itself:

  • test/e2e: runs as client for a Kubernetes cluster. The e2e.test binary is used for conformance testing.
  • test/e2e_node: runs on the same node as a kublet instance. Used for testing kubelet.
  • test/e2e_kubeadm: test suite for kubeadm.

Usage of the framework outside of Kubernetes is possible, but not encouraged. Downstream users have to be prepared to deal with API changes.

Code Organization

The core framework is the k8s.io/kubernetes/test/e2e/framework package. It contains functionality that all E2E suites are expected to need:

  • connecting to the apiserver
  • managing per-test namespaces
  • logging (Logf)
  • failure handling (Fail, Failf)
  • writing concise JUnit test results

It also contains a TestContext with settings that can be controlled via command line flags. For historic reasons, this also contains settings for individual tests or packages that are not part of the core framework.

Optional functionality is placed in sub packages like test/e2e/framework/pod. The core framework does not depend on those. Sub packages may depend on the core framework.

The advantages of splitting the code like this are:

  • leaner go doc packages by grouping related functions together
  • not forcing all E2E suites to import all functionality
  • avoiding import cycles

Execution Flow

When a test suite gets invoked, the top-level Describe calls register the callbacks that define individual tests, but does not invoke them yet. After that init phase, command line flags are parsed and the Describe callbacks are invoked. Those then define the actual tests for the test suite. Command line flags can be used to influence the test definitions.

Now Context/BeforeEach/AfterEach/It define code that will be called later when executing a specific test. During this setup phase, f := framework.NewDefaultFramework("some tests") creates a Framework instance for one or more tests. NewDefaultFramework initializes that instance anew for each test with a BeforeEach callback. Starting with Kubernetes 1.26, that instance gets cleaned up after all other code for a test has been invoked, so the following code is correct:

f := framework.NewDefaultFramework("some tests")

ginkgo.AfterEach(func() {
    # Do something with f.ClientSet.
}

ginkgo.It("test something", func() {
    # The actual test.
})

Optional functionality can be injected into each test by adding a callback to NewFrameworkExtensions in an init function. NewDefaultFramework will invoke those callbacks as if the corresponding code had been added to each test like this:

f := framework.NewDefaultFramework("some tests")

optional.SomeCallback(f)

SomeCallback then can register additional BeforeEach or AfterEach callbacks that use the test's Framework instance.

When a test runs, callbacks defined for it with BeforeEach and AfterEach are called in first-in-first-out order. Since the migration to ginkgo v2 in Kubernetes 1.25, the AfterEach callback is called also when there has been a test failure. This can be used to run cleanup code for a test reliably. However, ginkgo.DeferCleanup is often a better alternative. Its callbacks are executed in first-in-last-out order.

test/e2e/framework/internal/unittests/cleanup/cleanup.go shows how these different callbacks can be used and in which order they are going to run.

Documentation

Overview

Package framework contains provider-independent helper code for building and running E2E tests with Ginkgo. The actual Ginkgo test suites gets assembled by combining this framework, the optional provider support code and specific tests via a separate .go file like Kubernetes' test/e2e.go.

Index

Constants

View Source 
const (
	// KubeletPort is the default port for the kubelet server on each host machine.
	// May be overridden by a flag at startup.
	KubeletPort = 10250
	// KubeControllerManagerPort is the default port for the controller manager status server.
	// May be overridden by a flag at startup.
	KubeControllerManagerPort = 10257
)
View Source 
const (
	// PodListTimeout is how long to wait for the pod to be listable.
	PodListTimeout = time.Minute

	// PodStartTimeout is how long to wait for the pod to be started.
	PodStartTimeout = 5 * time.Minute

	// PodStartShortTimeout is same as `PodStartTimeout` to wait for the pod to be started, but shorter.
	// Use it case by case when we are sure pod start will not be delayed.
	// minutes by slow docker pulls or something else.
	PodStartShortTimeout = 2 * time.Minute

	// PodDeleteTimeout is how long to wait for a pod to be deleted.
	PodDeleteTimeout = 5 * time.Minute

	// PodGetTimeout is how long to wait for a pod to be got.
	PodGetTimeout = 2 * time.Minute

	// PodEventTimeout is how much we wait for a pod event to occur.
	PodEventTimeout = 2 * time.Minute

	// ServiceStartTimeout is how long to wait for a service endpoint to be resolvable.
	ServiceStartTimeout = 3 * time.Minute

	// Poll is how often to Poll pods, nodes and claims.
	Poll = 2 * time.Second

	// PollShortTimeout is the short timeout value in polling.
	PollShortTimeout = 1 * time.Minute

	// ServiceAccountProvisionTimeout is how long to wait for a service account to be provisioned.
	// service accounts are provisioned after namespace creation
	// a service account is required to support pod creation in a namespace as part of admission control
	ServiceAccountProvisionTimeout = 2 * time.Minute

	// SingleCallTimeout is how long to try single API calls (like 'get' or 'list'). Used to prevent
	// transient failures from failing tests.
	SingleCallTimeout = 5 * time.Minute

	// NodeReadyInitialTimeout is how long nodes have to be "ready" when a test begins. They should already
	// be "ready" before the test starts, so this is small.
	NodeReadyInitialTimeout = 20 * time.Second

	// PodReadyBeforeTimeout is how long pods have to be "ready" when a test begins.
	PodReadyBeforeTimeout = 5 * time.Minute

	// ClaimProvisionShortTimeout is same as `ClaimProvisionTimeout` to wait for claim to be dynamically provisioned, but shorter.
	// Use it case by case when we are sure this timeout is enough.
	ClaimProvisionShortTimeout = 1 * time.Minute

	// ClaimProvisionTimeout is how long claims have to become dynamically provisioned.
	ClaimProvisionTimeout = 5 * time.Minute

	// RestartNodeReadyAgainTimeout is how long a node is allowed to become "Ready" after it is restarted before
	// the test is considered failed.
	RestartNodeReadyAgainTimeout = 5 * time.Minute

	// RestartPodReadyAgainTimeout is how long a pod is allowed to become "running" and "ready" after a node
	// restart before test is considered failed.
	RestartPodReadyAgainTimeout = 5 * time.Minute

	// SnapshotCreateTimeout is how long for snapshot to create snapshotContent.
	SnapshotCreateTimeout = 5 * time.Minute

	// SnapshotDeleteTimeout is how long for snapshot to delete snapshotContent.
	SnapshotDeleteTimeout = 5 * time.Minute
)

DEPRECATED constants. Use the timeouts in framework.Framework instead.

View Source 
const (
	// DefaultNamespaceDeletionTimeout is timeout duration for waiting for a namespace deletion.
	DefaultNamespaceDeletionTimeout = 5 * time.Minute
)
View Source 
const (

	// DefaultNumNodes is the number of nodes. If not specified, then number of nodes is auto-detected
	DefaultNumNodes = -1
)

Variables

View Source 
var (
	// BusyBoxImage is the image URI of BusyBox.
	BusyBoxImage = imageutils.GetE2EImage(imageutils.BusyBox)

	// ProvidersWithSSH are those providers where each node is accessible with SSH
	ProvidersWithSSH = []string{"gce", "gke", "aws", "local"}

	// ServeHostnameImage is a serve hostname image name.
	ServeHostnameImage = imageutils.GetE2EImage(imageutils.Agnhost)
)
View Source 
var (
	// NewFrameworkExtensions lists functions that get called by
	// NewFramework after constructing a new framework and after
	// calling ginkgo.BeforeEach for the framework.
	//
	// This can be used by extensions of the core framework to modify
	// settings in the framework instance or to add additional callbacks
	// with gingko.BeforeEach/AfterEach/DeferCleanup.
	//
	// When a test runs, functions will be invoked in this order:
	// - BeforeEaches defined by tests before f.NewDefaultFramework
	//   in the order in which they were defined (first-in-first-out)
	// - f.BeforeEach
	// - BeforeEaches defined by tests after f.NewDefaultFramework
	// - It callback
	// - all AfterEaches in the order in which they were defined
	// - all DeferCleanups with the order reversed (first-in-last-out)
	// - f.AfterEach
	//
	// Because a test might skip test execution in a BeforeEach that runs
	// before f.BeforeEach, AfterEach callbacks that depend on the
	// framework instance must check whether it was initialized. They can
	// do that by checking f.ClientSet for nil. DeferCleanup callbacks
	// don't need to do this because they get defined when the test
	// runs.
	NewFrameworkExtensions []func(f *Framework)
)
View Source 
var RunID = uuid.NewUUID()

RunID is a unique identifier of the e2e run. Beware that this ID is not the same for all tests in the e2e run, because each Ginkgo node creates it separately.

Functions

func APIAddress added in v1.20.0 

func APIAddress() string

APIAddress returns a address of an instance.

func AfterReadingAllFlags added in v1.7.0 

func AfterReadingAllFlags(t *TestContextType)

AfterReadingAllFlags makes changes to the context after all flags have been read and prepares the process for a test run.

func AnnotatedLocation added in v1.26.0 

func AnnotatedLocation(annotation string) types.CodeLocation

AnnotatedLocation can be used to provide more informative source code locations by passing the result as additional parameter to a BeforeEach/AfterEach/DeferCleanup/It/etc.

func AppendContainerCommandGroupIfNeeded added in v1.19.0 

func AppendContainerCommandGroupIfNeeded(args []string) []string

AppendContainerCommandGroupIfNeeded returns container command group parameter if necessary.

func CheckTestingNSDeletedExcept 

func CheckTestingNSDeletedExcept(c clientset.Interface, skip string) error

CheckTestingNSDeletedExcept checks whether all e2e based existing namespaces are in the Terminating state and waits until they are finally deleted. It ignores namespace skip.

func ConformanceIt added in v1.9.0 

func ConformanceIt(text string, body interface{}) bool

ConformanceIt is wrapper function for ginkgo It. Adds "[Conformance]" tag and makes static analysis easier.

func CoreDump 

func CoreDump(dir string)

CoreDump SSHs to the master and all nodes and dumps their logs into dir. It shells out to cluster/log-dump/log-dump.sh to accomplish this.

func CreateGinkgoConfig added in v1.25.0 

func CreateGinkgoConfig() (types.SuiteConfig, types.ReporterConfig)

func CreateTestingNS 

func CreateTestingNS(baseName string, c clientset.Interface, labels map[string]string) (*v1.Namespace, error)

CreateTestingNS should be used by every test, note that we append a common prefix to the provided test name. Please see NewFramework instead of using this directly.

func DeleteNamespaces 

func DeleteNamespaces(c clientset.Interface, deleteFilter, skipFilter []string) ([]string, error)

DeleteNamespaces deletes all namespaces that match the given delete and skip filters. Filter is by simple strings.Contains; first skip filter, then delete filter. Returns the list of deleted namespaces or an error.

func EnsureLoadBalancerResourcesDeleted 

func EnsureLoadBalancerResourcesDeleted(ip, portRange string) error

EnsureLoadBalancerResourcesDeleted ensures that cloud load balancer resources that were created are actually cleaned up. Currently only implemented for GCE/GKE.

func ExpectConsistOf added in v1.18.0 

func ExpectConsistOf(actual interface{}, extra interface{}, explain ...interface{})

ExpectConsistOf expects actual contains precisely the extra elements. The ordering of the elements does not matter.

func ExpectEmpty added in v1.18.0 

func ExpectEmpty(actual interface{}, explain ...interface{})

ExpectEmpty expects actual is empty

func ExpectEqual added in v1.16.0 

func ExpectEqual(actual interface{}, extra interface{}, explain ...interface{})

ExpectEqual expects the specified two are the same, otherwise an exception raises

func ExpectError added in v1.15.0 

func ExpectError(err error, explain ...interface{})

ExpectError expects an error happens, otherwise an exception raises

func ExpectHaveKey added in v1.18.0 

func ExpectHaveKey(actual interface{}, key interface{}, explain ...interface{})

ExpectHaveKey expects the actual map has the key in the keyset

func ExpectNoError 

func ExpectNoError(err error, explain ...interface{})

ExpectNoError checks if "err" is set, and if so, fails assertion while logging the error.

func ExpectNoErrorWithOffset added in v1.7.0 

func ExpectNoErrorWithOffset(offset int, err error, explain ...interface{})

ExpectNoErrorWithOffset checks if "err" is set, and if so, fails assertion while logging the error at "offset" levels above its caller (for example, for call chain f -> g -> ExpectNoErrorWithOffset(1, ...) error would be logged for "f").

func ExpectNotEqual added in v1.16.0 

func ExpectNotEqual(actual interface{}, extra interface{}, explain ...interface{})

ExpectNotEqual expects the specified two are not the same, otherwise an exception raises

func Fail added in v1.17.0 

func Fail(msg string, callerSkip ...int)

Fail is a replacement for ginkgo.Fail which logs the problem as it occurs together with a stack trace and then calls ginkgowrapper.Fail.

func Failf 

func Failf(format string, args ...interface{})

Failf logs the fail info, including a stack trace starts with its direct caller (for example, for call chain f -> g -> Failf("foo", ...) error would be logged for "g").

func GenerateSecureToken added in v1.20.0 

func GenerateSecureToken(tokenLen int) (string, error)

GenerateSecureToken returns a string of length tokenLen, consisting of random bytes encoded as base64 for use as a Bearer Token during communication with an APIServer

func GetControlPlaneAddresses added in v1.20.0 

func GetControlPlaneAddresses(c clientset.Interface) []string

GetControlPlaneAddresses returns all IP addresses on which the kubelet can reach the control plane. It may return internal and external IPs, even if we expect for e.g. internal IPs to be used (issue #56787), so that we can be sure to block the control plane fully during tests.

func GetGroupNodes added in v1.7.0 

func GetGroupNodes(group string) ([]string, error)

GetGroupNodes returns a node name for the specified node group

func GetProviders added in v1.14.0 

func GetProviders() []string

GetProviders returns the names of all currently registered providers.

func GroupSize added in v1.7.0 

func GroupSize(group string) (int, error)

GroupSize returns the size of an instance group

func LoadClientset added in v1.5.0 

func LoadClientset() (*clientset.Clientset, error)

LoadClientset returns clientset for connecting to kubernetes clusters.

func LoadConfig 

func LoadConfig() (config *restclient.Config, err error)

LoadConfig returns a config for a rest client with the UserAgent set to include the current test name.

func Logf 

func Logf(format string, args ...interface{})

Logf logs the info.

func MasterOSDistroIs added in v1.8.0 

func MasterOSDistroIs(supportedMasterOsDistros ...string) bool

MasterOSDistroIs returns true if the master OS distro is included in the supportedMasterOsDistros. Otherwise false.

func NodeOSArchIs added in v1.19.0 

func NodeOSArchIs(supportedNodeOsArchs ...string) bool

NodeOSArchIs returns true if the node OS arch is included in the supportedNodeOsArchs. Otherwise false.

func NodeOSDistroIs added in v1.6.0 

func NodeOSDistroIs(supportedNodeOsDistros ...string) bool

NodeOSDistroIs returns true if the node OS distro is included in the supportedNodeOsDistros. Otherwise false.

func PrettyPrintJSON 

func PrettyPrintJSON(metrics interface{}) string

PrettyPrintJSON converts metrics to JSON format.

func ProviderIs 

func ProviderIs(providers ...string) bool

ProviderIs returns true if the provider is included is the providers. Otherwise false.

func PrunedStack added in v1.17.0 

func PrunedStack(skip int) []byte

PrunedStack is a wrapper around debug.Stack() that removes information about the current goroutine and optionally skips some of the initial stack entries. With skip == 0, the returned stack will start with the caller of PruneStack. From the remaining entries it automatically filters out useless ones like entries coming from Ginkgo.

This is a modified copy of PruneStack in https://github.com/onsi/ginkgo/v2/blob/f90f37d87fa6b1dd9625e2b1e83c23ffae3de228/internal/codelocation/code_location.go#L25:

func RandomSuffix added in v1.14.0 

func RandomSuffix() string

RandomSuffix provides a random sequence to append to pods,services,rcs.

func RegisterClusterFlags added in v1.4.0 

func RegisterClusterFlags(flags *flag.FlagSet)

RegisterClusterFlags registers flags specific to the cluster e2e test suite.

func RegisterCommonFlags added in v1.4.0 

func RegisterCommonFlags(flags *flag.FlagSet)

RegisterCommonFlags registers flags common to all e2e test suites. The flag set can be flag.CommandLine (if desired) or a custom flag set that then gets passed to viperconfig.ViperizeFlags.

The other Register*Flags methods below can be used to add more test-specific flags. However, those settings then get added regardless whether the test is actually in the test suite.

For tests that have been converted to registering their options themselves, copy flags from test/e2e/framework/config as shown in HandleFlags.

func RegisterProvider added in v1.13.0 

func RegisterProvider(name string, factory Factory)

RegisterProvider is expected to be called during application init, typically by an init function in a provider package.

func ResizeGroup added in v1.7.0 

func ResizeGroup(group string, size int32) error

ResizeGroup resizes an instance group

func RunCmd 

func RunCmd(command string, args ...string) (string, string, error)

RunCmd runs cmd using args and returns its stdout and stderr. It also outputs cmd's stdout and stderr to their respective OS streams.

func RunCmdEnv added in v1.5.4 

func RunCmdEnv(env []string, command string, args ...string) (string, string, error)

RunCmdEnv runs cmd with the provided environment and args and returns its stdout and stderr. It also outputs cmd's stdout and stderr to their respective OS streams.

func StartCmdAndStreamOutput 

func StartCmdAndStreamOutput(cmd *exec.Cmd) (stdout, stderr io.ReadCloser, err error)

StartCmdAndStreamOutput returns stdout and stderr after starting the given cmd.

func TryKill 

func TryKill(cmd *exec.Cmd)

TryKill is rough equivalent of ctrl+c for cleaning up processes. Intended to be run in defer.

func WaitForDefaultServiceAccountInNamespace 

func WaitForDefaultServiceAccountInNamespace(c clientset.Interface, namespace string) error

WaitForDefaultServiceAccountInNamespace waits for the default service account to be provisioned the default service account is what is associated with pods when they do not specify a service account as a result, pods are not able to be provisioned in a namespace until the service account is provisioned

func WaitForGroupSize added in v1.7.0 

func WaitForGroupSize(group string, size int32) error

WaitForGroupSize waits for node instance group reached the desired size

func WaitForKubeRootCAInNamespace added in v1.23.6 

func WaitForKubeRootCAInNamespace(c clientset.Interface, namespace string) error

WaitForKubeRootCAInNamespace waits for the configmap kube-root-ca.crt containing the service account CA trust bundle to be provisioned in the specified namespace so that pods do not have to retry mounting the config map (which creates noise that hides other issues in the Kubelet).

func WaitForNamespacesDeleted 

func WaitForNamespacesDeleted(c clientset.Interface, namespaces []string, timeout time.Duration) error

WaitForNamespacesDeleted waits for the namespaces to be deleted.

func WaitForServiceEndpointsNum 

func WaitForServiceEndpointsNum(c clientset.Interface, namespace, serviceName string, expectNum int, interval, timeout time.Duration) error

WaitForServiceEndpointsNum waits until the amount of endpoints that implement service to expectNum.

func WatchEventSequenceVerifier added in v1.19.0 

func WatchEventSequenceVerifier(ctx context.Context, dc dynamic.Interface, resourceType schema.GroupVersionResource, namespace string, resourceName string, listOptions metav1.ListOptions, expectedWatchEvents []watch.Event, scenario func(*watchtools.RetryWatcher) []watch.Event, retryCleanup func() error)

WatchEventSequenceVerifier ... manages a watch for a given resource, ensures that events take place in a given order, retries the test on failure 

testContext         cancellation signal across API boundaries, e.g: context.TODO()
dc                  sets up a client to the API
resourceType        specify the type of resource
namespace           select a namespace
resourceName        the name of the given resource
listOptions         options used to find the resource, recommended to use listOptions.labelSelector
expectedWatchEvents array of events which are expected to occur
scenario            the test itself
retryCleanup        a function to run which ensures that there are no dangling resources upon test failure

this tooling relies on the test to return the events as they occur the entire scenario must be run to ensure that the desired watch events arrive in order (allowing for interweaving of watch events) 

if an expected watch event is missing we elect to clean up and run the entire scenario again

we try the scenario three times to allow the sequencing to fail a couple of times

Types

type ClientConfigGetter 

type ClientConfigGetter func() (*restclient.Config, error)

ClientConfigGetter is a func that returns getter to return a config.

type CloudConfig 

type CloudConfig struct {
	APIEndpoint       string
	ProjectID         string
	Zone              string   // for multizone tests, arbitrarily chosen zone
	Zones             []string // for multizone tests, use this set of zones instead of querying the cloud provider. Must include Zone.
	Region            string
	MultiZone         bool
	MultiMaster       bool
	Cluster           string
	MasterName        string
	NodeInstanceGroup string // comma-delimited list of groups' names
	NumNodes          int
	ClusterIPRange    string
	ClusterTag        string
	Network           string
	ConfigFile        string // for azure
	NodeTag           string
	MasterTag         string

	Provider ProviderInterface
}

CloudConfig holds the cloud configuration for e2e test suites.

type ClusterVerification 

type ClusterVerification struct {
	// contains filtered or unexported fields
}

ClusterVerification is a struct for a verification of cluster state.

func (*ClusterVerification) ForEach 

func (cl *ClusterVerification) ForEach(podFunc func(v1.Pod)) error

ForEach runs a function against every verifiable pod. Be warned that this doesn't wait for "n" pods to verify, so it may return very quickly if you have strict pod state requirements.

For example, if you require at least 5 pods to be running before your test will pass, its smart to first call "clusterVerification.WaitFor(5)" before you call clusterVerification.ForEach.

func (*ClusterVerification) WaitFor 

func (cl *ClusterVerification) WaitFor(atLeast int, timeout time.Duration) ([]v1.Pod, error)

WaitFor waits for some minimum number of pods to be verified, according to the PodStateVerification definition.

func (*ClusterVerification) WaitForOrFail 

func (cl *ClusterVerification) WaitForOrFail(atLeast int, timeout time.Duration)

WaitForOrFail provides a shorthand WaitFor with failure as an option if anything goes wrong.

type CreateTestingNSFn 

type CreateTestingNSFn func(baseName string, c clientset.Interface, labels map[string]string) (*v1.Namespace, error)

CreateTestingNSFn is a func that is responsible for creating namespace used for executing e2e tests.

type DumpAllNamespaceInfoAction added in v1.26.0 

type DumpAllNamespaceInfoAction func(f *Framework, namespace string)

DumpAllNamespaceInfoAction is called after each failed test for namespaces created for the test.

type Factory added in v1.13.0 

type Factory func() (ProviderInterface, error)

Factory is a func which operates provider specific behavior.

type FlakeReport added in v1.12.0 

type FlakeReport struct {
	Flakes     []string `json:"flakes"`
	FlakeCount int      `json:"flakeCount"`
	// contains filtered or unexported fields
}

FlakeReport is a struct for managing the flake report.

func NewFlakeReport added in v1.12.0 

func NewFlakeReport() *FlakeReport

NewFlakeReport returns a new flake report.

func (*FlakeReport) GetFlakeCount added in v1.12.0 

func (f *FlakeReport) GetFlakeCount() int

GetFlakeCount returns the flake count.

func (*FlakeReport) PrintHumanReadable added in v1.12.0 

func (f *FlakeReport) PrintHumanReadable() string

PrintHumanReadable returns string of flake report.

func (*FlakeReport) PrintJSON added in v1.12.0 

func (f *FlakeReport) PrintJSON() string

PrintJSON returns the summary of frake report with JSON format.

func (*FlakeReport) RecordFlakeIfError added in v1.12.0 

func (f *FlakeReport) RecordFlakeIfError(err error, optionalDescription ...interface{})

RecordFlakeIfError records the error (if non-nil) as a flake along with an optional description. This can be used as a replacement of framework.ExpectNoError() for non-critical errors that can be considered as 'flakes' to avoid causing failures in tests.

func (*FlakeReport) SummaryKind added in v1.12.0 

func (f *FlakeReport) SummaryKind() string

SummaryKind returns the summary of flake report.

type Framework 

type Framework struct {
	BaseName string

	// Set together with creating the ClientSet and the namespace.
	// Guaranteed to be unique in the cluster even when running the same
	// test multiple times in parallel.
	UniqueName string

	ClientSet                        clientset.Interface
	KubemarkExternalClusterClientSet clientset.Interface

	DynamicClient dynamic.Interface

	ScalesGetter scaleclient.ScalesGetter

	SkipNamespaceCreation bool          // Whether to skip creating a namespace
	Namespace             *v1.Namespace // Every test has at least one namespace unless creation is skipped

	NamespaceDeletionTimeout         time.Duration
	NamespacePodSecurityEnforceLevel admissionapi.Level // The pod security enforcement level for namespaces to be applied.

	// configuration for framework's client
	Options Options

	// Place where various additional data is stored during test run to be printed to ReportDir,
	// or stdout if ReportDir is not set once test ends.
	TestSummaries []TestDataSummary

	// Timeouts contains the custom timeouts used during the test execution.
	Timeouts *TimeoutContext

	// DumpAllNamespaceInfo is invoked by the framework to record
	// information about a namespace after a test failure.
	DumpAllNamespaceInfo DumpAllNamespaceInfoAction
	// contains filtered or unexported fields
}

Framework supports common operations used by e2e tests; it will keep a client & a namespace for you. Eventual goal is to merge this with integration test framework.

func NewDefaultFramework 

func NewDefaultFramework(baseName string) *Framework

NewDefaultFramework makes a new framework and sets up a BeforeEach which initializes the framework instance. It cleans up with a DeferCleanup, which runs last, so a AfterEach in the test still has a valid framework instance.

func NewFramework 

func NewFramework(baseName string, options Options, client clientset.Interface) *Framework

NewFramework creates a test framework.

func NewFrameworkWithCustomTimeouts added in v1.21.0 

func NewFrameworkWithCustomTimeouts(baseName string, timeouts *TimeoutContext) *Framework

NewFrameworkWithCustomTimeouts makes a framework with with custom timeouts.

func (*Framework) AddNamespacesToDelete added in v1.11.0 

func (f *Framework) AddNamespacesToDelete(namespaces ...*v1.Namespace)

AddNamespacesToDelete adds one or more namespaces to be deleted when the test completes.

func (*Framework) AfterEach 

func (f *Framework) AfterEach()

AfterEach deletes the namespace, after reading its events.

func (*Framework) BeforeEach 

func (f *Framework) BeforeEach()

BeforeEach gets a client and makes a namespace.

func (*Framework) ClientConfig added in v1.18.0 

func (f *Framework) ClientConfig() *rest.Config

ClientConfig an externally accessible method for reading the kube client config.

func (*Framework) CreateNamespace 

func (f *Framework) CreateNamespace(baseName string, labels map[string]string) (*v1.Namespace, error)

CreateNamespace creates a namespace for e2e testing.

func (*Framework) DeleteNamespace added in v1.20.0 

func (f *Framework) DeleteNamespace(name string)

DeleteNamespace can be used to delete a namespace. Additionally it can be used to dump namespace information so as it can be used as an alternative of framework deleting the namespace towards the end.

func (*Framework) NewClusterVerification 

func (f *Framework) NewClusterVerification(namespace *v1.Namespace, filter PodStateVerification) *ClusterVerification

NewClusterVerification creates a new cluster verification.

func (*Framework) RecordFlakeIfError added in v1.12.0 

func (f *Framework) RecordFlakeIfError(err error, optionalDescription ...interface{})

RecordFlakeIfError records flakeness info if error happens. NOTE: This function is not used at any places yet, but we are in progress for https://github.com/kubernetes/kubernetes/issues/66239 which requires this. Please don't remove this.

type KubeCluster 

type KubeCluster struct {
	Name    string `yaml:"name"`
	Cluster struct {
		CertificateAuthorityData string `yaml:"certificate-authority-data"`
		Server                   string `yaml:"server"`
	} `yaml:"cluster"`
}

KubeCluster is a struct for managing kubernetes cluster info.

type KubeConfig 

type KubeConfig struct {
	Contexts []struct {
		Name    string `yaml:"name"`
		Context struct {
			Cluster string `yaml:"cluster"`
			User    string
		} `yaml:"context"`
	} `yaml:"contexts"`

	Clusters []KubeCluster `yaml:"clusters"`

	Users []KubeUser `yaml:"users"`
}

KubeConfig is a struct for managing kubernetes config.

func (*KubeConfig) FindCluster added in v1.6.0 

func (kc *KubeConfig) FindCluster(name string) *KubeCluster

FindCluster returns cluster info which is the specified cluster name.

func (*KubeConfig) FindUser added in v1.6.0 

func (kc *KubeConfig) FindUser(name string) *KubeUser

FindUser returns user info which is the specified user name.

type KubeUser 

type KubeUser struct {
	Name string `yaml:"name"`
	User struct {
		Username string `yaml:"username"`
		Password string `yaml:"password" datapolicy:"password"`
		Token    string `yaml:"token" datapolicy:"token"`
	} `yaml:"user"`
}

KubeUser is a struct for managing kubernetes user info.

type NodeKillerConfig added in v1.14.0 

type NodeKillerConfig struct {
	// Enabled determines whether NodeKill should do anything at all.
	// All other options below are ignored if Enabled = false.
	Enabled bool
	// FailureRatio is a percentage of all nodes that could fail simultinously.
	FailureRatio float64
	// Interval is time between node failures.
	Interval time.Duration
	// JitterFactor is factor used to jitter node failures.
	// Node will be killed between [Interval, Interval + (1.0 + JitterFactor)].
	JitterFactor float64
	// SimulatedDowntime is a duration between node is killed and recreated.
	SimulatedDowntime time.Duration
	// NodeKillerStopCh is a channel that is used to notify NodeKiller to stop killing nodes.
	NodeKillerStopCh chan struct{}
}

NodeKillerConfig describes configuration of NodeKiller -- a utility to simulate node failures.

TODO: move this and the corresponding command line flags into test/e2e/framework/node.

type NodeTestContextType added in v1.4.0 

type NodeTestContextType struct {
	// NodeE2E indicates whether it is running node e2e.
	NodeE2E bool
	// Name of the node to run tests on.
	NodeName string
	// NodeConformance indicates whether the test is running in node conformance mode.
	NodeConformance bool
	// PrepullImages indicates whether node e2e framework should prepull images.
	PrepullImages bool
	// KubeletConfig is the kubelet configuration the test is running against.
	KubeletConfig kubeletconfig.KubeletConfiguration
	// ImageDescription is the description of the image on which the test is running.
	ImageDescription string
	// RuntimeConfig is a map of API server runtime configuration values.
	RuntimeConfig map[string]string
	// SystemSpecName is the name of the system spec (e.g., gke) that's used in
	// the node e2e test. If empty, the default one (system.DefaultSpec) is
	// used. The system specs are in test/e2e_node/system/specs/.
	SystemSpecName string
	// RestartKubelet restarts Kubelet unit when the process is killed.
	RestartKubelet bool
	// ExtraEnvs is a map of environment names to values.
	ExtraEnvs map[string]string
}

NodeTestContextType is part of TestContextType, it is shared by all node e2e test.

type NullProvider added in v1.13.0 

type NullProvider struct{}

NullProvider is the default implementation of the ProviderInterface which doesn't do anything.

func (NullProvider) CleanupServiceResources added in v1.13.0 

func (n NullProvider) CleanupServiceResources(c clientset.Interface, loadBalancerName, region, zone string)

CleanupServiceResources is a base implementation which cleans up service resources.

func (NullProvider) CreatePD added in v1.13.0 

func (n NullProvider) CreatePD(zone string) (string, error)

CreatePD is a base implementation which creates PD.

func (NullProvider) CreatePVSource added in v1.13.0 

func (n NullProvider) CreatePVSource(zone, diskName string) (*v1.PersistentVolumeSource, error)

CreatePVSource is a base implementation which creates PV source.

func (NullProvider) CreateShare added in v1.24.0 

func (n NullProvider) CreateShare() (string, string, string, error)

func (NullProvider) DeleteNode added in v1.14.0 

func (n NullProvider) DeleteNode(node *v1.Node) error

DeleteNode is a base implementation which deletes a node.

func (NullProvider) DeletePD added in v1.13.0 

func (n NullProvider) DeletePD(pdName string) error

DeletePD is a base implementation which deletes PD.

func (NullProvider) DeletePVSource added in v1.13.0 

func (n NullProvider) DeletePVSource(pvSource *v1.PersistentVolumeSource) error

DeletePVSource is a base implementation which deletes PV source.

func (NullProvider) DeleteShare added in v1.24.0 

func (n NullProvider) DeleteShare(accountName, shareName string) error

func (NullProvider) EnableAndDisableInternalLB added in v1.13.0 

func (n NullProvider) EnableAndDisableInternalLB() (enable, disable func(svc *v1.Service))

EnableAndDisableInternalLB is a base implementation which returns functions for enabling/disabling an internal LB.

func (NullProvider) EnsureLoadBalancerResourcesDeleted added in v1.13.0 

func (n NullProvider) EnsureLoadBalancerResourcesDeleted(ip, portRange string) error

EnsureLoadBalancerResourcesDeleted is a base implementation which ensures load balancer is deleted.

func (NullProvider) FrameworkAfterEach added in v1.13.0 

func (n NullProvider) FrameworkAfterEach(f *Framework)

FrameworkAfterEach is a base implementation which does AfterEach.

func (NullProvider) FrameworkBeforeEach added in v1.13.0 

func (n NullProvider) FrameworkBeforeEach(f *Framework)

FrameworkBeforeEach is a base implementation which does BeforeEach.

func (NullProvider) GetGroupNodes added in v1.13.0 

func (n NullProvider) GetGroupNodes(group string) ([]string, error)

GetGroupNodes is a base implementation which returns group nodes.

func (NullProvider) GroupSize added in v1.13.0 

func (n NullProvider) GroupSize(group string) (int, error)

GroupSize returns the size of an instance group

func (NullProvider) LoadBalancerSrcRanges added in v1.13.0 

func (n NullProvider) LoadBalancerSrcRanges() []string

LoadBalancerSrcRanges is a base implementation which returns the ranges of ips used by load balancers.

func (NullProvider) ResizeGroup added in v1.13.0 

func (n NullProvider) ResizeGroup(string, int32) error

ResizeGroup is a base implementation which resizes group.

type Options added in v1.15.0 

type Options struct {
	ClientQPS    float32
	ClientBurst  int
	GroupVersion *schema.GroupVersion
}

Options is a struct for managing test framework options.

type PodStateVerification 

type PodStateVerification struct {
	// Optional: only pods that have k=v labels will pass this filter.
	Selectors map[string]string

	// Required: The phases which are valid for your pod.
	ValidPhases []v1.PodPhase

	// Optional: only pods passing this function will pass the filter
	// Verify a pod.
	// As an optimization, in addition to specifying filter (boolean),
	// this function allows specifying an error as well.
	// The error indicates that the polling of the pod spectrum should stop.
	Verify func(v1.Pod) (bool, error)

	// Optional: only pods with this name will pass the filter.
	PodName string
}

PodStateVerification represents a verification of pod state. Any time you have a set of pods that you want to operate against or query, this struct can be used to declaratively identify those pods.

type ProviderInterface added in v1.13.0 

type ProviderInterface interface {
	FrameworkBeforeEach(f *Framework)
	FrameworkAfterEach(f *Framework)

	ResizeGroup(group string, size int32) error
	GetGroupNodes(group string) ([]string, error)
	GroupSize(group string) (int, error)

	DeleteNode(node *v1.Node) error

	CreatePD(zone string) (string, error)
	DeletePD(pdName string) error
	CreateShare() (string, string, string, error)
	DeleteShare(accountName, shareName string) error

	CreatePVSource(zone, diskName string) (*v1.PersistentVolumeSource, error)
	DeletePVSource(pvSource *v1.PersistentVolumeSource) error

	CleanupServiceResources(c clientset.Interface, loadBalancerName, region, zone string)

	EnsureLoadBalancerResourcesDeleted(ip, portRange string) error
	LoadBalancerSrcRanges() []string
	EnableAndDisableInternalLB() (enable, disable func(svc *v1.Service))
}

ProviderInterface contains the implementation for certain provider-specific functionality.

func SetupProviderConfig added in v1.13.0 

func SetupProviderConfig(providerName string) (ProviderInterface, error)

SetupProviderConfig validates the chosen provider and creates an interface instance for it.

type TestContextType 

type TestContextType struct {
	KubeConfig         string
	KubeContext        string
	KubeAPIContentType string
	KubeletRootDir     string
	CertDir            string
	Host               string
	BearerToken        string `datapolicy:"token"`
	// TODO: Deprecating this over time... instead just use gobindata_util.go , see #23987.
	RepoRoot string
	// ListImages will list off all images that are used then quit
	ListImages bool

	// ListConformanceTests will list off all conformance tests that are available then quit
	ListConformanceTests bool

	// Provider identifies the infrastructure provider (gce, gke, aws)
	Provider string

	// Tooling is the tooling in use (e.g. kops, gke).  Provider is the cloud provider and might not uniquely identify the tooling.
	Tooling string

	CloudConfig    CloudConfig
	KubectlPath    string
	OutputDir      string
	ReportDir      string
	ReportPrefix   string
	Prefix         string
	MinStartupPods int
	// Timeout for waiting for system pods to be running
	SystemPodsStartupTimeout    time.Duration
	EtcdUpgradeStorage          string
	EtcdUpgradeVersion          string
	GCEUpgradeScript            string
	ContainerRuntimeEndpoint    string
	ContainerRuntimeProcessName string
	ContainerRuntimePidFile     string
	// SystemdServices are comma separated list of systemd services the test framework
	// will dump logs for.
	SystemdServices string
	// DumpSystemdJournal controls whether to dump the full systemd journal.
	DumpSystemdJournal       bool
	ImageServiceEndpoint     string
	MasterOSDistro           string
	NodeOSDistro             string
	NodeOSArch               string
	VerifyServiceAccount     bool
	DeleteNamespace          bool
	DeleteNamespaceOnFailure bool
	AllowedNotReadyNodes     int
	CleanStart               bool
	// If set to 'true' or 'all' framework will start a goroutine monitoring resource usage of system add-ons.
	// It will read the data every 30 seconds from all Nodes and print summary during afterEach. If set to 'master'
	// only master Node will be monitored.
	GatherKubeSystemResourceUsageData string
	GatherLogsSizes                   bool
	GatherMetricsAfterTest            string
	GatherSuiteMetricsAfterTest       bool
	MaxNodesToGather                  int
	// If set to 'true' framework will gather ClusterAutoscaler metrics when gathering them for other components.
	IncludeClusterAutoscalerMetrics bool
	// Currently supported values are 'hr' for human-readable and 'json'. It's a comma separated list.
	OutputPrintType string
	// NodeSchedulableTimeout is the timeout for waiting for all nodes to be schedulable.
	NodeSchedulableTimeout time.Duration
	// SystemDaemonsetStartupTimeout is the timeout for waiting for all system daemonsets to be ready.
	SystemDaemonsetStartupTimeout time.Duration
	// CreateTestingNS is responsible for creating namespace used for executing e2e tests.
	// It accepts namespace base name, which will be prepended with e2e prefix, kube client
	// and labels to be applied to a namespace.
	CreateTestingNS CreateTestingNSFn
	// If set to true test will dump data about the namespace in which test was running.
	DumpLogsOnFailure bool
	// Disables dumping cluster log from master and nodes after all tests.
	DisableLogDump bool
	// Path to the GCS artifacts directory to dump logs from nodes. Logexporter gets enabled if this is non-empty.
	LogexporterGCSPath string
	// Node e2e specific test context
	NodeTestContextType

	// The DNS Domain of the cluster.
	ClusterDNSDomain string

	// The configuration of NodeKiller.
	NodeKiller NodeKillerConfig

	// The Default IP Family of the cluster ("ipv4" or "ipv6")
	IPFamily string

	// NonblockingTaints is the comma-delimeted string given by the user to specify taints which should not stop the test framework from running tests.
	NonblockingTaints string

	// ProgressReportURL is the URL which progress updates will be posted to as tests complete. If empty, no updates are sent.
	ProgressReportURL string

	// SriovdpConfigMapFile is the path to the ConfigMap to configure the SRIOV device plugin on this host.
	SriovdpConfigMapFile string

	// SpecSummaryOutput is the file to write ginkgo.SpecSummary objects to as tests complete. Useful for debugging and test introspection.
	SpecSummaryOutput string

	// DockerConfigFile is a file that contains credentials which can be used to pull images from certain private registries, needed for a test.
	DockerConfigFile string

	// SnapshotControllerPodName is the name used for identifying the snapshot controller pod.
	SnapshotControllerPodName string

	// SnapshotControllerHTTPPort the port used for communicating with the snapshot controller HTTP endpoint.
	SnapshotControllerHTTPPort int

	// RequireDevices makes mandatory on the environment on which tests are run 1+ devices exposed through device plugins.
	// With this enabled The e2e tests requiring devices for their operation can assume that if devices aren't reported, the test can fail
	RequireDevices bool

	// Enable volume drivers which are disabled by default. See test/e2e/storage/in_tree_volumes.go for details.
	EnabledVolumeDrivers []string
}

TestContextType contains test settings and global state. Due to historic reasons, it is a mixture of items managed by the test framework itself, cloud providers and individual tests. The goal is to move anything not required by the framework into the code which uses the settings.

The recommendation for those settings is:

  • They are stored in their own context structure or local variables.
  • The standard `flag` package is used to register them. The flag name should follow the pattern <part1>.<part2>....<partn> where the prefix is unlikely to conflict with other tests or standard packages and each part is in lower camel case. For example, test/e2e/storage/csi/context.go could define storage.csi.numIterations.
  • framework/config can be used to simplify the registration of multiple options with a single function call: var storageCSI { NumIterations `default:"1" usage:"number of iterations"` } _ config.AddOptions(&storageCSI, "storage.csi")
  • The direct use Viper in tests is possible, but discouraged because it only works in test suites which use Viper (which is not required) and the supported options cannot be discovered by a test suite user.

Test suite authors can use framework/viper to make all command line parameters also configurable via a configuration file. 

var TestContext TestContextType

TestContext should be used by all tests to access common context data.

func (TestContextType) ClusterIsIPv6 added in v1.17.0 

func (tc TestContextType) ClusterIsIPv6() bool

ClusterIsIPv6 returns true if the cluster is IPv6

type TestDataSummary 

type TestDataSummary interface {
	SummaryKind() string
	PrintHumanReadable() string
	PrintJSON() string
}

TestDataSummary is an interface for managing test data.

type TimeoutContext added in v1.21.0 

type TimeoutContext struct {
	// PodStart is how long to wait for the pod to be started.
	PodStart time.Duration

	// PodStartShort is same as `PodStart`, but shorter.
	// Use it in a case-by-case basis, mostly when you are sure pod start will not be delayed.
	PodStartShort time.Duration

	// PodStartSlow is same as `PodStart`, but longer.
	// Use it in a case-by-case basis, mostly when you are sure pod start will take longer than usual.
	PodStartSlow time.Duration

	// PodDelete is how long to wait for the pod to be deleted.
	PodDelete time.Duration

	// ClaimProvision is how long claims have to become dynamically provisioned.
	ClaimProvision time.Duration

	// DataSourceProvision is how long claims have to become dynamically provisioned from source claim.
	DataSourceProvision time.Duration

	// ClaimProvisionShort is the same as `ClaimProvision`, but shorter.
	ClaimProvisionShort time.Duration

	// ClaimBound is how long claims have to become bound.
	ClaimBound time.Duration

	// PVReclaim is how long PVs have to become reclaimed.
	PVReclaim time.Duration

	// PVBound is how long PVs have to become bound.
	PVBound time.Duration

	// PVCreate is how long PVs have to be created.
	PVCreate time.Duration

	// PVDelete is how long PVs have to become deleted.
	PVDelete time.Duration

	// PVDeleteSlow is the same as PVDelete, but slower.
	PVDeleteSlow time.Duration

	// SnapshotCreate is how long for snapshot to create snapshotContent.
	SnapshotCreate time.Duration

	// SnapshotDelete is how long for snapshot to delete snapshotContent.
	SnapshotDelete time.Duration

	// SnapshotControllerMetrics is how long to wait for snapshot controller metrics.
	SnapshotControllerMetrics time.Duration
}

TimeoutContext contains timeout settings for several actions.

func NewTimeoutContextWithDefaults added in v1.21.0 

func NewTimeoutContextWithDefaults() *TimeoutContext

NewTimeoutContextWithDefaults returns a TimeoutContext with default values.

Source Files

View all Source files

Directories

Path Synopsis
Package config simplifies the declaration of configuration options.
 Package config simplifies the declaration of configuration options.
init
Package init sets debug.DumpAllNamespaceInfo as implementation in the framework and enables log size verification and resource gathering.
 Package init sets debug.DumpAllNamespaceInfo as implementation in the framework and enables log size verification and resource gathering.
internal
output
unittests/cleanup
init
Package init installs GrabBeforeEach and GrabAfterEach as callbacks for gathering data before and after a test.
 Package init installs GrabBeforeEach and GrabAfterEach as callbacks for gathering data before and after a test.
init
Package init registers node.AllNodesReady.
 Package init registers node.AllNodesReady.
pod
output
aws
azure
gce
kubemark
openstack
vsphere
Package testfiles provides a wrapper around various optional ways of retrieving additional files needed during a test run: - builtin bindata - filesystem access
 Package testfiles provides a wrapper around various optional ways of retrieving additional files needed during a test run: - builtin bindata - filesystem access

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.