test

README

Tests

To run tests:

# Unit tests
go test ./...

# Integration tests (against your current kube cluster)
go test -v -count=1 -tags=e2e ./test

Unit tests

Unit tests live side by side with the code they are testing and can be run with:

go test ./...

By default go test will not run the integration tests, which need -tags=e2e to be enabled.

Unit testing Controllers

Kubernetes client-go provides a number of fake clients and objects for unit testing. The ones we will be using are:

1. Fake kubernetes client: Provides a fake REST interface to interact with Kubernetes API
2. Fake pipeline client : Provides a fake REST PipelineClient Interface to interact with Pipeline CRDs.

You can create a fake PipelineClient for the Controller under test like this:

import (
    fakepipelineclientset "github.com/knative/build-pipeline/pkg/client/clientset/versioned/fake
)
pipelineClient := fakepipelineclientset.NewSimpleClientset()

This pipelineClient is initialized with no runtime objects. You can also initialie the client with kubernetes objects and can interact with them using the pipelineClient.Pipeline()

 import (
     v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )

 obj := &v1alpha1.PipelineRun{
  ObjectMeta: metav1.ObjectMeta{
    Name:      "name",
    Namespace: "namespace",
  },
  Spec: v1alpha1.PipelineRunSpec{
    PipelineRef: v1alpha1.PipelineRef{
      Name:       "test-pipeline",
      APIVersion: "a1",
    },
}}
pipelineClient := fakepipelineclientset.NewSimpleClientset(obj)
objs := pipelineClient.Pipeline().PipelineRuns("namespace").List(v1.ListOptions{})
//You can verify if List was called in yout test like this
action :=  pipelineClient.Actions()[0]
if action.GetVerb() != "list" {
    t.Errorf("expected list to be called, found %s", action.GetVerb())
}

To test the Controller for crd objects, we need to adding test crd objects to the informers so that the listers can access these.

To add test PipelineRun objects to the listers, you can

pipelineClient := fakepipelineclientset.NewSimpleClientset()
sharedInfomer := informers.NewSharedInformerFactory(pipelineClient, 0)
pipelineRunsInformer := sharedInfomer.Pipeline().V1alpha1().PipelineRuns()

obj := &v1alpha1.PipelineRun{
  ObjectMeta: metav1.ObjectMeta{
    Name:      "name",
    Namespace: "namespace",
  },
  Spec: v1alpha1.PipelineRunSpec{
    PipelineRef: v1alpha1.PipelineRef{
      Name:       "test-pipeline",
      APIVersion: "a1",
    },
}}
pipelineRunsInformer.Informer().GetIndexer().Add(obj)

Integration tests

Setup

As well as requiring the environment variable KO_DOCKER_REPO variable, you may also require authentication inside the Build to run the Kaniko e2e test. If so, setting KANIKO_SECRET_CONFIG_FILE to be the path to a GCP service account JSON key which has permissions to push to the registry specified in KO_DOCKER_REPO will enable Kaniko to use those credentials when pushing.

To quickly create a service account usable with the e2e tests:

PROJECT_ID=your-gcp-project
ACCOUNT_NAME=service-account-name
gcloud config set project $PROJECT_ID

# create the service account
gcloud iam service-accounts create $ACCOUNT_NAME --display-name $ACCOUNT_NAME
EMAIL=$(gcloud iam service-accounts list | grep $ACCOUNT_NAME | awk '{print $2}')

# add the storage.admin policy to the account so it can push containers
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:$EMAIL --role roles/storage.admin

# create the JSON key
gcloud iam service-accounts keys create config.json --iam-account $EMAIL

export KANIKO_SECRET_CONFIG_FILE="$PWD/config.json"

Running

Integration tests live in this directory. To run these tests, you must provide go with -tags=e2e. By default the tests run agains your current kubeconfig context, but you can change that and other settings with the flags:

go test -v -count=1 -tags=e2e ./test
go test -v -tags=e2e -count=1 ./test --kubeconfig ~/special/kubeconfig --cluster myspecialcluster

You can also use all of flags defined in knative/pkg/test.

Flags

• By default the e2e tests against the current cluster in ~/.kube/config using the environment specified in your environment variables.
• Since these tests are fairly slow, running them with logging enabled is recommended (-v).
• Using --logverbose to see the verbose log output from test as well as from k8s libraries.
• Using -count=1 is the idiomatic way to disable test caching

You can use test flags to control the environment your tests run against, i.e. override your environment variables:

go test -v -tags=e2e -count=1 ./test --kubeconfig ~/special/kubeconfig --cluster myspecialcluster

Tests importing github.com/knative/build-pipline/test recognize these all flags added by knative/pkg/test.

Note the environment variable K8S_CLUSTER_OVERRIDE, while used by knative/serving and not by this project, will override the cluster used by the integration tests since they use the same libs to get these flags.

One test case

To run one e2e test case, e.g. TestTaskRun, use the -run flag with go test:

go test -v -tags=e2e -count=1 ./test -run ^TestTaskRun$

Adding integration tests

In the test dir you will find several libraries in the test package you can use in your tests.

This library exists partially in this directory and partially in knative/pkg/test.

The libs in this dir can:

• init.go initializes anything needed globally be the tests
• Get access to client objects
• Generate random names
• Poll Pipeline resources

All integration tests must be marked with the e2e build constraint so that go test ./... can be used to run only the unit tests, i.e.:

// +build e2e

Get access to client objects

To initialize client objects use the command line flags which describe the environment:

func setup(t *testing.T) *test.Clients {
    clients, err := test.NewClients(kubeconfig, cluster, namespaceName)
    if err != nil {
        t.Fatalf("Couldn't initialize clients: %v", err)
    }
    return clients
}

The Clients struct contains initialized clients for accessing:

• Kubernetes objects
• Pipelines
• TODO: incrementally add clients for other types

For example, to create a Pipeline:

_, err = clients.PipelineClient.Pipelines.Create(test.Route(namespaceName, pipelineName))

And you can use the client to clean up resources created by your test (e.g. in your test cleanup):

func tearDown(clients *test.Clients) {
    if clients != nil {
        clients.Delete([]string{routeName}, []string{configName})
    }
}

See clients.go.

Generate random names

You can use the function AppendRandomString to create random names for crds or anything else, so that your tests can use unique names each time they run.

namespace := test.AppendRandomString('arendelle')

See randstring.go.

Poll Pipeline resources

After creating Pipeline resources or making changes to them, you will need to wait for the system to realize those changes. You can use polling methods to check the resources reach the desired state.

The WaitFor* functions use the kubernetes wait package. To poll they use PollImmediate and the return values of the function you provide behave the same as ConditionFunc: a bool to indicate if the function should stop or continue polling, and an error to indicate if there has been an error.

For example, you can poll a TaskRun object to wait for it to have a Status.Condition:

err = WaitForTaskRunState(c, hwTaskRunName, func(tr *v1alpha1.TaskRun) (bool, error) {
    if len(tr.Status.Conditions) > 0 {
        return true, nil
    }
    return false, nil
}, "TaskRunHasCondition")

Metrics will be emitted for these Wait method tracking how long test poll for.

Presubmit tests

presubmit-tests.sh is the entry point for all tests run on presubmit by Prow.

You can run this locally with:

test/presubmit-tests.sh
test/presubmit-tests.sh --build-tests
test/presubmit-tests.sh --unit-tests

Prow is configured in the knative config.yaml in knative/test-infra via the sections for knative/build-pipeline.

Running presubmit integration tests

The presubmit integration tests entrypoint will run:

• The integration tests
• A sanity check deployment of our example CRDs

When run using Prow, integration tests will try to get a new cluster using boskos and these hardcoded GKE projects, which only the knative/test-infra OWNERS have access to.

If you would like to run the integration tests against your cluster, you can use the K8S_CLUSTER_OVERRIDE environment variable to force the scripts to use your own cluster, provide KO_DOCKER_REPO (as specified in the DEVELOPMENT.md), use e2e-tests.sh directly and provide the --run-tests argument:

export K8S_CLUSTER_OVERRIDE=my_k8s_cluster # corresponds to a `context` in your kubeconfig
export KO_DOCKER_REPO=gcr.io/my_docker_repo # required for deployments using `ko`
test/e2e-tests.sh --run-tests

Or you can set $PROJECT_ID to a GCP project and rely on kubetest to setup a cluster for you:

export K8S_CLUSTER_OVERRIDE=
export PROJECT_ID=my_gcp_project
export KO_DOCKER_REPO=gcr.io/my_docker_repo # required for deployments using `ko`
test/presubmit-tests.sh --integration-tests

Documentation

Index

• func AppendRandomString(prefix string) string
• func CollectBuildLogs(c *clients, buildName, namespace string, logger *logging.BaseLogger)
• func GetLogMessages(logs *observer.ObservedLogs) []string
• func WaitForPipelineRunState(c *clients, name string, inState func(r *v1alpha1.PipelineRun) (bool, error), ...) error
• func WaitForPodState(c *clients, name string, namespace string, ...) error
• func WaitForServiceExternalIPState(c *clients, namespace, name string, ...) error
• func WaitForTaskRunState(c *clients, name string, inState func(r *v1alpha1.TaskRun) (bool, error), ...) error
• type Clients
  • func GetPipelineRunController(d Data) (*controller.Impl, *observer.ObservedLogs, Clients)
  • func GetTaskRunController(d Data) (*controller.Impl, *observer.ObservedLogs, Clients)
• type Data
• type Informers

Functions

func AppendRandomString

func AppendRandomString(prefix string) string

AppendRandomString will generate a random string that begins with prefix. This is useful if you want to make sure that your tests can run at the same time against the same environment without conflicting. This method will seed rand with the current time when called for the first time.

func CollectBuildLogs

func CollectBuildLogs(c *clients, buildName, namespace string, logger *logging.BaseLogger)

CollectBuildLogs will get the build logs for a task run

func GetLogMessages

func GetLogMessages(logs *observer.ObservedLogs) []string

GetLogMessages returns a list of all string logs in logs.

func WaitForPipelineRunState

func WaitForPipelineRunState(c *clients, name string, inState func(r *v1alpha1.PipelineRun) (bool, error), desc string) error

WaitForPipelineRunState polls the status of the PipelineRun called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForPodState

func WaitForPodState(c *clients, name string, namespace string, inState func(r *corev1.Pod) (bool, error), desc string) error

WaitForPodState polls the status of the Pod called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForServiceExternalIPState

func WaitForServiceExternalIPState(c *clients, namespace, name string, inState func(s *corev1.Service) (bool, error), desc string) error

WaitForServiceExternalIPState polls the status of the a k8s Service called name from client every interval until an external ip is assigned indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

func WaitForTaskRunState

func WaitForTaskRunState(c *clients, name string, inState func(r *v1alpha1.TaskRun) (bool, error), desc string) error

WaitForTaskRunState polls the status of the TaskRun called name from client every interval until inState returns `true` indicating it is done, returns an error or timeout. desc will be used to name the metric that is emitted to track how long it took for name to get into the state checked by inState.

Types

type Clients

type Clients struct {
	Pipeline *fakepipelineclientset.Clientset
	Build    *fakebuildclientset.Clientset
	Kube     *fakekubeclientset.Clientset
}

Clients holds references to clients which are useful for reconciler tests.

func GetPipelineRunController

func GetPipelineRunController(d Data) (*controller.Impl, *observer.ObservedLogs, Clients)

GetPipelineRunController returns an instance of the PipelineRun controller/reconciler that has been seeded with d, where d represents the state of the system (existing resources) needed for the test.

func GetTaskRunController

func GetTaskRunController(d Data) (*controller.Impl, *observer.ObservedLogs, Clients)

GetTaskRunController returns an instance of the TaskRun controller/reconciler that has been seeded with d, where d represents the state of the system (existing resources) needed for the test.

type Data

type Data struct {
	PipelineRuns      []*v1alpha1.PipelineRun
	Pipelines         []*v1alpha1.Pipeline
	TaskRuns          []*v1alpha1.TaskRun
	Tasks             []*v1alpha1.Task
	PipelineParams    []*v1alpha1.PipelineParams
	PipelineResources []*v1alpha1.PipelineResource
	Builds            []*buildv1alpha1.Build
}

Data represents the desired state of the system (i.e. existing resources) to seed controllers with.

type Informers

type Informers struct {
	PipelineRun      informersv1alpha1.PipelineRunInformer
	Pipeline         informersv1alpha1.PipelineInformer
	TaskRun          informersv1alpha1.TaskRunInformer
	Task             informersv1alpha1.TaskInformer
	PipelineParams   informersv1alpha1.PipelineParamsInformer
	PipelineResource informersv1alpha1.PipelineResourceInformer
	Build            buildinformersv1alpha1.BuildInformer
}

Informers holds references to informers which are useful for reconciler tests.