test

package
Version: v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 29, 2019 License: Apache-2.0 Imports: 33 Imported by: 0

README

Test

This directory contains tests and testing docs for Knative Serving:

The conformance tests are a subset of the end to end test with more strict requirements around what can be tested.

If you want to add more tests, see adding_tests.md.

Presubmit tests

presubmit-tests.sh is the entry point for both the end-to-end tests and the conformance tests

This script, and consequently, the e2e and conformance tests will be run before every code submission. You can run these tests manually with:

test/presubmit-tests.sh

Note that to run presubmit-tests.sh or e2e-tests.sh scripts, you'll need kubernetes kubetest installed:

go get -u k8s.io/test-infra/kubetest

Running unit tests

To run all unit tests:

go test ./...

By default go test will not run the e2e tests, which need -tags=e2e to be enabled.

Running end to end tests

To run the e2e tests and the conformance tests, you need to have a running environment that meets the e2e test environment requirements, and you need to specify the build tag e2e.

go test -v -tags=e2e -count=1 ./test/conformance/...
go test -v -tags=e2e -count=1 ./test/e2e

Running performance tests

To run the performance tests, you need to have a running environment that meets the test environment requirements, and you need to specify the build tag performance.

go test -v -tags=performance -count=1 ./test/performance
Running a single test case

To run one e2e test case, e.g. TestAutoscaleUpDownUp, use the -run flag with go test:

go test -v -tags=e2e -count=1 ./test/e2e -run ^TestAutoscaleUpDownUp$
Running tests in short mode

Running tests in short mode excludes some large-scale E2E tests and saves time/resources required for running the test suite. To run the tests in short mode, use the -short flag with go test

go test -v -tags=e2e -count=1 -short ./test/e2e

To get a better idea where the flag is used, search for testing.Short() throughout the test source code.

Environment requirements

These tests require:

  1. A running Knative Serving cluster.

  2. The knative-testing resources:

    ko apply -f test/config
    
  3. A docker repo containing the test images

Common Flags

You can use test flags to control the environment your tests run against, i.e. override your environment variables:

go test -v -tags=e2e -count=1 ./test/conformance/... --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo
go test -v -tags=e2e -count=1 ./test/e2e --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo

Test images

Building the test images

Note: this is only required when you run conformance/e2e tests locally with go test commands.

The upload-test-images.sh script can be used to build and push the test images used by the conformance and e2e tests. The script expects your environment to be setup as described in DEVELOPMENT.md.

To run the script for all end to end test images:

./test/upload-test-images.sh

A docker tag may be passed as an optional parameter. This can be useful on Minikube in tandem with the --tag flag:

eval $(minikube docker-env)
./test/upload-test-images.sh any-old-tag
Adding new test images

New test images should be placed in ./test/test_images.

Flags

These flags are useful for running against an existing cluster, making use of your existing environment setup.

Tests importing github.com/knative/serving/test recognize these flags:

Overridding docker repo

The --dockerrepo argument lets you specify the docker repo from which images used by your tests should be pulled. This will default to the value of your KO_DOCKER_REPO environment variable if not specified.

go test -v -tags=e2e -count=1 ./test/conformance/... --dockerrepo gcr.myhappyproject
go test -v -tags=e2e -count=1 ./test/e2e --dockerrepo gcr.myhappyproject
Using a docker tag

The default docker tag used for the test images is latest, which can be problematic on Minikube. To avoid having to configure a remote container registry to support the Always pull policy for latest tags, you can have the tests use a specific tag:

go test -v -tags=e2e -count=1 ./test/conformance/... --tag any-old-tag
go test -v -tags=e2e -count=1 ./test/e2e --tag any-old-tag

Of course, this implies that you tagged the images when you uploaded them.

Using a custom ingress endpoint

Some environments (like minikube) do not support a Loadbalancer to make Knative services externally available. These environments usually rely on rewriting the Loadbalancer to a NodePort. The external address of such a NodePort is usually not easily obtained within the cluster automatically, but can be provided from the outside through the --ingressendpoint flag. For a minikube setup for example, you'd want to run tests against the default ingressgateway (port number 31380) running on the minikube node:

go test -v -tags=e2e -count=1 ./test/conformance/... --ingressendpoint "$(minikube ip):31380"
go test -v -tags=e2e -count=1 ./test/e2e --ingressendpoint "$(minikube ip):31380"
Using a resolvable domain

If you set up your cluster using the getting started docs, Routes created in the test will use the domain example.com, unless the route has label app=prod in which case they will use the domain prod-domain.com. Since these domains will not be resolvable to deployments in your test cluster, in order to make a request against the endpoint, the test use the IP assigned to the service istio-ingressgateway in the namespace istio-system and spoof the Host in the header.

If you have configured your cluster to use a resolvable domain, you can use the --resolvabledomain flag to indicate that the test should make requests directly against Route.Status.Domain and does not need to spoof the Host.

Documentation

Index

Constants

View Source
const (
	// Test image names
	Autoscale           = "autoscale"
	Failing             = "failing"
	HelloVolume         = "hellovolume"
	HelloWorld          = "helloworld"
	HTTPProxy           = "httpproxy"
	InvalidHelloWorld   = "invalidhelloworld" // Not a real image
	PizzaPlanet1        = "pizzaplanetv1"
	PizzaPlanet2        = "pizzaplanetv2"
	Protocols           = "protocols"
	Runtime             = "runtime"
	SingleThreadedImage = "singlethreaded"
	Timeout             = "timeout"
	WorkingDir          = "workingdir"

	// Constants for test image output.
	PizzaPlanetText1 = "What a spaceport!"
	PizzaPlanetText2 = "Re-energize yourself with a slice of pepperoni!"
	HelloWorldText   = "Hello World! How about some tasty noodles?"

	ConcurrentRequests = 50
	// We expect to see 100% of requests succeed for traffic sent directly to revisions.
	// This might be a bad assumption.
	MinDirectPercentage = 1
	// We expect to see at least 25% of either response since we're routing 50/50.
	// This might be a bad assumption.
	MinSplitPercentage = 0.25
)

Constants for test images located in test/test_images.

View Source
const (
	// ServingNamespace is the default namespace for serving e2e tests
	ServingNamespace = "serving-tests"

	// AlternativeServingNamespace is a different namepace to run cross-
	// namespace tests in.
	AlternativeServingNamespace = "serving-tests-alt"

	// E2EMetricExporter is the name for the metrics exporter logger
	E2EMetricExporter = "e2e-metrics"

	// ConformanceConfigMap is the name of the configmap to propagate env variables from
	ConformanceConfigMap = "conformance-test-configmap"
	// ConformanceSecret is the name of the secret to propagate env variables from
	ConformanceSecret = "conformance-test-secret"
	// EnvKey is the configmap/secret key which contains test value
	EnvKey = "testKey"
	// EnvValue is the configmap/secret test value to match env variable with
	EnvValue = "testValue"
)
View Source
const (
	// PollInterval is how frequently e2e tests will poll for updates.
	PollInterval = 1 * time.Second
	// PollTimeout is how long e2e tests will wait for resource updates when polling.
	PollTimeout = 10 * time.Minute

	// HelloVolumePath is the path to the test volume.
	HelloVolumePath = "/hello/world"
)

Variables

View Source
var AppendRandomString = helpers.AppendRandomString

AppendRandomString will generate a random string that begins with prefix. This is useful if you want to make sure that your tests can run at the same time against the same environment without conflicting. This method will seed rand with the current time when called for the first time.

View Source
var MakeK8sNamePrefix = helpers.MakeK8sNamePrefix

MakeK8sNamePrefix will convert each chunk of non-alphanumeric character into a single dash and also convert camelcase tokens into dash-delimited lowercase tokens.

View Source
var ObjectNameForTest = helpers.ObjectNameForTest

ObjectNameForTest generates a random object name based on the test name.

View Source
var ServingFlags = initializeServingFlags()

ServingFlags holds the flags or defaults for knative/serving settings in the user's environment.

Functions

func AssertProberDefault added in v0.4.0

func AssertProberDefault(t *testing.T, p Prober)

AssertProberDefault is a helper for stopping the Prober and checking its SLI against the default SLO, which requires perfect responses. This takes `testing.T` so that it may be used in `defer`.

func CheckSLO added in v0.4.0

func CheckSLO(slo float64, name string, p Prober) error

CheckSLO compares the SLI of the given prober against the SLO, erroring if too low.

func CleanupOnInterrupt

func CleanupOnInterrupt(cleanup func())

CleanupOnInterrupt will execute the function cleanup if an interrupt signal is caught

func CreateBytePatch added in v0.9.0

func CreateBytePatch(before, after interface{}) ([]byte, error)

TODO(dangerd): Remove this and use duck.CreateBytePatch after release-0.9 CreateBytePatch is a helper function that creates the same content as CreatePatch, but returns in []byte format instead of JSONPatch.

func ListenAndServeGracefully added in v0.3.0

func ListenAndServeGracefully(addr string, handler func(w http.ResponseWriter, r *http.Request))

ListenAndServeGracefully calls into ListenAndServeGracefullyWithPattern by passing handler to handle requests for "/"

func ListenAndServeGracefullyWithHandler added in v0.8.0

func ListenAndServeGracefullyWithHandler(addr string, handler http.Handler)

ListenAndServeGracefullyWithPattern creates an HTTP server, listens on the defined address and handles incoming requests with the given handler. It blocks until SIGTERM is received and the underlying server has shutdown gracefully.

func SubServiceNameForTest added in v0.4.0

func SubServiceNameForTest(t *testing.T, subsvc string) string

SubServiceNameForTest generates a random service name based on the test name and the given subservice name.

func TearDown added in v0.4.0

func TearDown(clients *Clients, names ResourceNames)

TearDown will delete created names using clients.

Types

type Clients

type Clients struct {
	KubeClient         *test.KubeClient
	ServingAlphaClient *ServingAlphaClients
	ServingBetaClient  *ServingBetaClients
	ServingClient      *ServingClients
	NetworkingClient   *NetworkingClients
	Dynamic            dynamic.Interface
	SharedClient       sharedclientset.Interface
}

Clients holds instances of interfaces for making requests to Knative Serving.

func NewClients

func NewClients(configPath string, clusterName string, namespace string) (*Clients, error)

NewClients instantiates and returns several clientsets required for making request to the Knative Serving cluster specified by the combination of clusterName and configPath. Clients can make requests within namespace.

func Setup added in v0.7.0

func Setup(t *testing.T) *Clients

Setup creates client to run Knative Service requests

type NetworkingClients added in v0.10.0

type NetworkingClients struct {
	ServerlessServices networkingv1alpha1.ServerlessServiceInterface
}

NetworkingClients holds instances of interfaces for making requests to Knative networking clients.

type Prober added in v0.4.0

type Prober interface {
	// SLI returns the "service level indicator" for the prober, which is the observed
	// success rate of the probes.  This will panic if the prober has not been stopped.
	SLI() (total int64, failures int64)

	// Stop terminates the prober, returning any observed errors.
	// Implementations may choose to put additional requirements on
	// the prober, which may cause this to block (e.g. a minimum number
	// of probes to achieve a population suitable for SLI measurement).
	Stop() error
}

Prober is the interface for a prober, which checks the result of the probes when stopped.

func RunRouteProber added in v0.2.0

func RunRouteProber(logf logging.FormatLogger, clients *Clients, url *url.URL) Prober

RunRouteProber starts a single Prober of the given domain.

type ProberManager added in v0.4.0

type ProberManager interface {
	// The ProberManager should expose a way to collectively reason about spawned
	// probes as a sort of aggregating Prober.
	Prober

	// Spawn creates a new Prober
	Spawn(url *url.URL) Prober

	// Foreach iterates over the probers spawned by this ProberManager.
	Foreach(func(url *url.URL, p Prober))
}

ProberManager is the interface for spawning probers, and checking their results.

func NewProberManager added in v0.4.0

func NewProberManager(logf logging.FormatLogger, clients *Clients, minProbes int64) ProberManager

NewProberManager creates a new manager for probes.

type ResourceNames

type ResourceNames struct {
	Config        string
	Route         string
	Revision      string
	Service       string
	TrafficTarget string
	URL           *url.URL
	Image         string
}

ResourceNames holds names of various resources.

type ServingAlphaClients added in v0.7.0

ServingAlphaClients holds instances of interfaces for making requests to knative serving clients

func (*ServingAlphaClients) Delete added in v0.7.1

func (clients *ServingAlphaClients) Delete(routes []string, configs []string, services []string) error

Delete will delete all Routes and Configs with the names routes and configs, if clients has been successfully initialized.

type ServingBetaClients added in v0.7.0

ServingBetaClients holds instances of interfaces for making requests to knative serving clients

type ServingClients added in v0.2.0

type ServingClients struct {
	Routes    servingv1.RouteInterface
	Configs   servingv1.ConfigurationInterface
	Revisions servingv1.RevisionInterface
	Services  servingv1.ServiceInterface
}

ServingClients holds instances of interfaces for making requests to knative serving clients

type ServingEnvironmentFlags added in v0.2.0

type ServingEnvironmentFlags struct {
	ResolvableDomain bool // Resolve Route controller's `domainSuffix`
}

ServingEnvironmentFlags holds the e2e flags needed only by the serving repo.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL