kcptc

README

kcp-testcontainers

testcontainers-go module running a sharded kcp instance for tests.

A kcp-testcontainers instance is always deployed under the constraints the kcp project defines as the minimum viable architecture:

• At least one front-proxy
• At least two shards
• One cache-server

With the front-proxy and the shards' virtual workspaces being accessible to clients.

This testcontainer module is ideal to write integration or e2e tests for operators targeting kcp directly or a system utilizing kcp.

Usage

Import this module:

import kcptc "github.com/kcp-dev/kcp-testcontainers"

Create an instance and create workspaces:

instance, err := kcptc.Run(ctx, kcptc.Options{})

// create workspaces (parents included)
path, err := instance.CreateWorkspace(ctx, "root:my:workspace")
// create workspace with generated name (parents included)
path, err = instance.CreateWorkspace(ctx, "root:test-")
// kcp's workspace options apply to the last workspace in the path; wait until
// the front-proxy serves every workspace in the path, not only until Ready
path, err = instance.CreateWorkspace(ctx, "root:org:team",
	kcptc.With(kcptesting.WithType(logicalcluster.NewPath("root"), "organization")),
	kcptc.WaitForReachable(),
)

// rest.Config for a workspace
cfg := instance.RESTConfig("root:my:workspace")
// controller-runtime client for a workspace
cl, err := instance.Client("root:my:workspace", client.Options{})
// rest.Config not scoped to a workspace, e.g. for a virtual workspace URL
// (every serving certificate is valid for the Docker host the ports are
// published on, so no TLS overrides are needed)
cfg = instance.BaseConfig()

Sharing across processes

When running go tests each tested package is run as its own process.

kcptc.Shared implements sharing instances across processes by giving an instance a name and using testcontainers SessionID:

// a shared instance that is reused across processes
shared, err := kcptc.Shared(ctx, "shared-instance", kcptc.Options{})
sharedWS, err := shared.CreateWorkspace(ctx, "root:my:workspace")
sharedCL, err := shared.Client("root:my:workspace", client.Options{})

To prevent drift for the configurations a good option is to have a framework package with functions that call Shared:

func Kcp(ctx context.Context) (*kcptc.Instance, error) {
	return kcptc.Shared(ctx, "default", kcptc.Options{})
}

func KcpWithMigrations(ctx context.Context) (*kcptc.Instance, error) {
	return kcptc.Shared(ctx, "second", kcptc.Options{
		FeatureGates: []string{"LogicalClusterMigration"},
	})
}

Documentation

Index

• Constants
• type Instance
  • func Run(ctx context.Context, opts Options) (*Instance, error)
  • func Shared(ctx context.Context, name string, opts Options) (*Instance, error)
  • func (in *Instance) BaseConfig() *rest.Config
  • func (in *Instance) Client(path string, opts client.Options) (client.Client, error)
  • func (in *Instance) Config() *rest.Config
  • func (in *Instance) CreateWorkspace(ctx context.Context, path string, opts ...WorkspaceOption) (logicalcluster.Path, error)
  • func (in *Instance) ExternalURL(ctx context.Context, inNetwork string) (string, error)
  • func (in *Instance) RESTConfig(path string) *rest.Config
  • func (in *Instance) ShardConfig(ctx context.Context, name string) (*rest.Config, error)
  • func (in *Instance) ShardNames() []string
• type Options
• type WorkspaceOption
  • func WaitForReachable() WorkspaceOption
  • func With(opts ...kcptesting.UnprivilegedWorkspaceOption) WorkspaceOption

Constants

const (
	// DefaultVersion is the default kcp version deployed by kcp-testcontainers.
	DefaultVersion = "v0.33.6"
	// DefaultImage is the default image deployed by kcp-testcontainers.
	DefaultImage = "ghcr.io/kcp-dev/kcp"
	// DefaultTag is the default image tag deployed by kcp-testcontainers.
	DefaultTag = DefaultImage + ":" + DefaultVersion

	// DefaultStartupTimeout bounds waiting for the whole topology to become ready.
	DefaultStartupTimeout = 2 * time.Minute
)

const (
	AliasCache      = "cache"
	AliasFrontProxy = "front-proxy"
	ShardA          = "root"
	ShardB          = "shard-b"
)

Network aliases of the components.

Types

type Instance

type Instance struct {
	Cache      testcontainers.Container
	FrontProxy testcontainers.Container
	ShardA     testcontainers.Container
	ShardB     testcontainers.Container
	// contains filtered or unexported fields
}

Instance is a running sharded kcp: cache-server, two shards and a front-proxy.

func Run

func Run(ctx context.Context, opts Options) (*Instance, error)

Run starts a private kcp instance.

func Shared

func Shared(ctx context.Context, name string, opts Options) (*Instance, error)

Shared starts a shared kcp instance. Within a go test run instances are shared across processes via the name.

If the passed options differ then the first process to create the instance wins. There is no drift check.

func (*Instance) BaseConfig

func (in *Instance) BaseConfig() *rest.Config

BaseConfig returns an admin rest.Config through the front-proxy that is not scoped to a workspace, for clients that append /clusters/<path> themselves or that are pointed at a URL kcp advertises, such as a virtual workspace URL from an APIExportEndpointSlice.

func (*Instance) Client

func (in *Instance) Client(path string, opts client.Options) (client.Client, error)

Client returns a controller-runtime client for the workspace at path. A nil opts.Scheme defaults to the Kubernetes types plus kcp's tenancy, core and apis groups.

func (*Instance) Config

func (in *Instance) Config() *rest.Config

Config returns an admin rest.Config for the root workspace through the front-proxy.

func (*Instance) CreateWorkspace

func (in *Instance) CreateWorkspace(ctx context.Context, path string, opts ...WorkspaceOption) (logicalcluster.Path, error)

CreateWorkspace creates the workspace at path, including missing parents. A path segment ending in "-" is used as a GenerateName prefix. The full path with generated names is returned.

func (*Instance) ExternalURL

func (in *Instance) ExternalURL(ctx context.Context, inNetwork string) (string, error)

ExternalURL rewrites an in-network URL, e.g. a virtual workspace URL https://shard-b:6444/services/..., to the host-mapped endpoint.

func (*Instance) RESTConfig

func (in *Instance) RESTConfig(path string) *rest.Config

RESTConfig returns an admin rest.Config for the workspace at path through the front-proxy.

func (*Instance) ShardConfig

func (in *Instance) ShardConfig(ctx context.Context, name string) (*rest.Config, error)

ShardConfig returns a system:masters rest.Config talking to shard name directly, bypassing the front-proxy.

func (*Instance) ShardNames

func (in *Instance) ShardNames() []string

ShardNames returns the names of the shards.

type Options

type Options struct {
	// Image is the kcp image including tag.
	// Defaults to [DefaultTag].
	Image string

	// StartupTimeout bounds waiting for all components to become ready.
	// Defaults to [DefaultStartupTimeout].
	StartupTimeout time.Duration

	// FeatureGates that should be enabled in the instance.
	// Format is <gate>=<bool>, e.g. `CacheAPIs=true`.
	FeatureGates []string

	// FrontProxy is [testcontainers.ContainerCustomizer] applied to the front-proxy container after settings derived from the other options.
	FrontProxy []testcontainers.ContainerCustomizer
	// ShardA is [testcontainers.ContainerCustomizer] applied to the first shard container after settings derived from the other options.
	ShardA []testcontainers.ContainerCustomizer
	// ShardB is [testcontainers.ContainerCustomizer] applied to the second shard container after settings derived from the other options.
	ShardB []testcontainers.ContainerCustomizer
	// Cache is [testcontainers.ContainerCustomizer] applied to the cache-server container after settings derived from the other options.
	Cache []testcontainers.ContainerCustomizer
}

Options configures the kcp topology. Customizers apply to the container request of the named component.

type WorkspaceOption

type WorkspaceOption func(*workspaceRequest)

WorkspaceOption configures CreateWorkspace.

func WaitForReachable

func WaitForReachable() WorkspaceOption

WaitForReachable makes CreateWorkspace wait until every workspace in the path is served through the front-proxy, not only Ready. Requests to a Ready workspace fail until the front-proxy has picked up its logical cluster.

func With

func With(opts ...kcptesting.UnprivilegedWorkspaceOption) WorkspaceOption

With applies kcp's workspace options (type, name, shard, ...) to the last workspace in the path.