kcp

README

kcp-testcontainer

testcontainers-go module running a kcp server for tests.

Two modes are supported. Single and Sharded.

Single is a single kcp shard and a great way to get cheap control planes.

Sharded is a full setup with a front-proxy, two shards and a cache-server and a great way to test kcp-native operators.

Usage

Import this module:

import kcptc "github.com/ntnn/kcp-testcontainer"

Create an instance, either Single, Sharded, SingleOnce or ShardedOnce and create workspaces:

single, err := kcptc.Single(ctx, kcptc.DefaultImage)

// create workspaces (parents included)
path, err := single.CreateWorkspace(ctx, "root:my:workspace")
// create workspace with generated name (parents included)
path, err = single.CreateWorkspace(ctx, "root:test-")

// rest.Config for a workspace
cfg, err := single.RESTConfig(ctx, "root:my:workspace")
// controller-runtime client for a workspace
cl, err := single.Client(ctx, "root:my:workspace", client.Options{})

Why not envtest?

Starting multiple envtest environments is also an option, but each is its own process and consumer memory and cpu. This might not be a big problem on large machines but on smaller laptops resource consumption is a concern.

E.g. ten control planes (envtest environments or kcp workspaces) cost ~380 MiB in memory in kcp and 3.1GiB with envtest.

Here are benchmarks comparing envtest with kcp-testcontainer deploying ten control planes over five passes:

> ./benchmarks/run.sh -count 10 -runs 5
env          min ms   avg ms   max ms  avg mem MiB
envtest       20310    21531    23568       3101.6
kcp           16295    18042    19105        379.8

On top of that kcp's workspaces are garbage collected, envtest has disabled the GC.

However this is only useful if needing multiple control planes - if only a few control planes are needed and memory isn't a concern it is still faster to use envtest directly.

> ./benchmarks/run.sh -count 1 -runs 5
env          min ms   avg ms   max ms  avg mem MiB
envtest        1987     2249     2585        341.1
kcp           15897    16254    16802        256.7

kcp takes longer because it does the heavy lifting for multiple control planes during startup, but it pays off in memory consumption and when needing multiple control planes.

Documentation

Overview

Package kcp provides a testcontainers-go module running a kcp server.

Index

• Constants
• type ShardedInstance
  • func Sharded(ctx context.Context, img string, opts ...ShardedOption) (*ShardedInstance, error)
  • func ShardedOnce(ctx context.Context, img string, opts ...ShardedOption) (*ShardedInstance, error)
  • func (in *ShardedInstance) Client(ctx context.Context, path string, opts client.Options) (client.Client, error)
  • func (in *ShardedInstance) CreateWorkspace(ctx context.Context, path string, opts ...WorkspaceOption) (string, error)
  • func (in *ShardedInstance) ExternalURL(ctx context.Context, inNetwork string) (string, error)
  • func (in *ShardedInstance) Kubeconfig(_ context.Context) ([]byte, error)
  • func (in *ShardedInstance) RESTConfig(ctx context.Context, path string) (*rest.Config, error)
  • func (in *ShardedInstance) Terminate(ctx context.Context, opts ...testcontainers.TerminateOption) error
• type ShardedOption
  • func WithCacheCustomizers(opts ...testcontainers.ContainerCustomizer) ShardedOption
  • func WithFrontProxyCustomizers(opts ...testcontainers.ContainerCustomizer) ShardedOption
  • func WithShardCustomizers(shard string, opts ...testcontainers.ContainerCustomizer) ShardedOption
• type SingleInstance
  • func Single(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*SingleInstance, error)
  • func SingleOnce(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*SingleInstance, error)
  • func (in *SingleInstance) Client(ctx context.Context, path string, opts client.Options) (client.Client, error)
  • func (in *SingleInstance) CreateWorkspace(ctx context.Context, path string, opts ...WorkspaceOption) (string, error)
  • func (in *SingleInstance) Kubeconfig(_ context.Context) ([]byte, error)
  • func (in *SingleInstance) RESTConfig(ctx context.Context, path string) (*rest.Config, error)
• type WorkspaceOption
  • func WithLocation(w tenancyv1alpha1.WorkspaceLocation) WorkspaceOption
  • func WithShard(name string) WorkspaceOption

Constants

const (
	AliasCache      = "cache"
	AliasShard0     = "root" // TODO(ntnn): rename to kcp-0 once root shard is no longer a thing
	AliasShard1     = "kcp-1"
	AliasFrontProxy = "front-proxy"
)

Network aliases of the sharded topology components.

const DefaultImage = "ghcr.io/kcp-dev/kcp:v0.32.3"

DefaultImage is the image kcp-testcontainer uses when an empty string is passed.

Types

type ShardedInstance

type ShardedInstance struct {
	FrontProxy testcontainers.Container
	Shard0     testcontainers.Container
	Shard1     testcontainers.Container
	Cache      testcontainers.Container
	// contains filtered or unexported fields
}

ShardedInstance is a full kcp instance with front-proxy, two shards and a cache-server.

func Sharded

func Sharded(ctx context.Context, img string, opts ...ShardedOption) (*ShardedInstance, error)

Sharded starts a sharded kcp instance: cache-server, root shard, shard-1 and front-proxy, all from img on a dedicated network.

func ShardedOnce

func ShardedOnce(ctx context.Context, img string, opts ...ShardedOption) (*ShardedInstance, error)

ShardedOnce is like Sharded but always returns the same ShardedInstance.

This is once per process. Deploying across multiple processes is not recommended since the kcp instance relies on pki, which for kcp-testcontainer is held in memory.

For a persistent sharded test setup use envtest.Sharded from kcp's multicluster-provider: https://pkg.go.dev/github.com/kcp-dev/multicluster-provider@main/envtest#Sharded

func (*ShardedInstance) Client

func (in *ShardedInstance) Client(ctx context.Context, path string, opts client.Options) (client.Client, error)

Client returns a controller-runtime client for the workspace at path.

func (*ShardedInstance) CreateWorkspace

func (in *ShardedInstance) CreateWorkspace(ctx context.Context, path string, opts ...WorkspaceOption) (string, error)

CreateWorkspace creates the workspace at path, including missing parents. A path segment ending in "-" is used as a GenerateName prefix. The full path with generated names is returned. Options apply only to the last workspace in the path.

func (*ShardedInstance) ExternalURL

func (in *ShardedInstance) ExternalURL(ctx context.Context, inNetwork string) (string, error)

ExternalURL rewrites an in-network URL (e.g. a virtual workspace URL https://kcp-1:6444/services/...) to the host-mapped endpoint.

func (*ShardedInstance) Kubeconfig

func (in *ShardedInstance) Kubeconfig(_ context.Context) ([]byte, error)

Kubeconfig returns the admin kubeconfig. The current context is the root workspace.

func (*ShardedInstance) RESTConfig

func (in *ShardedInstance) RESTConfig(ctx context.Context, path string) (*rest.Config, error)

RESTConfig returns an admin rest.Config for the workspace at path.

func (*ShardedInstance) Terminate

func (in *ShardedInstance) Terminate(ctx context.Context, opts ...testcontainers.TerminateOption) error

Terminate tears down all containers and the network.

type ShardedOption

type ShardedOption func(*shardedConfig) error

ShardedOption configures the kcp components.

func WithCacheCustomizers

func WithCacheCustomizers(opts ...testcontainers.ContainerCustomizer) ShardedOption

WithCacheCustomizers applies customizers to the cache-server container.

func WithFrontProxyCustomizers

func WithFrontProxyCustomizers(opts ...testcontainers.ContainerCustomizer) ShardedOption

WithFrontProxyCustomizers applies customizers to the front-proxy container request.

func WithShardCustomizers

func WithShardCustomizers(shard string, opts ...testcontainers.ContainerCustomizer) ShardedOption

WithShardCustomizers applies customizers to the shard container request.

type SingleInstance

type SingleInstance struct {
	testcontainers.Container
	// contains filtered or unexported fields
}

SingleInstance is a running a single kcp root shard.

func Single

func Single(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*SingleInstance, error)

Single starts a single root-shard kcp container.

func SingleOnce

func SingleOnce(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*SingleInstance, error)

SingleOnce is like Single but always returns the same SingleInstance.

This is once per process. To deploy a single instance overall use testcontainers.WithReuseByName.

func (*SingleInstance) Client

func (in *SingleInstance) Client(ctx context.Context, path string, opts client.Options) (client.Client, error)

Client returns a controller-runtime client for the workspace at path.

func (*SingleInstance) CreateWorkspace

func (in *SingleInstance) CreateWorkspace(ctx context.Context, path string, opts ...WorkspaceOption) (string, error)

CreateWorkspace creates the workspace at path, including missing parents. A path segment ending in "-" is used as a GenerateName prefix. The full path with generated names is returned. Options apply only to the last workspace in the path.

func (*SingleInstance) Kubeconfig

func (in *SingleInstance) Kubeconfig(_ context.Context) ([]byte, error)

Kubeconfig returns the admin kubeconfig. The current context is the root workspace.

func (*SingleInstance) RESTConfig

func (in *SingleInstance) RESTConfig(ctx context.Context, path string) (*rest.Config, error)

RESTConfig returns an admin rest.Config for the workspace at path.

type WorkspaceOption

type WorkspaceOption func(ws *tenancyv1alpha1.Workspace)

WorkspaceOption mutates a workspace before creation.

func WithLocation

func WithLocation(w tenancyv1alpha1.WorkspaceLocation) WorkspaceOption

WithLocation sets the location of the workspace.

func WithShard

func WithShard(name string) WorkspaceOption

WithShard schedules the workspace on the given shard.