eksconfig

Documentation

Overview

Package eksconfig defines EKS test configuration.

Index

• type ClusterState
• type Config
  • func Load(p string) (cfg *Config, err error)
  • func NewDefault() *Config
  • func (cfg *Config) KubectlCommands() (s string)
  • func (cfg *Config) SSHCommands() (s string)
  • func (cfg *Config) SetClusterUpTook(d time.Duration)
  • func (cfg *Config) Sync() (err error)
  • func (cfg *Config) UpdateFromEnvs() error
  • func (cfg *Config) ValidateAndSetDefaults() error

Types

type ClusterState

type ClusterState struct {
	// ClusterARN is the cluster ARN.
	ClusterARN string `json:"cluster-arn,omitempty"`

	// Status is the cluster status from EKS API.
	// It's either CREATING, ACTIVE, DELETING, FAILED, "DELETE_COMPLETE".
	// Reference: https://docs.aws.amazon.com/eks/latest/APIReference/API_Cluster.html#AmazonEKS-Type-Cluster-status.
	Status string `json:"status,omitempty"` // read-only to user

	StatusRoleCreated       bool `json:"status-role-created"`        // read-only to user
	StatusPolicyAttached    bool `json:"status-policy-attached"`     // read-only to user
	StatusVPCCreated        bool `json:"status-vpc-created"`         // read-only to user
	StatusClusterCreated    bool `json:"status-cluster-created"`     // read-only to user
	StatusKeyPairCreated    bool `json:"status-key-pair-created"`    // read-only to user
	StatusWorkerNodeCreated bool `json:"status-worker-node-created"` // read-only to user

	// Created is the timestamp of cluster creation.
	Created time.Time `json:"created,omitempty"` // read-only to user

	// UpTook is total duration that took to set up cluster up and running.
	// Does not include sub-project resource creation (e.g. ALB Ingress Controller).
	UpTook string `json:"up-took,omitempty"` // read-only to user

	// ServiceRoleWithPolicyName is the name of the EKS cluster service role with policy.
	// Prefixed with cluster name and suffixed with 'SERVICE-ROLE'.
	ServiceRoleWithPolicyName string `json:"service-role-with-policy-name,omitempty"`
	// ServiceRolePolicies is the list of policy ARNs to create cluster service role with.
	ServiceRolePolicies []string `json:"service-role-policies,omitempty"`
	// ServiceRoleWithPolicyARN is the ARN of the created cluster service role.
	ServiceRoleWithPolicyARN string `json:"service-role-with-policy-arn,omitempty"`

	// Endpoint is the cluster endpoint of the EKS cluster, required for KUBECONFIG write.
	Endpoint string `json:"endpoint,omitempty"`
	// CA is the EKS cluster CA, required for KUBECONFIG write.
	CA string `json:"ca,omitempty"`
	// CADecoded is the decoded EKS cluster CA, required for k8s.io/client-go.
	CADecoded string `json:"ca-decoded,omitempty"`

	// WorkerNodeGroupStatus is the status Kubernetes worker node group.
	// "READY" when they successfully join the EKS cluster as worker nodes.
	WorkerNodeGroupStatus string `json:"worker-node-group-status,omitempty"`
	// WorkerNodes is a list of worker nodes.
	WorkerNodes map[string]ec2config.Instance `json:"worker-nodes,omitempty"`

	// WorkerNodeLogs is a list of worker node log file paths, fetched via SSH.
	WorkerNodeLogs map[string]string `json:"worker-node-logs,omitempty"`

	// CFStackWorkerNodeGroupName is the name of cloudformation stack for worker node group.
	CFStackWorkerNodeGroupName string `json:"cf-stack-worker-node-group-name,omitempty"`
	// CFStackWorkerNodeGroupStatus is the last cloudformation status of node group stack.
	CFStackWorkerNodeGroupStatus string `json:"cf-stack-worker-node-group-status,omitempty"`
	// CFStackWorkerNodeGroupKeyPairName is required for node group creation.
	CFStackWorkerNodeGroupKeyPairName string `json:"cf-stack-worker-node-group-key-pair-name,omitempty"`
	// CFStackWorkerNodeGroupSecurityGroupID is the security group ID
	// that worker node cloudformation stack created.
	CFStackWorkerNodeGroupSecurityGroupID string `json:"cf-stack-worker-node-group-security-group-id,omitempty"`
	// CFStackWorkerNodeGroupAutoScalingGroupName is the name of worker node auto scaling group.
	CFStackWorkerNodeGroupAutoScalingGroupName string `json:"cf-stack-worker-node-group-auto-scaling-group-name,omitempty"`

	// CFStackWorkerNodeGroupWorkerNodeInstanceRoleARN is the ARN of NodeInstance role of node group.
	// Required to enable worker nodes to join cluster.
	// Update this after creating node group stack
	CFStackWorkerNodeGroupWorkerNodeInstanceRoleARN string `json:"cf-stack-worker-node-group-worker-node-instance-role-arn,omitempty"`
	// contains filtered or unexported fields
}

ClusterState contains EKS cluster specific states. Deployer is expected to write and read this. Read-only to kubetest.

type Config

type Config struct {
	// Tag is the tag used for S3 bucket name.
	// If empty, deployer auto-populates it.
	Tag string `json:"tag,omitempty"`
	// ClusterName is the cluster name.
	// If empty, deployer auto-populates it.
	ClusterName string `json:"cluster-name,omitempty"`

	// EKSTags defines EKS create cluster tags.
	EKSTags map[string]string `json:"eks-tags,omitempty"`
	// EKSRequestHeader defines EKS create cluster request header.
	EKSRequestHeader map[string]string `json:"eks-request-header,omitempty"`
	// EKSResolverURL defines an AWS resolver endpoint for EKS.
	// Must be left empty to use production EKS service.
	EKSResolverURL string `json:"eks-resolver-url"`
	// EKSSigningName is the EKS create request signing name.
	EKSSigningName string `json:"eks-signing-name"`

	// AWSK8sTesterPath is the path to download the "aws-k8s-tester".
	// This is required for Kubernetes kubetest plugin.
	AWSK8sTesterPath string `json:"aws-k8s-tester-path,omitempty"`
	// AWSK8sTesterDownloadURL is the download URL to download "aws-k8s-tester" binary from.
	// It's only used for "kubetest" deployer interface.
	AWSK8sTesterDownloadURL string `json:"aws-k8s-tester-download-url,omitempty"`
	// KubectlPath is the path to download the "kubectl".
	KubectlPath string `json:"kubectl-path,omitempty"`
	// KubectlDownloadURL is the download URL to download "kubectl" binary from.
	// https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html
	KubectlDownloadURL string `json:"kubectl-download-url,omitempty"`
	// AWSIAMAuthenticatorPath is the path to download the "aws-iam-authenticator".
	// This is required for Kubernetes kubetest plugin.
	AWSIAMAuthenticatorPath string `json:"aws-iam-authenticator-path,omitempty"`
	// AWSIAMAuthenticatorDownloadURL is the download URL to download "aws-iam-authenticator" binary from.
	AWSIAMAuthenticatorDownloadURL string `json:"aws-iam-authenticator-download-url,omitempty"`

	// ConfigPath is the configuration file path.
	// Deployer is expected to update this file with latest status.
	ConfigPath string `json:"config-path,omitempty"`
	// ConfigPathBucket is the path inside S3 bucket.
	ConfigPathBucket string `json:"config-path-bucket,omitempty"` // read-only to user
	ConfigPathURL    string `json:"config-path-url,omitempty"`    // read-only to user

	// KubeConfigPath is the file path of KUBECONFIG for the EKS cluster.
	// If empty, auto-generate one.
	// Deployer is expected to delete this on cluster tear down.
	KubeConfigPath string `json:"kubeconfig-path,omitempty"` // read-only to user
	// KubeConfigPathBucket is the path inside S3 bucket.
	KubeConfigPathBucket string `json:"kubeconfig-path-bucket,omitempty"` // read-only to user
	KubeConfigPathURL    string `json:"kubeconfig-path-url,omitempty"`    // read-only to user

	// DestroyAfterCreate is true to automatically tear down cluster.
	DestroyAfterCreate bool `json:"destroy-after-create"`
	// DestroyWaitTime is the duration to sleep before cluster tear down.
	// Be ignored if "DestroyAfterCreate" is false.
	DestroyWaitTime time.Duration `json:"destroy-wait-time,omitempty"`

	// AWSAccountID is the AWS account ID.
	AWSAccountID string `json:"aws-account-id,omitempty"`
	// AWSCredentialToMountPath is the file path to AWS credential.
	// Required for AWS ALB Ingress Controller deployments and other AWS specific tests.
	// If not empty, deployer is expected to mount the file as a secret object "aws-cred-aws-k8s-tester",
	// to the path "/etc/aws-cred-aws-k8s-tester/aws-cred-aws-k8s-tester", under "kube-system" namespace.
	// Path must be an absolute path, although it will try to parse '~/.aws' or '${HOME}/.aws'.
	// If "AWS_SHARED_CREDENTIALS_FILE" is specified, this field will overwritten.
	AWSCredentialToMountPath string `json:"aws-credential-to-mount-path,omitempty"`
	// AWSRegion is the AWS geographic area for EKS deployment.
	// If empty, set default region.
	AWSRegion string `json:"aws-region,omitempty"`

	// EnableWorkerNodeSSH is true to enable SSH access to worker nodes.
	EnableWorkerNodeSSH bool `json:"enable-worker-node-ssh"`
	// EnableWorkerNodeHA is true to use all 3 subnets to create worker nodes.
	// Note that at least 2 subnets are required for EKS cluster.
	EnableWorkerNodeHA bool `json:"enable-worker-node-ha"`
	// EnableWorkerNodePrivilegedPortAccess is true to allow control plane to
	// talk to worker nodes through their privileged ports (i.e ports 1-1024).
	EnableWorkerNodePrivilegedPortAccess bool `json:"enable-worker-node-privileged-port-access"`

	// VPCID is the VPC ID.
	VPCID string `json:"vpc-id"`
	// SubnetIDs is the subnet IDs.
	SubnetIDs []string `json:"subnet-ids"`
	// SecurityGroupID is the default security group ID.
	SecurityGroupID string `json:"security-group-id"`

	// WorkerNodeCFTemplatePath is the file path of worker node template.
	// If empty, download https://raw.githubusercontent.com/awslabs/amazon-eks-ami/master/amazon-eks-nodegroup.yaml.
	WorkerNodeCFTemplatePath string `json:"worker-node-cf-template-path,omitempty"`
	// WorkerNodeCFTemplateAdditionalParameterKeys defines a list of additional cloudformation parameter keys.
	WorkerNodeCFTemplateAdditionalParameterKeys []string `json:"worker-node-cf-template-additional-parameter-keys,omitempty"`
	// WorkerNodePrivateKeyPath is the file path to store node group key pair private key.
	// Thus, deployer must delete the private key right after node group creation.
	// MAKE SURE PRIVATE KEY NEVER GETS UPLOADED TO CLOUD STORAGE AND DELETE AFTER USE!!!
	WorkerNodePrivateKeyPath string `json:"worker-node-private-key-path"`
	// WorkerNodeAMIType is either "amazon-linux-2" or "amazon-linux-2-gpu".
	// Be ignored if "WorkerNodeAMIID" is specified.
	// Must be non-empty if "WorkerNodeAMIID" is NOT specified.
	WorkerNodeAMIType string `json:"worker-node-ami-type"`
	// WorkerNodeUserName is the user name for worker node SSH access.
	WorkerNodeUserName string `json:"worker-node-user-name"`
	// WorkerNodeAMIID is the Amazon EKS worker node AMI ID for the specified Region.
	// Reference https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html.
	// Leave empty to auto-populate from SSM parameter.
	WorkerNodeAMIID string `json:"worker-node-ami-id"`
	// WorkerNodeAMIName is the name of the worker node AMI.
	// Leave empty to auto-populate from SSM parameter.
	WorkerNodeAMIName string `json:"worker-node-ami-name"`
	// WorkerNodeInstanceType is the EC2 instance type for worker nodes.
	WorkerNodeInstanceType string `json:"worker-node-instance-type"`
	// WorkerNodeASGMin is the minimum number of nodes in worker node ASG.
	WorkerNodeASGMin int `json:"worker-node-asg-min,omitempty"`
	// WorkerNodeASGMax is the maximum number of nodes in worker node ASG.
	WorkerNodeASGMax int `json:"worker-node-asg-max,omitempty"`
	// WorkerNodeASGDesiredCapacity is the desired capacity of Node Group ASG.
	WorkerNodeASGDesiredCapacity int `json:"worker-node-asg-desired-capacity,omitempty"`

	// WorkerNodeVolumeSizeGB is the maximum number of nodes in worker node ASG.
	// If empty, set default value.
	WorkerNodeVolumeSizeGB int `json:"worker-node-volume-size-gb,omitempty"`

	// KubernetesVersion is the version of Kubernetes cluster.
	// If empty, set default version.
	KubernetesVersion string `json:"kubernetes-version,omitempty"`
	// PlatformVersion is the platform version of EKS.
	// Read-only to user.
	PlatformVersion string `json:"platform-version,omitempty"`

	// LogLevel configures log level. Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
	LogLevel string `json:"log-level"`
	// LogOutputs is a list of log outputs. Valid values are 'default', 'stderr', 'stdout', or file names.
	// Logs are appended to the existing file, if any.
	// Multiple values are accepted. If empty, it sets to 'default', which outputs to stderr.
	// See https://godoc.org/go.uber.org/zap#Open and https://godoc.org/go.uber.org/zap#Config for more details.
	LogOutputs []string `json:"log-outputs,omitempty"`
	// LogOutputToUploadPath is the aws-k8s-tester log file path to upload to cloud storage.
	// Must be left empty.
	// This will be overwritten by cluster name.
	LogOutputToUploadPath       string `json:"log-output-to-upload-path,omitempty"`
	LogOutputToUploadPathBucket string `json:"log-output-to-upload-path-bucket,omitempty"`
	LogOutputToUploadPathURL    string `json:"log-output-to-upload-path-url,omitempty"`

	// LogAccess is true to enable AWS API access logs (e.g. ALB access logs).
	// Automatically uploaded to S3 bucket named by cluster name.
	// https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html
	// https://github.com/kubernetes-sigs/aws-alb-ingress-controller/blob/master/docs/ingress-resources.md
	LogAccess bool `json:"log-access"`

	// UploadTesterLogs is true to auto-upload log files.
	UploadTesterLogs bool `json:"upload-tester-logs"`
	// UploadKubeConfig is true to auto-upload KUBECONFIG file.
	UploadKubeConfig bool `json:"upload-kubeconfig"`
	// UploadWorkerNodeLogs is true to auto-upload worker node log files.
	UploadWorkerNodeLogs bool `json:"upload-worker-node-logs"`
	// UploadBucketExpireDays is the number of days for a S3 bucket to expire.
	// Set 0 to not expire.
	UploadBucketExpireDays int `json:"upload-bucket-expire-days"`

	// UpdatedAt is the timestamp when the configuration has been updated.
	// Read only to 'Config' struct users.
	UpdatedAt time.Time `json:"updated-at,omitempty"` // read-only to user

	// ClusterState is the EKS status state.
	// Deployer is expected to keep this in sync.
	// Read-only to kubetest.
	ClusterState *ClusterState `json:"cluster-state,omitempty"`

	// CFStackVPCName is the name of VPC cloudformation stack.
	// Read-only. Only used to create a new one.
	CFStackVPCName string `json:"cf-stack-vpc-name,omitempty"`
	// CFStackVPCStatus is the last cloudformation status of VPC stack.
	CFStackVPCStatus string `json:"cf-stack-vpc-status,omitempty"`
	// CFStackVPCParameterVPCBlock is CIDR range for the VPC.
	// This should be a valid private (RFC 1918) CIDR range.
	CFStackVPCParameterVPCBlock      string `json:"cf-stack-vpc-parameter-vpc-block"`
	CFStackVPCParameterSubnet01Block string `json:"cf-stack-vpc-parameter-subnet-01-block"`
	CFStackVPCParameterSubnet02Block string `json:"cf-stack-vpc-parameter-subnet-02-block"`
	CFStackVPCParameterSubnet03Block string `json:"cf-stack-vpc-parameter-subnet-03-block"`
}

Config defines EKS test configuration.

func Load

func Load(p string) (cfg *Config, err error)

Load loads configuration from YAML. Useful when injecting shared configuration via ConfigMap.

Example usage:

import "github.com/aws/aws-k8s-tester/eksconfig"
cfg := eksconfig.Load("test.yaml")
err := cfg.ValidateAndSetDefaults()

Do not set default values in this function. "ValidateAndSetDefaults" must be called separately, to prevent overwriting previous data when loaded from disks.

func NewDefault

func NewDefault() *Config

NewDefault returns a copy of the default configuration.

func (*Config) KubectlCommands

func (cfg *Config) KubectlCommands() (s string)

KubectlCommands returns the SSH commands.

func (*Config) SSHCommands

func (cfg *Config) SSHCommands() (s string)

SSHCommands returns the SSH commands.

func (*Config) SetClusterUpTook

func (cfg *Config) SetClusterUpTook(d time.Duration)

SetClusterUpTook updates 'ClusterUpTook' field.

func (*Config) Sync

func (cfg *Config) Sync() (err error)

Sync persists current configuration and states to disk.

func (*Config) UpdateFromEnvs

func (cfg *Config) UpdateFromEnvs() error

UpdateFromEnvs updates fields from environmental variables.

func (*Config) ValidateAndSetDefaults

func (cfg *Config) ValidateAndSetDefaults() error

ValidateAndSetDefaults returns an error for invalid configurations. And updates empty fields with default values. At the end, it writes populated YAML to aws-k8s-tester config path.