aws-k8s-tester

module
Version: v1.6.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2021 License: Apache-2.0

README

aws-k8s-tester

Go Report Card Godoc Releases LICENSE

https://github.com/kubernetes/enhancements/blob/master/keps/provider-aws/20181126-aws-k8s-tester.md

aws-k8s-tester is a set of utilities and libraries for "testing" Kubernetes on AWS.

  • Implements test-infra/kubetest2 interface.
  • Uses AWS CloudFormation for resource creation.
  • Supports automatic rollback and resource deletion.
  • Flexible add-on support via environmental variables.
  • Extensible as a Go package; eks.Tester.Up to create EKS.
  • Performance tests suites.

The main goal is to create "temporary" EC2 instances or EKS clusters for "testing" purposes:

Install

https://github.com/aws/aws-k8s-tester/releases

aws-k8s-tester eks

Make sure AWS credential is located in your machine:

# confirm credential is valid
aws sts get-caller-identity --query Arn --output text

See the following for more fields:

# easiest way, use the defaults
# creates role, VPC, EKS cluster
rm -rf /tmp/${USER}-test-eks-prod*
aws-k8s-tester eks create cluster --enable-prompt=true -p /tmp/${USER}-test-prod-eks.yaml
aws-k8s-tester eks delete cluster --enable-prompt=true -p /tmp/${USER}-test-prod-eks.yaml

# advanced options can be set via environmental variables
# e.g. node groups, managed node groups, add-ons
rm -rf /tmp/${USER}-test-eks*
AWS_K8S_TESTER_EKS_PARTITION=aws \
AWS_K8S_TESTER_EKS_REGION=us-west-2 \
AWS_K8S_TESTER_EKS_LOG_COLOR=true \
AWS_K8S_TESTER_EKS_S3_BUCKET_CREATE=true \
AWS_K8S_TESTER_EKS_S3_BUCKET_CREATE_KEEP=true \
AWS_K8S_TESTER_EKS_COMMAND_AFTER_CREATE_CLUSTER="aws eks describe-cluster --name GetRef.Name" \
AWS_K8S_TESTER_EKS_COMMAND_AFTER_CREATE_ADD_ONS="aws eks describe-cluster --name GetRef.Name" \
AWS_K8S_TESTER_EKS_PARAMETERS_ENCRYPTION_CMK_CREATE=true \
AWS_K8S_TESTER_EKS_PARAMETERS_ROLE_CREATE=true \
AWS_K8S_TESTER_EKS_PARAMETERS_VERSION=1.17 \
AWS_K8S_TESTER_EKS_PARAMETERS_VPC_CREATE=true \
AWS_K8S_TESTER_EKS_CLIENTS=5 \
AWS_K8S_TESTER_EKS_CLIENT_QPS=30 \
AWS_K8S_TESTER_EKS_CLIENT_BURST=20 \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ROLE_CREATE=true \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_FETCH_LOGS=false \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ASGS='{"GetRef.Name-ng-al2-cpu":{"name":"GetRef.Name-ng-al2-cpu","remote-access-user-name":"ec2-user","ami-type":"AL2_x86_64","image-id":"","image-id-ssm-parameter":"/aws/service/eks/optimized-ami/1.17/amazon-linux-2/recommended/image_id","instance-types":["c5.xlarge"],"volume-size":40,"asg-min-size":2,"asg-max-size":2,"asg-desired-capacity":2,"kubelet-extra-args":""},"GetRef.Name-ng-al2-gpu":{"name":"GetRef.Name-ng-al2-gpu","remote-access-user-name":"ec2-user","ami-type":"AL2_x86_64_GPU","image-id":"","image-id-ssm-parameter":"/aws/service/eks/optimized-ami/1.17/amazon-linux-2-gpu/recommended/image_id","instance-types":["p3.8xlarge"],"volume-size":40,"asg-min-size":1,"asg-max-size":1,"asg-desired-capacity":1,"kubelet-extra-args":""},"GetRef.Name-ng-bottlerocket":{"name":"GetRef.Name-ng-bottlerocket","remote-access-user-name":"ec2-user","ami-type":"BOTTLEROCKET_x86_64","image-id":"","image-id-ssm-parameter":"/aws/service/bottlerocket/aws-k8s-1.15/x86_64/latest/image_id","ssm-document-cfn-stack-name":"GetRef.Name-install-bottlerocket","ssm-document-name":"GetRef.Name-InstallBottlerocket","ssm-document-create":true,"ssm-document-commands":"enable-admin-container","ssm-document-execution-timeout-seconds":3600,"instance-types":["c5.xlarge"],"volume-size":40,"asg-min-size":2,"asg-max-size":2,"asg-desired-capacity":2}}' \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_ROLE_CREATE=true \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_FETCH_LOGS=false \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_MNGS='{"GetRef.Name-mng-al2-cpu":{"name":"GetRef.Name-mng-al2-cpu","remote-access-user-name":"ec2-user","release-version":"","ami-type":"AL2_x86_64","instance-types":["c5.xlarge"],"volume-size":40,"asg-min-size":2,"asg-max-size":2,"asg-desired-capacity":2},"GetRef.Name-mng-al2-gpu":{"name":"GetRef.Name-mng-al2-gpu","remote-access-user-name":"ec2-user","release-version":"","ami-type":"AL2_x86_64_GPU","instance-types":["p3.8xlarge"],"volume-size":40,"asg-min-size":1,"asg-max-size":1,"asg-desired-capacity":1}}' \
AWS_K8S_TESTER_EKS_ADD_ON_FLUENTD_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_METRICS_SERVER_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CONFORMANCE_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_APP_MESH_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CSI_EBS_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_KUBERNETES_DASHBOARD_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_PROMETHEUS_GRAFANA_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_NLB_HELLO_WORLD_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_NLB_GUESTBOOK_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_ALB_2048_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_JOBS_PI_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_JOBS_ECHO_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CRON_JOBS_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CSRS_LOCAL_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CONFIGMAPS_LOCAL_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_SECRETS_LOCAL_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_WORDPRESS_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_JUPYTER_HUB_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CUDA_VECTOR_ADD_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CLUSTER_LOADER_LOCAL_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_HOLLOW_NODES_LOCAL_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_STRESSER_LOCAL_ENABLE=true \
aws-k8s-tester eks create cluster --enable-prompt=true -p /tmp/${USER}-test-eks.yaml

<<COMMENT
# to delete
aws-k8s-tester eks delete cluster --enable-prompt=true -p /tmp/${USER}-test-eks.yaml

# run "eks create config" to check/edit configuration file first 
aws-k8s-tester eks create config -p /tmp/${USER}-test-eks.yaml

# run the following command with those envs overwrites configuration, and create
aws-k8s-tester eks create cluster --enable-prompt=true -p /tmp/${USER}-test-eks.yaml
COMMENT

<<COMMENT
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text);
echo ${ACCOUNT_ID}
CLUSTER_ARN=arn:aws:eks:us-west-2:${ACCOUNT_ID}:cluster/${USER}-test-eks
echo ${CLUSTER_ARN}

# to assign a non-random cluster name
# if empty, name is auto-generated
AWS_K8S_TESTER_EKS_NAME=${USER}-test-eks \

# to create/delete a S3 bucket for test artifacts
AWS_K8S_TESTER_EKS_S3_BUCKET_CREATE=true \

# to reuse an existing S3 bucket
AWS_K8S_TESTER_EKS_S3_BUCKET_CREATE=false \
AWS_K8S_TESTER_EKS_S3_BUCKET_NAME=${BUCKET_NAME} \

# to automatically create EC2 key-pair
AWS_K8S_TESTER_EKS_REMOTE_ACCESS_KEY_CREATE=true \

# to reuse an existing EC2 key-pair
AWS_K8S_TESTER_EKS_REMOTE_ACCESS_KEY_CREATE=false \
AWS_K8S_TESTER_EKS_REMOTE_ACCESS_KEY_NAME=${KEY_NAME} \
AWS_K8S_TESTER_EKS_REMOTE_ACCESS_PRIVATE_KEY_PATH=${KEY_PATH} \

# to reuse an existing role for "EKS cluster"
AWS_K8S_TESTER_EKS_PARAMETERS_ROLE_CREATE=false \
AWS_K8S_TESTER_EKS_PARAMETERS_ROLE_ARN=${CLUSTER_ROLE_ARN} \

# to reuse an existing VPC
AWS_K8S_TESTER_EKS_PARAMETERS_VPC_CREATE=false \
AWS_K8S_TESTER_EKS_PARAMETERS_VPC_ID=${VPC_ID} \

# to reuse an existing role for "Node Group"
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ROLE_CREATE=false \
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ROLE_ARN=${NG_ROLE_ARN} \

# to reuse an existing role for "Managed Node Group"
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_ROLE_CREATE=false \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_ROLE_ARN=${MNG_ROLE_ARN} \

# to reuse an existing role for "Fargate"
AWS_K8S_TESTER_EKS_ADD_ON_FARGATE_ROLE_CREATE=false \
AWS_K8S_TESTER_EKS_ADD_ON_FARGATE_ROLE_ARN=${FARGATE_ROLE_ARN} \

# to user ${USER} in node groups
AWS_K8S_TESTER_EKS_ADD_ON_NODE_GROUPS_ASGS={\"${USER}-test-eks-ng-al2-cpu\":{\"name\":\"${USER}-test-eks-ng-al2-cpu\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"AL2_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/eks/optimized-ami/1.15/amazon-linux-2/recommended/image_id\",\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40,\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1,\"kubelet-extra-args\":\"\"},\"${USER}-test-eks-ng-bottlerocket\":{\"name\":\"${USER}-test-eks-ng-bottlerocket\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"BOTTLEROCKET_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/bottlerocket/aws-k8s-1.15/x86_64/latest/image_id\",\"ssm-document-cfn-stack-name\":\"${USER}-install-bottle-rocket\",\"ssm-document-name\":\"${USER}InstallBottleRocket\",\"ssm-document-create\":true,\"ssm-document-commands\":\"enable-admin-container\",\"ssm-document-execution-timeout-seconds\":3600,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40,\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1}} \
AWS_K8S_TESTER_EKS_ADD_ON_MANAGED_NODE_GROUPS_MNGS={\"${USER}-test-eks-mng-al2-cpu\":{\"name\":\"${USER}-test-eks-mng-al2-cpu\",\"remote-access-user-name\":\"ec2-user\",\"release-version\":\"\",\"ami-type\":\"AL2_x86_64\",\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40,\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1}} \

# to build ECR images for remote tester add-ons
cd ${GOPATH}/src/github.com/aws/aws-k8s-tester
make docker-push ACCOUNT_ID=YOUR_ACCOUNT_ID TAG=latest

AWS_K8S_TESTER_EKS_ADD_ON_CLUSTER_LOADER_REMOTE_ENABLE=true \
AWS_K8S_TESTER_EKS_ADD_ON_CLUSTER_LOADER_REMOTE_RUNS=1 \
AWS_K8S_TESTER_EKS_ADD_ON_CLUSTER_LOADER_REMOTE_REPOSITORY_ACCOUNT_ID=YOUR_ACCOUNT_ID \
AWS_K8S_TESTER_EKS_ADD_ON_CLUSTER_LOADER_REMOTE_REPOSITORY_NAME=aws/aws-k8s-tester \
AWS_K8S_TESTER_EKS_ADD_ON_CLUSTER_LOADER_REMOTE_REPOSITORY_IMAGE_TAG=latest \
COMMENT

This will create an EKS cluster with a worker node (takes about 20 minutes).

Once cluster is created, check cluster state using AWS CLI:

aws eks describe-cluster \
  --name ${EKS_CLUSTER_NAME} \
  --query cluster.status

"ACTIVE"

Cluster states are persisted on disk and S3 bucket.

EKS tester uses this file to record status.

cat /tmp/config.yaml

# or
less +FG /tmp/config.yaml

ec2-utils

Make sure AWS credential is located in your machine:

# confirm credential is valid
aws sts get-caller-identity --query Arn --output text

See the following for more fields:

# easiest way, use the defaults
# creates role, VPC, EC2 ASG
rm -rf /tmp/${USER}-test-ec2*
ec2-utils create instances --enable-prompt=true -p /tmp/${USER}-test-ec2.yaml
ec2-utils delete instances --enable-prompt=true -p /tmp/${USER}-test-ec2.yaml

# advanced options can be set via environmental variables
rm -rf /tmp/${USER}-test-ec2*
AWS_K8S_TESTER_EC2_ON_FAILURE_DELETE=true \
AWS_K8S_TESTER_EC2_PARTITION=aws \
AWS_K8S_TESTER_EC2_REGION=us-west-2 \
AWS_K8S_TESTER_EC2_S3_BUCKET_CREATE=true \
AWS_K8S_TESTER_EC2_S3_BUCKET_CREATE_KEEP=true \
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_KEY_CREATE=true \
AWS_K8S_TESTER_EC2_ASGS_FETCH_LOGS=true \
AWS_K8S_TESTER_EC2_ASGS='{"GetRef.Name-al2-cpu":{"name":"GetRef.Name-al2-cpu","remote-access-user-name":"ec2-user","ami-type":"AL2_x86_64","image-id-ssm-parameter":"/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2","instance-types":["c5.xlarge"],"volume-size":40,"asg-min-size":1,"asg-max-size":1,"asg-desired-capacity":1},"GetRef.Name-bottlerocket":{"name":"GetRef.Name-bottlerocket","remote-access-user-name":"ec2-user","ami-type":"BOTTLEROCKET_x86_64","image-id-ssm-parameter":"/aws/service/bottlerocket/aws-k8s-1.15/x86_64/latest/image_id","ssm-document-cfn-stack-name":"GetRef.Name-install-bottlerocket","ssm-document-name":"GetRef.Name-install-bottlerocket","ssm-document-create":true,"ssm-document-commands":"enable-admin-container","ssm-document-execution-timeout-seconds":3600,"instance-types":["c5.xlarge"],"volume-size":40,"asg-min-size":1,"asg-max-size":1,"asg-desired-capacity":1}}' \
AWS_K8S_TESTER_EC2_ROLE_CREATE=true \
AWS_K8S_TESTER_EC2_VPC_CREATE=true \
ec2-utils create instances --enable-prompt=true -p /tmp/${USER}-test-ec2.yaml

<<COMMENT
# to delete
ec2-utils delete instances --enable-prompt=true -p /tmp/${USER}-test-ec2.yaml

# run "ec2 create config" to check/edit configuration file first 
ec2-utils create config -p /tmp/${USER}-test-ec2.yaml
ec2-utils create instances -p /tmp/${USER}-test-ec2.yaml

# run the following command with those envs overwrites configuration, and create
ec2-utils create instances --enable-prompt=true -p /tmp/${USER}-test-ec2.yaml
COMMENT

<<COMMENT
# to config a fixed name for EC2 ASG
AWS_K8S_TESTER_EC2_NAME=${NAME} \

# to create/delete a S3 bucket for test artifacts
AWS_K8S_TESTER_EC2_S3_BUCKET_CREATE=true \

# to reuse an existing S3 bucket
AWS_K8S_TESTER_EC2_S3_BUCKET_CREATE=false \
AWS_K8S_TESTER_EC2_S3_BUCKET_NAME=${BUCKET_NAME} \

# to automatically create EC2 key-pair
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_KEY_CREATE=true \

# to reuse an existing EC2 key-pair
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_KEY_CREATE=false \
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_KEY_NAME=${KEY_NAME} \
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_PRIVATE_KEY_PATH=${KEY_PATH} \

# to reuse an existing role
AWS_K8S_TESTER_EC2_ROLE_CREATE=false \
AWS_K8S_TESTER_EC2_ROLE_ARN=${ROLE_ARN} \

# to reuse an existing VPC
AWS_K8S_TESTER_EC2_VPC_CREATE=false \
AWS_K8S_TESTER_EC2_VPC_ID=${VPC_ID} \

# to use ${USER}
AWS_K8S_TESTER_EC2_ASGS={\"${USER}-test-ec2-al2-cpu\":{\"name\":\"${USER}-test-ec2-al2-cpu\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"AL2_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2\",,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40,\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1},\"${USER}-test-ec2-bottlerocket\":{\"name\":\"${USER}-test-ec2-bottlerocket\",\"remote-access-user-name\":\"ec2-user\",\"ami-type\":\"BOTTLEROCKET_x86_64\",\"image-id-ssm-parameter\":\"/aws/service/bottlerocket/aws-k8s-1.15/x86_64/latest/image_id\",\"ssm-document-cfn-stack-name\":\"${USER}-install-bottlerocket\",\"ssm-document-name\":\"${USER}InstallBottleRocket\",\"ssm-document-create\":true,\"ssm-document-commands\":\"enable-admin-container\",\"ssm-document-execution-timeout-seconds\":3600,,\"instance-types\":[\"c5.xlarge\"],\"volume-size\":40,\"asg-min-size\":1,\"asg-max-size\":1,\"asg-desired-capacity\":1}} \
COMMENT

eks-utils apis

Install eks-utils from https://github.com/aws/aws-k8s-tester/releases.

AWS_K8S_TESTER_VERSION=${LATEST_RELEASE_VERSION}

DOWNLOAD_URL=https://github.com/aws/aws-k8s-tester/releases/download
rm -rf /tmp/aws-k8s-tester
rm -rf /tmp/eks-utils

if [[ "${OSTYPE}" == "linux"* ]]; then
  curl -L ${DOWNLOAD_URL}/${AWS_K8S_TESTER_VERSION}/eks-utils-${AWS_K8S_TESTER_VERSION}-linux-amd64 -o /tmp/eks-utils
elif [[ "${OSTYPE}" == "darwin"* ]]; then
  curl -L ${DOWNLOAD_URL}/${AWS_K8S_TESTER_VERSION}/eks-utils-${AWS_K8S_TESTER_VERSION}-darwin-amd64 -o /tmp/eks-utils
fi

chmod +x /tmp/eks-utils
/tmp/eks-utils version

kube-apiserver-audit logs will show:

{
    "kind": "Event",
    "apiVersion": "audit.k8s.io/v1",
    "level": "Request",
    "auditID": "b7ff399c-2d27-4f47-98b4-0b87c0ceb436",
    "stage": "ResponseComplete",
    "requestURI": "/api/v1/nodes?limit=30",
    "verb": "list",
    "user": {
        "username": "system:unsecured",
        "groups": [
            "system:masters",
            "system:authenticated"
        ]
    },
    "sourceIPs": [
        "127.0.0.1"
    ],
    "userAgent": "eks-utils/v0.0.0 (linux/amd64) kubernetes/$Format",
    "objectRef": {
        "resource": "nodes",
        "apiVersion": "v1"
    },
    "responseStatus": {
        "metadata": {},
        "code": 200
    },
    "requestReceivedTimestamp": "2020-06-03T17:45:21.368826Z",
    "stageTimestamp": "2020-06-03T17:45:21.372015Z"
}

eks-utils apis helps with API deprecation (e.g. https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.17.md#deprecations-and-removals).

WARNING: kubectl internally converts API versions in the response (see kubernetes/issues#58131). Which means kubectl get output may have different API versions than the one persisted in etcd. Upstream Kubernetes recommends upgrading deprecated API with get and put:

the simplest approach is to get/put every object after upgrades. objects that don't need migration will no-op (they won't even increment resourceVersion in etcd). objects that do need migration will persist in the new preferred storage version

Which means there's no way in client-side to find all resources created with deprecated API groups. The only way to ensure API group upgrades is list all resources, and execute get and put with the latest API group version. If the resource has already latest API version, it will be no-op. Otherwise, it will upgrade to the latest API version.

eks-utils apis will help with the list calls with proper pagination and generate get and put scripts for the cluster:

# to check supported API groups from current kube-apiserver
eks-utils apis \
  --kubeconfig /tmp/kubeconfig.yaml \
  supported

# to write API upgrade/rollback scripts and YAML files in "/tmp/eks-utils"
#
# make sure to set proper "--batch-limit" and "--batch-interval"
# to not overload EKS master; if it's set too high, it can affect
# production workloads slowing down kube-apiserver
rm -rf /tmp/eks-utils-resources
eks-utils apis \
  --kubeconfig /tmp/kubeconfig.yaml \
  --enable-prompt \
  deprecate \
  --batch-limit 10 \
  --batch-interval 2s \
  --dir /tmp/eks-utils-resources

# this command does not apply or create any resources
# it only lists the resources that need be upgraded

# if there's any resources that needs upgrade,
# it writes patched YAML file, original YAML file,
# bash scripts to update and rollback
find /tmp/eks-utils-resources

etcd-utils k8s list

etcd-utils k8s list helps with API deprecation (e.g. https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.17.md#deprecations-and-removals).

WARNING: kubectl internally converts API versions in the response (see kubernetes/issues#58131). Which means kubectl get output may have different API versions than the one persisted in etcd . Upstream Kubernetes recommends upgrading deprecated API with get and put:

the simplest approach is to get/put every object after upgrades. objects that don't need migration will no-op (they won't even increment resourceVersion in etcd). objects that do need migration will persist in the new preferred storage version

To minimize the impact of list calls, etcd-utils k8s list reads keys with leadership election and pagination; only a single worker can run at a time.

# to list all deployments with etcd pagination + k8s decoder
etcd-utils k8s \
  --endpoints http://localhost:2379 \
  list \
  --prefixes /registry/deployments \
  --output /tmp/etcd-utils-k8s-list.output.yaml

# or ".json"

Directories

Path Synopsis
client module
cmd
aws-k8s-tester
aws-k8s-tester is a set of AWS Kubernetes test commands.
aws-k8s-tester is a set of AWS Kubernetes test commands.
aws-k8s-tester/eks
Package eks implements EKS related commands.
Package eks implements EKS related commands.
aws-k8s-tester/version
Package version implements version command.
Package version implements version command.
cw-utils
cw-utils is a set of AWS CloudWatch utilities commands.
cw-utils is a set of AWS CloudWatch utilities commands.
cw-utils/metrics-image
Package metricsimage implements AWS CloudWatch related commands.
Package metricsimage implements AWS CloudWatch related commands.
cw-utils/version
Package version implements version command.
Package version implements version command.
ec2-utils
ec2-utils is a set of AWS EC2 commands.
ec2-utils is a set of AWS EC2 commands.
ec2-utils/create
Package create implements "ec2-utils create" commands.
Package create implements "ec2-utils create" commands.
ec2-utils/delete
Package delete implements "ec2-utils delete" commands.
Package delete implements "ec2-utils delete" commands.
ec2-utils/version
Package version implements version command.
Package version implements version command.
ecr-utils
ecr-utils is a set of AWS utilities commands.
ecr-utils is a set of AWS utilities commands.
ecr-utils/create-repo
Package createrepo implements "ecr-utils create-repo" commands.
Package createrepo implements "ecr-utils create-repo" commands.
ecr-utils/set-policy
Package setpolicy implements "ecr-utils set-policy" commands.
Package setpolicy implements "ecr-utils set-policy" commands.
ecr-utils/version
Package version implements version command.
Package version implements version command.
eks-utils
eks-utils is a set of EKS utilities commands.
eks-utils is a set of EKS utilities commands.
eks-utils/apis
Package apis implements EKS API related commands.
Package apis implements EKS API related commands.
eks-utils/nodes
Pcmdkage nodes implements EKS node related commands.
Pcmdkage nodes implements EKS node related commands.
eks-utils/version
Package version implements version command.
Package version implements version command.
etcd-utils
etcd-utils is a set of etcd utilities commands.
etcd-utils is a set of etcd utilities commands.
etcd-utils/k8s
Package k8s implements etcd k8s related commands.
Package k8s implements etcd k8s related commands.
etcd-utils/version
Package version implements version command.
Package version implements version command.
s3-utils
s3-utils is a set of AWS utilities commands.
s3-utils is a set of AWS utilities commands.
s3-utils/cp
Package cp implements "aws s3 cp" commands.
Package cp implements "aws s3 cp" commands.
s3-utils/version
Package version implements version command.
Package version implements version command.
sts-utils
sts-utils is a set of AWS utilities commands.
sts-utils is a set of AWS utilities commands.
sts-utils/get-caller-identity
Package getcalleridentity implements "aws sts get-caller-identity" commands.
Package getcalleridentity implements "aws sts get-caller-identity" commands.
sts-utils/version
Package version implements version command.
Package version implements version command.
e2e
framework
Package framework implements e2e testing.
Package framework implements e2e testing.
framework/resource
Package resource implements e2e testing resource.
Package resource implements e2e testing resource.
framework/utils
Package utils implements e2e testing utils.
Package utils implements e2e testing utils.
resources
Package resources implements Kubernetes resources utilities.
Package resources implements Kubernetes resources utilities.
utils
Package utils implements Kubernetes utilities.
Package utils implements Kubernetes utilities.
tester Module
Package ec2 implements testing utilities using EC2.
Package ec2 implements testing utilities using EC2.
Package ec2config defines EC2 configuration.
Package ec2config defines EC2 configuration.
gen
gen generates ec2config documentation.
gen generates ec2config documentation.
eks
Package eks implements EKS cluster operations.
Package eks implements EKS cluster operations.
alb-2048
Package alb2048 implements ALB plugin that installs 2048.
Package alb2048 implements ALB plugin that installs 2048.
amazon-eks-ami-issue-454
This is intended to test a soft lockup issue described here: https://github.com/awslabs/amazon-eks-ami/issues/454 This is based off of the following repro: https://github.com/mmerkes/eks-k8s-repro-assistant/tree/master/scenarios/decompression-loop
This is intended to test a soft lockup issue described here: https://github.com/awslabs/amazon-eks-ami/issues/454 This is based off of the following repro: https://github.com/mmerkes/eks-k8s-repro-assistant/tree/master/scenarios/decompression-loop
app-mesh
Package appmesh implements App Mesh add-on.
Package appmesh implements App Mesh add-on.
cluster
Package cluster implements EKS cluster tester.
Package cluster implements EKS cluster tester.
cluster-loader
Package clusterloader implements cluster loader.
Package clusterloader implements cluster loader.
cluster-loader/local
Package local implements local cluster loader.
Package local implements local cluster loader.
cluster-loader/remote
Package remote implements remote cluster loader.
Package remote implements remote cluster loader.
cluster/version-upgrade
Package versionupgrade implements EKS cluster version upgrade tester.
Package versionupgrade implements EKS cluster version upgrade tester.
cluster/wait
Package wait implements cluster waiter.
Package wait implements cluster waiter.
cluster/wait-v2
Package wait_v2 implements cluster waiter.
Package wait_v2 implements cluster waiter.
cni-vpc
Package cnivpc installs "https://github.com/aws/amazon-vpc-cni-k8s".
Package cnivpc installs "https://github.com/aws/amazon-vpc-cni-k8s".
configmaps
Package configmaps implements tester for ConfigMap.
Package configmaps implements tester for ConfigMap.
configmaps/local
Package local implements tester for ConfigMap.
Package local implements tester for ConfigMap.
configmaps/remote
Package remote implements tester for ConfigMap.
Package remote implements tester for ConfigMap.
conformance
Package conformance implements Conformance tests.
Package conformance implements Conformance tests.
cron-jobs
Package cronjobs creates CronJob objects in Kubernetes.
Package cronjobs creates CronJob objects in Kubernetes.
csi-ebs
Package csiebs installs "aws-ebs-csi-driver".
Package csiebs installs "aws-ebs-csi-driver".
csrs
Package csrs implements tester for CertificateSigningRequest.
Package csrs implements tester for CertificateSigningRequest.
csrs/local
Package local implements tester for CertificateSigningRequest.
Package local implements tester for CertificateSigningRequest.
csrs/remote
Package remote implements tester for CertificateSigningRequest.
Package remote implements tester for CertificateSigningRequest.
cuda-vector-add
Package cudavectoradd implements tester for CUDA GPU Test.
Package cudavectoradd implements tester for CUDA GPU Test.
cw-agent
Package cwagent implements CloudWatch agent plugin.
Package cwagent implements CloudWatch agent plugin.
fargate
Package fargate implements tester for Fargate.
Package fargate implements tester for Fargate.
fargate/wait
Package wait implements Fargate wait polling functions.
Package wait implements Fargate wait polling functions.
fluentd
Package fluentd implements Fluentd plugin.
Package fluentd implements Fluentd plugin.
gpu
Package gpu implements GPU plugin.
Package gpu implements GPU plugin.
helm
Package helm implements helm utilities.
Package helm implements helm utilities.
irsa
Package irsa implements tester for IAM Roles for Service Accounts (IRSA).
Package irsa implements tester for IAM Roles for Service Accounts (IRSA).
irsa-fargate
Package irsafargate implements tester for "IAM Roles for Service Accounts (IRSA)" Pod with Fargate.
Package irsafargate implements tester for "IAM Roles for Service Accounts (IRSA)" Pod with Fargate.
jobs-echo
Package jobsecho creates Job objects in Kubernetes.
Package jobsecho creates Job objects in Kubernetes.
jobs-pi
Package jobspi creates example Job objects in Kubernetes.
Package jobspi creates example Job objects in Kubernetes.
jupyter-hub
Package jupyterhub implements Jupyter Hub add-on.
Package jupyterhub implements Jupyter Hub add-on.
kubeflow
Package kubeflow implements kubeflow add-on.
Package kubeflow implements kubeflow add-on.
kubernetes-dashboard
Package kubernetesdashboard implements Kubernetes dashboard add-on.
Package kubernetesdashboard implements Kubernetes dashboard add-on.
metrics-server
Package metricsserver implements Kubernetes metrics server.
Package metricsserver implements Kubernetes metrics server.
mng
Package mng defines AWS EKS Managed Node Group configuration.
Package mng defines AWS EKS Managed Node Group configuration.
mng/scale
Package scale implements EKS cluster scaler tester.
Package scale implements EKS cluster scaler tester.
mng/version-upgrade
Package versionupgrade implements EKS cluster version upgrade tester.
Package versionupgrade implements EKS cluster version upgrade tester.
mng/wait
Package wait implements node waiter.
Package wait implements node waiter.
ng
Package ng implements EKS worker nodes with a custom AMI.
Package ng implements EKS worker nodes with a custom AMI.
ng/autoscaler
Package autoscaler implements various auto-scaler.
Package autoscaler implements various auto-scaler.
ng/wait
Package wait implements node waiter.
Package wait implements node waiter.
nlb-guestbook
Package nlbguestbook implements NLB plugin with a simple guestbook service.
Package nlbguestbook implements NLB plugin with a simple guestbook service.
nlb-hello-world
Package nlbhelloworld implements NLB plugin with a simple hello world service.
Package nlbhelloworld implements NLB plugin with a simple hello world service.
php-apache
Package phpapache implements PHP Apache with a simple PHP app.
Package phpapache implements PHP Apache with a simple PHP app.
prometheus-grafana
Package prometheusgrafana implements Prometheus/Grafana add-on.
Package prometheusgrafana implements Prometheus/Grafana add-on.
secrets
Package secrets implements Secrets plugin.
Package secrets implements Secrets plugin.
secrets/local
Package local implements Secrets plugin.
Package local implements Secrets plugin.
secrets/remote
Package remote implements Secrets plugin.
Package remote implements Secrets plugin.
stresser
Package stresser implements cluster load tests.
Package stresser implements cluster load tests.
stresser/local
Package local implements cluster local load tests.
Package local implements cluster local load tests.
stresser/remote
Package remote implements cluster remote load tests.
Package remote implements cluster remote load tests.
tester
Package tester defines EKS tester interface.
Package tester defines EKS tester interface.
wordpress
Package wordpress implements wordpress add-on.
Package wordpress implements wordpress add-on.
Package eksconfig defines EKS test configuration.
Package eksconfig defines EKS test configuration.
gen
gen generates eksconfig documentation.
gen generates eksconfig documentation.
k8s-tester module
aqua Module
armory Module
clusterloader Module
cni Module
configmaps Module
conformance Module
csi-ebs Module
csi-efs Module
csrs Module
epsagon Module
falco Module
fluent-bit Module
helm Module
jobs-echo Module
jobs-pi Module
kubecost Module
nlb-guestbook Module
php-apache Module
secrets Module
splunk Module
stress Module
sysdig Module
tester Module
vault Module
wordpress Module
pkg
aws
Package aws implements wrappers for AWS API calls.
Package aws implements wrappers for AWS API calls.
aws/cfn
Package cfn implements common CloudFormation utilities.
Package cfn implements common CloudFormation utilities.
aws/cw
Package cw implements common CloudWatch utilities.
Package cw implements common CloudWatch utilities.
aws/ec2
Package ec2 implements various AWS EC2 operations.
Package ec2 implements various AWS EC2 operations.
aws/ec2/instance-types
instance-types auto-generates EC2 instance types from AWS API.
instance-types auto-generates EC2 instance types from AWS API.
aws/ec2/metadata
Package metadata queries the EC2 metadata.
Package metadata queries the EC2 metadata.
aws/ecr
Package ecr implements ECR utilities.
Package ecr implements ECR utilities.
aws/elb
Package elb implements ELB utilities.
Package elb implements ELB utilities.
aws/iam
Package iam implements various IAM components.
Package iam implements various IAM components.
aws/s3
Package s3 implements S3 utilities.
Package s3 implements S3 utilities.
aws/ssm
Package ssm implements common SSM utilities.
Package ssm implements common SSM utilities.
cloud
Package cloud wraps AWS API.
Package cloud wraps AWS API.
ctxhandler
Package ctxhandler implements context handler.
Package ctxhandler implements context handler.
ctxutil
Package ctxutil implements context utilities.
Package ctxutil implements context utilities.
etcd-client
Package etcdclient implements etcd client utilities.
Package etcdclient implements etcd client utilities.
fileutil
Package fileutil implements file utilities.
Package fileutil implements file utilities.
github
Package github implements github utilities.
Package github implements github utilities.
httputil
Package httputil implements various HTTP operations.
Package httputil implements various HTTP operations.
k8s-client
Package k8sclient implements various k8s and EKS client utils.
Package k8sclient implements various k8s and EKS client utils.
k8s-client/eks-deprecate
Package eksdeprecate defines deprecated APIs for EKS.
Package eksdeprecate defines deprecated APIs for EKS.
k8s-object
Package k8sobject implements Kubernetes object utilities.
Package k8sobject implements Kubernetes object utilities.
logutil
Package logutil implements various log utilities.
Package logutil implements various log utilities.
metrics
Package metrics defines metrics utilities.
Package metrics defines metrics utilities.
process
Package process implements process utilities.
Package process implements process utilities.
randutil
Package randutil implements random utilities.
Package randutil implements random utilities.
spinner
Package spinner implements spinner.
Package spinner implements spinner.
terminal
Package terminal implements terminal related utilities.
Package terminal implements terminal related utilities.
timeutil
Package timeutil implements time utilities.
Package timeutil implements time utilities.
user
Package user implements system user utilities.
Package user implements system user utilities.
Package ssh implements various SSH commands.
Package ssh implements various SSH commands.
utils module
Package version defines aws-k8s-tester version.
Package version defines aws-k8s-tester version.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL