awless-templates

README

Twitter | Wiki | Changelog

awless templates

Repository to collect official, verified and runnable templates for the awless CLI

Here are some non exhaustive Examples of what you can do with templates. You can also read more about awless templates

Continuous Integration

On each change all templates are verified & compiled against the latest version of awless.

You can run the verification locally with:

go get github.com/wallix/awless  # if needed
go test -v

Examples

• ECS Autoscaling Cluster
• Awless readonly group
• Pre-defined policies for awless users
• Awless readwrite group
• Create a simple (insecure) CockroachDB cluster
• Create a simple postgres instance
• Group of instances scaling with CPU consumption
• Install awless scheduler
• Create an instance accessible with ssh with a new keypair
• Create an instance with preinstalled awless with completion
• Create an instance with preconfigured awless and awless-scheduler
• Create an instance with tags and public IP
• Create a classic Kafka infra
• Create VPC with a Linux host bastion
• Attach usual readonly AWS policies (set of permissions) on group
• Create a public network enabling routing from the Internet
• Create a AWS role with usual readonly policies that applies on a resource
• Create a AWS role with usual readonly policies that applies on a user
• Create a static website on S3
• Simple wordpress deployment
• Upload Image from local file
• Create a user with its SDK/Shell access key and console password
• Create a VPC with its internet routing gateway
• Create a VPC with 3 internal subnets
• Highly-available wordpress behind a loadbalancer, with a RDS database

ECS Autoscaling Cluster

-> Minimal awless version required: v0.1.1

Note that the AMI in this template is working only in eu-west-1 region

tags: autoscaling, container, infra

First, create the ECS cluster with awless create containercluster name={cluster.name}. Then, create a policy to allow to connect to ECS

policy = create policy name=AWSEC2ContainerServiceforEC2Role effect=Allow resource="*" description="Access for ECS containers" action=ecs:DeregisterContainerInstance,ecs:DiscoverPollEndpoint,ecs:Poll,ecs:RegisterContainerInstance,ecs:StartTelemetrySession,ecs:Submit*,ecr:GetAuthorizationToken,ecr:BatchCheckLayerAvailability,ecr:GetDownloadUrlForLayer,ecr:BatchGetImage,logs:CreateLogStream,logs:PutLogEvent

Set role name variable

roleName = AWSEC2ContainerServiceRole

Create a AWS role that applies on a resource

create role name=$roleName principal-service="ec2.amazonaws.com" sleep-after=15

Attach the policy to the role

attach policy arn=$policy role=$roleName

Create the ECS instances launch configuration. The instances must be launched with a userdata file containing:

!/bin/bash
echo ECS_CLUSTER=ecs-cluster-name >> /etc/ecs/ecs.config

launchconfig = create launchconfiguration image=ami-95f8d2f3 keypair={instance.keypair} name=ECSClusterLaunchconfig type={instance.type} userdata={instance.userdata} role=$roleName

Create the scalinggroup

create scalinggroup desired-capacity={scalinggroup.desired-capacity} launchconfiguration=$launchconfig max-size={scalinggroup.desired-capacity} min-size={scalinggroup.desired-capacity} name=ecsClusterScalingGroup subnets={instance.subnets}

Run it locally with: awless run repo:ECS_autoscaling_cluster -v

Awless readonly group

Here we define a group that allow users in that group to use the awless CLI in a readonly mode (i.e. sync, listing).

Create group name variable:

groupName = AwlessReadOnlyPermissionsGroup

Create the group:

create group name=$groupName

Attach corresponding readonly AWS policies (set of permissions) on group related to the awless services:

attach policy arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonSNSReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonSQSReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonVPCReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AutoScalingReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/IAMReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonRoute53ReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AWSLambdaReadOnlyAccess group=$groupName

Run it locally with: awless run repo:awless_readonly_group -v

Pre-defined policies for awless users

-> Minimal awless version required: v0.1.1

Useful pre-defined readonly & readwrite policies for awless users

Infra resources

create policy name=AwlessInfraReadonlyPolicy effect=Allow resource="*" description="Readonly access to infra resources" action=ec2:Describe*,autoscaling:Describe*,elasticloadbalancing:Describe*

Access resources

create policy name=AwlessAccessReadonlyPolicy effect=Allow resource="*" description="Readonly access to access resources" action=iam:GenerateCredentialReport,iam:GenerateServiceLastAccessedDetails,iam:Get*,iam:List*,sts:Get*

Storage resources

create policy name=AwlessStorageReadonlyPolicy effect=Allow resource="*" description="Readonly access to storage resources" action=s3:Get*,s3:List*

Messaging resources

create policy name=AwlessMessagingReadonlyPolicy effect=Allow resource="*" description="Readonly access to notification and queueing for messaging resources" action=sns:GetTopicAttributes,sns:List*,sqs:GetQueueAttributes,sqs:ListQueues

Lambda resources

create policy name=AwlessLambdaReadonlyPolicy effect=Allow resource="*" description="Readonly access to lambda resources" action=cloudwatch:Describe*,cloudwatch:Get*,cloudwatch:List*,cognito-identity:ListIdentityPools,cognito-sync:GetCognitoEvents,dynamodb:BatchGetItem,dynamodb:DescribeStream,dynamodb:DescribeTable,dynamodb:GetItem,dynamodb:ListStreams,dynamodb:ListTables,dynamodb:Query,dynamodb:Scan,events:List*,events:Describe*,iam:ListRoles,kinesis:DescribeStream,kinesis:ListStreams,lambda:List*,lambda:Get*,logs:DescribeMetricFilters,logs:GetLogEvents,logs:DescribeLogGroups,logs:DescribeLogStreams,s3:Get*,s3:List*,sns:ListTopics,sns:ListSubscriptions,sns:ListSubscriptionsByTopic,sqs:ListQueues,tag:GetResources,kms:ListAliases,ec2:DescribeVpcs,ec2:DescribeSubnets,ec2:DescribeSecurityGroups,iot:GetTopicRules,iot:ListTopicRules,iot:ListPolicies,iot:ListThings,iot:DescribeEndpoint

DNS resources

create policy name=AwlessDNSReadonlyPolicy effect=Allow resource="*" description="Readonly access to DNS resources" action=route53:Get*,route53:List*,route53:TestDNSAnswer,route53domains:Get*,route53domains:List*

Monitoring resources

create policy name=AwlessMonitoringReadonlyPolicy effect=Allow resource="*" description="Readonly access to monitoring resources" action=autoscaling:Describe*,cloudwatch:Describe*,cloudwatch:Get*,cloudwatch:List*,logs:Get*,logs:Describe*,logs:TestMetricFilter,sns:Get*,sns:List*

CDN resources

create policy name=AwlessCDNReadonlyPolicy effect=Allow resource="*" description="Readonly access to CDN resources" action=acm:ListCertificates,cloudfront:Get*,cloudfront:List*,iam:ListServerCertificates,route53:List*,waf:ListWebACLs,waf:GetWebACL

Cloud formation resources

create policy name=AwlessCloudFormationReadonlyPolicy effect=Allow resource="*" description="Readonly access to CloudFormation resources" action=cloudformation:DescribeStacks,cloudformation:DescribeStackEvents,cloudformation:DescribeStackResource,cloudformation:DescribeStackResources,cloudformation:GetTemplate,cloudformation:List*

Run it locally with: awless run repo:awless_readonly_policies -v

Awless readwrite group

Here we define a group that allow users in that group to use the awless CLI in write mode.

Create group name variable:

groupName = AwlessReadWritePermissionsGroup

Create the group:

create group name=$groupName

Attach corresponding AWS policies (set of permissions) on group related to the awless services:

attach policy arn=arn:aws:iam::aws:policy/AmazonEC2FullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonS3FullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonSNSFullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonSQSFullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonVPCFullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AutoScalingFullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonRDSFullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonRoute53FullAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AWSLambdaFullAccess group=$groupName

Note that we keep the IAM access readonly

attach policy arn=arn:aws:iam::aws:policy/IAMReadOnlyAccess group=$groupName

Run it locally with: awless run repo:awless_readwrite_group -v

Create a simple (insecure) CockroachDB cluster

Deploying an insecure multi-node CockroachDB cluster with HAProxy to distribute client traffic. See https://www.cockroachlabs.com/docs/manual-deployment-insecure.html

Create a new VPC with 3 private subnets and a public one (i.e. with internet gateway)

vpc = create vpc cidr=10.0.0.0/16 name=vpc_10.0.0.0_16
gateway = create internetgateway
attach internetgateway id=$gateway vpc=$vpc

Private subnets in multi AZ for cockroachdb cluster

subnet_1 = create subnet cidr=10.0.0.0/24 vpc=$vpc name=sub_10.0.0.0_24 availabilityzone={zone1}
subnet_2 = create subnet cidr=10.0.1.0/24 vpc=$vpc name=sub_10.0.1.0_24 availabilityzone={zone2}
subnet_3 = create subnet cidr=10.0.2.0/24 vpc=$vpc name=sub_10.0.2.0_24 availabilityzone={zone3}

Public subnet that will hosts the HAproxy

subnet_4 = create subnet cidr=10.0.3.0/24 vpc=$vpc name=sub_10.0.3.0_24 availabilityzone={zone1}
update subnet id=$subnet_4 public=true

Routing to attach internet gateway to public subnet

igw_rtable = create routetable vpc=$vpc
attach routetable id=$igw_rtable subnet=$subnet_4
create route cidr=0.0.0.0/0 gateway=$gateway table=$igw_rtable

Public IP for a NAT Gateway

pubip = create elasticip domain=vpc

Add a NAT Gateway to the public subnet

natgw = create natgateway elasticip-id=$pubip subnet=$subnet_4

Wait for the NAT Gateway

check natgateway id=$natgw state=available timeout=180

Routing to attach nat gateway to private subnets

natgw_rtable = create routetable vpc=$vpc
attach routetable id=$natgw_rtable subnet=$subnet_1
attach routetable id=$natgw_rtable subnet=$subnet_2
attach routetable id=$natgw_rtable subnet=$subnet_3
create route cidr=0.0.0.0/0 gateway=$natgw table=$natgw_rtable

Create firewall for SSH access

sshfirewall = create securitygroup vpc=$vpc description=ssh-access name=AccessSSH
update securitygroup id=$sshfirewall inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=22

Create firewall for cockroachdb node TCP access

nodefirewall = create securitygroup vpc=$vpc description=cockroachdb-node-access name=CockroachDBAccess
update securitygroup id=$nodefirewall inbound=authorize protocol=tcp cidr=10.0.0.0/16 portrange=26257

Create firewall for UI HTTP access

uifirewall = create securitygroup vpc=$vpc description=cockroachdb-ui-access name=CockroachUIAccess
update securitygroup id=$uifirewall inbound=authorize protocol=tcp cidr=10.0.0.0/16 portrange=8080

Create firewall for HTTP access

proxyfirewall = create securitygroup vpc=$vpc description=proxy-http-access name=ProxyAccess
update securitygroup id=$proxyfirewall inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=80
update securitygroup id=$proxyfirewall inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=443

Create a role with policy for ec2 resources so that an instance can list other instances using a local awless

create role name=DiscoverCockroachNodeRole principal-service="ec2.amazonaws.com" sleep-after=20
attach policy role=DiscoverCockroachNodeRole arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess

Create the cockroachdb nodes

node_1 = create instance subnet=$subnet_1 keypair={ssh.keypair} image={cockroachnode.ubuntu.ami} type=t2.medium count=1 role=DiscoverCockroachNodeRole name=cockroachdb-node-1 securitygroup=$sshfirewall userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/ubuntu/cockroach_insecure_node.sh
check instance id=$node_1 state=running timeout=180
node_2 = create instance subnet=$subnet_2 keypair={ssh.keypair} image={cockroachnode.ubuntu.ami} type=t2.medium count=1 role=DiscoverCockroachNodeRole name=cockroachdb-node-2 securitygroup=$sshfirewall userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/ubuntu/joining_cockroach_insecure_node.sh
check instance id=$node_2 state=running timeout=180
node_3 = create instance subnet=$subnet_3 keypair={ssh.keypair} image={cockroachnode.ubuntu.ami} type=t2.medium count=1 role=DiscoverCockroachNodeRole name=cockroachdb-node-3 securitygroup=$sshfirewall userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/ubuntu/joining_cockroach_insecure_node.sh
check instance id=$node_3 state=running timeout=180
haproxy = create instance subnet=$subnet_4 keypair={ssh.keypair} image={cockroachnode.ubuntu.ami} type=t2.micro count=1 role=DiscoverCockroachNodeRole name=cockroachdb-haproxy securitygroup=$sshfirewall userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/ubuntu/cockroach_haproxy.sh

Update instances with firewall definitions

attach securitygroup id=$proxyfirewall instance=$haproxy
attach securitygroup id=$nodefirewall instance=$haproxy
attach securitygroup id=$nodefirewall instance=$node_1
attach securitygroup id=$nodefirewall instance=$node_2
attach securitygroup id=$nodefirewall instance=$node_3
attach securitygroup id=$uifirewall instance=$node_1
attach securitygroup id=$uifirewall instance=$node_2
attach securitygroup id=$uifirewall instance=$node_3

Run it locally with: awless run repo:cockroach_insecure_cluster_with_haproxy -v

Full CLI example:

awless run repo:cockroach_insecure_cluster_with_haproxy cockroachnode.ubuntu.ami=$(awless search images canonical --id-only) ssh.keypair=my-ssh-keyname

Create a simple postgres instance

Create the corresponding secgroup and the basic postgres instance (into a existing dbsubnetgroup). Instance has only basic required properties filled in

postgres_sg = create securitygroup name=postgres description='Postgres access' vpc={vpc}
update securitygroup id=$postgres_sg inbound=authorize protocol=tcp portrange=5432 cidr={securitygroup.cidr}

Create the database and connect to it through: psql --host=? --port=5432 --username=? --password --dbname=?

create database engine=postgres id={database.identifier} subnetgroup={database.dbsubnetgroup}  password={password.minimum8chars} dbname={postgres.database.name} size=5 type=db.t2.small username={database.username} vpcsecuritygroups=$postgres_sg

Run it locally with: awless run repo:db_postgres -v

Group of instances scaling with CPU consumption

Create an autoscaling group of instances and watch their CPU to dynamically allocate/delete instances when needed.

tags: infra, autoscaling

Create the instances launch configuration

launchconfig = create launchconfiguration image={instance.image} keypair={instance.keypair} name=scalingLaunchConf type={instance.type}

Create the scalinggroup

create scalinggroup desired-capacity=2 launchconfiguration=$launchconfig max-size={instance.max-number} min-size={instance.min-number} name=instancesScalingGroup subnets={instance.subnets}

Create a scaling policy to add instances (scale-in) and a scaling policy to remove instances (scale-out)

adjustmentType = ChangeInCapacity
scalein = create scalingpolicy adjustment-scaling=1 adjustment-type=$adjustmentType name=policy-scaling-in scalinggroup=instancesScalingGroup
scaleout = create scalingpolicy adjustment-scaling=-1 adjustment-type=$adjustmentType name=policy-step-scaling-2 scalinggroup=instancesScalingGroup

metrics statistic functions

statFunction = Average
alarmThreshold = 75
monitoredMetric = CPUUtilization

Add a monitoring alarm to enable scalein when CPU load is above 75% during 2 * 5 min

create alarm namespace=AWS/EC2 dimensions=AutoScalingGroupName:instancesScalingGroup evaluation-periods=2 metric=$monitoredMetric name=scaleinAlarm operator=GreaterThanOrEqualToThreshold period=300 statistic-function=$statFunction threshold=$alarmThreshold
attach alarm name=scaleinAlarm action-arn=$scalein

Add a monitoring alarm to enable scaleout when CPU load is below 75% during 2 * 5 min

create alarm namespace=AWS/EC2 dimensions=AutoScalingGroupName:instancesScalingGroup evaluation-periods=2 metric=$monitoredMetric name=scaleoutAlarm operator=LessThanOrEqualToThreshold period=300 statistic-function=$statFunction threshold=$alarmThreshold
attach alarm name=scaleoutAlarm action-arn=$scaleout

Run it locally with: awless run repo:dynamic_autoscaling_watching_CPU -v

Install awless scheduler

Launch new instance running remote user data script installing awless

create instance name={instance.name} image={ubuntu.ami} type=t2.nano keypair={ssh.keypair} userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/ubuntu/install_awless_scheduler.sh role={role.name}

Run it locally with: awless run repo:install_awless_scheduler -v

Full CLI example:

awless run repo:install_awless_scheduler ubuntu.ami=$(aw search images canonical:ubuntu --id-only)

Create an instance accessible with ssh with a new keypair

tags: infra, ssh

Create a new security group for this instance

securitygroup = create securitygroup vpc={instance.vpc} description={securitygroup.description} name=ssh-from-internet

Authorize access on port 22 to instances in this security group

update securitygroup id=$securitygroup inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=22

Create a new keypair

keypair = create keypair name={keypair.name}

Create an instance in this security group accessible with the new keypair

create instance subnet={instance.subnet} image={instance.image} type={instance.type} keypair=$keypair name={instance.name} count=1 securitygroup=$securitygroup

Run it locally with: awless run repo:instance_ssh -v

Create an instance with preinstalled awless with completion

tags: infra, awless

role name variable

roleName = {awless.role-name}

Create a AWS role that applies on a resource

create role name=$roleName principal-service="ec2.amazonaws.com" sleep-after=10

Attach typical necessary awless readonly permissions to the role

attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonSNSReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonSQSReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonVPCReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AutoScalingReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/IAMReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonRoute53ReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AWSLambdaReadOnlyAccess

Launch new instance running remote user data script installing awless

create instance name=awless-commander type=t2.nano keypair={ssh.keypair} userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/install_awless.yml role=$roleName

Run it locally with: awless run repo:instance_with_awless -v

Create an instance with preconfigured awless and awless-scheduler

tags: infra, awless, awless-scheduler

Awless scheduler role variable

roleName = {awless-scheduler.role-name}

First we define a role that an EC2 instance can assume to use awless/awless-scheduler (write mode)

create role name=$roleName principal-service="ec2.amazonaws.com" sleep-after=10

Attach typical necessary awless readonly permissions to the role

attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonEC2FullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonS3FullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonSNSFullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonSQSFullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonVPCFullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AutoScalingFullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonRDSFullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonRoute53FullAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AWSLambdaFullAccess

We keep IAM on read only mode

attach policy role=$roleName arn=arn:aws:iam::aws:policy/IAMReadOnlyAccess

Launch new instance running remote user data script installing awless

create instance name=AwlessWithScheduler type=t2.nano keypair={ssh.keypair} userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/install_awless_suite.yml role=$roleName

Run it locally with: awless run repo:instance_with_awless_scheduler -v

Create an instance with tags and public IP

Create an instance with mulitple tags and attach to it an elastic IP

inst = create instance subnet={instance.subnet} image={instance.image} type={instance.type} keypair={instance.keypair} name={instance.name} securitygroup={instance.securitygroup}

Putting a tag on the instance

create tag resource=$inst key={instance.tagkey} value={instance.tagvalue}

Creating a elastic IP

pubip = create elasticip domain=vpc

Attaching the IP onto the instance

attach elasticip id=$pubip instance=$inst

Run it locally with: awless run repo:instance_with_tags_and_publicip -v

Create a classic Kafka infra

Create a classic Kafka infra: brokers, 1 zookeeper instance

Create securitygroup for SSH: opening port 22 for all IPs

sshsecgroup = create securitygroup vpc={main.vpc} description=SSHSecurityGroup name=SSHSecurityGroup
update securitygroup id=$sshsecgroup inbound=authorize protocol=tcp cidr={remote-access.cidr} portrange=22

Create securitygroup for Kafka instances (brokers & zookeeper)

kafkasecgroup = create securitygroup vpc={main.vpc} description=KafkaSecurityGroup name=KafkaSecurityGroup
update securitygroup id=$kafkasecgroup inbound=authorize protocol=tcp cidr={subnet.cidr} portrange=0-65535

Create a role with policy for ec2 resources so that an instance can list other instances using a local awless

create role name=AwlessEC2ReadonlyRole principal-service="ec2.amazonaws.com" sleep-after=20
attach policy role=AwlessEC2ReadonlyRole arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess

Create Zookeeper instance

zookeeper = create instance name=zookeeper image={redhat-ami} type={zookeeper-instance-type} keypair={keypair.name} subnet={main.subnet} securitygroup=$sshsecgroup userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/redhat/zookeeper.sh

Wait the Zookeeper instance is up and running

check instance id=$zookeeper state=running timeout=180

Create Kafka broker instances with role created above

broker_1 = create instance name=broker_1 image={redhat-ami} type={broker-instance-type} keypair={keypair.name} subnet={main.subnet} role=AwlessEC2ReadonlyRole securitygroup=$sshsecgroup userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/redhat/kafka.sh
broker_2 = create instance name=broker_2 image={redhat-ami} type={broker-instance-type} keypair={keypair.name} subnet={main.subnet} role=AwlessEC2ReadonlyRole securitygroup=$sshsecgroup userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/redhat/kafka.sh
broker_3 = create instance name=broker_3 image={redhat-ami} type={broker-instance-type} keypair={keypair.name} subnet={main.subnet} role=AwlessEC2ReadonlyRole securitygroup=$sshsecgroup userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/redhat/kafka.sh

Update instances with corresponding securitygroups

attach securitygroup id=$kafkasecgroup instance=$broker_1
attach securitygroup id=$kafkasecgroup instance=$broker_2
attach securitygroup id=$kafkasecgroup instance=$broker_3
attach securitygroup id=$kafkasecgroup instance=$zookeeper

Run it locally with: awless run repo:kafka_infra -v

Full CLI example:

awless run repo:kafka_infra redhat-ami=$(awless search images redhat --id-only) remote-access.cidr=$(awless whoami --ip-only)/32 broker-instance-type=t2.medium zookeeper-instance-type=t2.medium

Create VPC with a Linux host bastion

This template build this typical Linux bastion architecture except it only deploys one host bastion on one public subnet

tags: infra

Create a new VPC and make it public with an internet gateway

vpc = create vpc cidr=10.0.0.0/16 name=BastionVpc
gateway = create internetgateway
attach internetgateway id=$gateway vpc=$vpc

Create 2 private subnets each on a different availability zone That is where you will deploy resources only accessible through the bastion

create subnet cidr=10.0.0.0/19 name=PrivSubnet1 vpc=$vpc availabilityzone={zone1}
create subnet cidr=10.0.32.0/19 name=PrivSubnet2 vpc=$vpc availabilityzone={zone2}

Create the the public subnet hosting the bastion

pubSubnet = create subnet cidr=10.0.128.0/20 name=PubSubnet1 vpc=$vpc availabilityzone={zone1}
update subnet id=$pubSubnet public=true

Create a route table (with routing only allowed within VPC by default)

rtable = create routetable vpc=$vpc

Make the public subnet use the route table

attach routetable id=$rtable subnet=$pubSubnet
create route cidr=0.0.0.0/0 gateway=$gateway table=$rtable

Create the firewall with the remote access CIDR applied on each bastion host

bastionSecGroup = create securitygroup vpc=$vpc description=BastionSecGroup name=bastion-secgroup
update securitygroup id=$bastionSecGroup inbound=authorize protocol=tcp cidr={remoteaccess-cidr} portrange=22
update securitygroup id=$bastionSecGroup inbound=authorize protocol=icmp cidr={remoteaccess-cidr} portrange=any

Allow only a set of permitted actions for the 2 host bastions

create role name=BastionHostRole principal-service=ec2.amazonaws.com sleep-after=30
bastionEc2Policy = create policy name=BastionEc2Permissions action=ec2:DescribeAddresses,ec2:AssociateAddress resource="*" effect=Allow
attach policy role=BastionHostRole arn=$bastionEc2Policy

Create one elastic IPs for that will be dynamically aasigned to the host bastion by the bootstrap script

create elasticip domain=vpc

Create the autoscaling group

launchConfig = create launchconfiguration image={instance.image} keypair={keypair.name} securitygroups=$bastionSecGroup name=BastionHostsLaunchConfig type=t2.micro role=BastionHostRole userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/prepare_bastion.yml
create scalinggroup desired-capacity=1 launchconfiguration=$launchConfig max-size=1 min-size=1 name=autoscaling-instances-group subnets=$pubSubnet

Run it locally with: awless run repo:linux_bastion -v

Attach usual readonly AWS policies (set of permissions) on group

When you want your users to have a set of permissions, instead of attaching permissions directly on users it is a good practice and simpler to define a group having those permissions and then adding/removing as needed users from those groups.

tags: access, policy, role

groupName = {group-name}
attach policy arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonSNSReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonSQSReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonVPCReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AutoScalingReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/IAMReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess group=$groupName
attach policy arn=arn:aws:iam::aws:policy/AmazonRoute53ReadOnlyAccess group=$groupName

Run it locally with: awless run repo:policies_on_role -v

Create a public network enabling routing from the Internet

tags: infra

Create the subnet

subnet = create subnet cidr={subnet.cidr} vpc={subnet.vpc} name={subnet.name}

Allow instances in this network to have public IP addresses

update subnet id=$subnet public=true

Create a route table for this network

rtable = create routetable vpc={subnet.vpc}
attach routetable id=$rtable subnet=$subnet

Enable routing from the Internet to this subnet

create route cidr=0.0.0.0/0 gateway={vpc.internetgateway} table=$rtable

Run it locally with: awless run repo:public_subnet -v

Create a AWS role with usual readonly policies that applies on a resource

Create a AWS role that applies on a resource (retrieve the account id with awless whoami)

tags: access, policy, role

roleName = {role-name}
create role name=$roleName principal-service={aws-service}

Attach policy (set of permissions) to the created role

attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonSNSReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonSQSReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonVPCReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AutoScalingReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/IAMReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess
attach policy role=$roleName arn=arn:aws:iam::aws:policy/AmazonRoute53ReadOnlyAccess

Run it locally with: awless run repo:role_for_resource -v

Create a AWS role with usual readonly policies that applies on a user

Create a AWS role that applies on a user (retrieve the id with awless whoami)

tags: access, policy, user

accountRole = create role name={role-name} principal-account={aws-account-id}

Attach policy (set of permissions) to the created role

attach policy role={role-name} arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/AmazonSNSReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/AmazonSQSReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/AmazonVPCReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/AutoScalingReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/IAMReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess
attach policy role={role-name} arn=arn:aws:iam::aws:policy/AmazonRoute53ReadOnlyAccess

Create a policy to allow user with this policy to assume only this role You can then attach this policy to a user via awless attach policy arn=... user=jsmith

create policy name={assume-policy-name} effect=Allow action=sts:AssumeRole resource=$accountRole

Run it locally with: awless run repo:role_for_user -v

Create a static website on S3

tags: s3

Create the bucket where files will be stored

create bucket name={domain.name} acl=public-read

Publish this s3bucket as a website

update bucket name={domain.name} public-website=true redirect-hostname={domain.name}

Add files to the bucket with awless create s3object bucket={domain.name} file={input-file-path} acl=public-read

Run it locally with: awless run repo:s3website -v

Simple wordpress deployment

tags: infra

VPC and its Internet gateway

vpc = create vpc cidr=10.0.0.0/16 name=wordpress-vpc
igw = create internetgateway
attach internetgateway id=$igw vpc=$vpc

Subnet and its route table

subnet = create subnet cidr=10.0.0.0/24 vpc=$vpc name=wordpress-subnet
update subnet id=$subnet public=true
routetable = create routetable vpc=$vpc
attach routetable subnet=$subnet id=$routetable
create route cidr=0.0.0.0/0 gateway=$igw table=$routetable

Create a security group and authorize accesses from the Internet for port 22 and 80

secgroup = create securitygroup vpc=$vpc description="authorize ssh and http from internet" name=wordpress-secgroup
update securitygroup id=$secgroup inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=22
update securitygroup id=$secgroup inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=80

Create keypair and instance

keypair = create keypair name={keypair.name}
create instance name=wordpress-instance subnet=$subnet keypair=$keypair securitygroup=$secgroup userdata=https://raw.githubusercontent.com/zn3zman/AWS-WordPress-Creation/master/WP-Setup.sh

Run it locally with: awless run repo:simple_wordpress_infra -v

Upload Image from local file

This template uploads on s3 a local VM file (VHD, OVA, VMDK). Then it creates an AMI from the S3 object.

tags: infra, s3

Upload the image on s3

bucket = {image.bucket}
imageObject = create s3object bucket=$bucket file={image.filepath}

Create the AMI from the object on S3

import image description={image.description} bucket=$bucket s3object=$imageObject

Run it locally with: awless run repo:upload_image -v

Create a user with its SDK/Shell access key and console password

tags: access, user

username = {user.name}

Create user

create user name=$username

Create AWS Console password

create loginprofile username=$username password={user.console-password}

Create SDK/shell access key

create accesskey user=$username

Run it locally with: awless run repo:user -v

Create a VPC with its internet routing gateway

tags: infra, VPC

vpc = create vpc cidr={vpc.cidr} name={vpc.name}
gateway = create internetgateway
attach internetgateway id=$gateway vpc=$vpc

Run it locally with: awless run repo:vpc -v

Create a VPC with 3 internal subnets

Create a new VPC with private subnets (no internet gateway)

vpc = create vpc cidr=10.0.0.0/16 name=vpc_10.0.0.0_16
create subnet cidr=10.0.0.0/24 vpc=$vpc name=sub_10.0.0.0_24 availabilityzone={subnet1.zone}
create subnet cidr=10.0.1.0/24 vpc=$vpc name=sub_10.0.1.0_24 availabilityzone={subnet2.zone}

Run it locally with: awless run repo:vpc_with_subnets -v

Highly-available wordpress behind a loadbalancer, with a RDS database

-> Minimal awless version required: v0.1.1

tags: infra, rds, autoscaling

dbname={dbname}
dbhost={dbhost}
dbuser={dbuser}
dbpassword={dbpassword}

Create the load balancer with its security group, target group and listener

lbsecgroup = create securitygroup vpc={wordpress.vpc} description="authorize HTTP from the Internet" name=wordpress-lb-securitygroup
update securitygroup id=$lbsecgroup inbound=authorize protocol=tcp cidr=0.0.0.0/0 portrange=80
tg = create targetgroup name=wordpress-workers port=80 protocol=HTTP vpc={wordpress.vpc}
lb = create loadbalancer name=wordpress-loadbalancer subnets={wordpress.subnets} securitygroups=$lbsecgroup
create listener actiontype=forward loadbalancer=$lb port=80 protocol=HTTP targetgroup=$tg

Create the launch configuration for the instances and start it in a scaling group, to ensure having always 2 instances running

launchconf = create launchconfiguration image={instance.image} keypair={wordpress.keypair} name=wordpress-launch-configuration type=t2.micro userdata=https://raw.githubusercontent.com/wallix/awless-templates/master/userdata/install-wordpress.sh securitygroups={instances.securitygroup}
create scalinggroup desired-capacity=2 launchconfiguration=$launchconf max-size=2 min-size=2 name=wordpress-scalinggroup subnets={wordpress.subnets} targetgroups=$tg

Run it locally with: awless run repo:wordpress_ha -v