README
¶
💝 TFCW - Terraform Cloud Wrapper
Terraform Cloud Wrapper (TFCW) wraps the Terraform Cloud API. It provides an easy way to dynamically maintain configuration and particularily sensitive variables of Terraform Cloud (TFC) workspaces.
TL;DR
Use case: You need a token or API key for your terraform provider which is stored in Vault
First, you do not have to change any of your Terraform code, although you can eventually omit the remote backend block if you want to:
// terraform.tf
terraform {
backend "remote" {
hostname = "app.terraform.io"
organization = "acme"
workspaces {
name = "foo"
}
}
}
provider "cloudflare" {
version = "~> 2.0"
email = "foo@bar.com"
}
resource "cloudflare_zone" "example" {
zone = "example.com"
}
You need to add a new file within your Terraform folder (or anywhere you would like to store it) which can look like this:
// tfcw.hcl
envvar "CLOUDFLARE_API_TOKEN" {
vault {
address = "https://vault.acme.local"
path = "secret/cloudflare"
key = "api-token"
}
}
That's it, you now have a declarative way to ensure that your variable is picked up from Vault whenever you trigger a Terraform run, even if it is a remote operation!
// Render the variables on TFC
~$ tfcw render
INFO[] Checking workspace configuration
INFO[] Processing variables and updating their values on TFC
INFO[] Set variable 'CLOUDFLARE_API_TOKEN' (environment)
// Run terraform
~$ terraform plan
...
// Or all-in-one
~$ tfcw run create
INFO[] Checking workspace configuration
INFO[] Processing variables and updating their values on TFC
INFO[] Set variable 'CLOUDFLARE_API_TOKEN' (environment)
INFO[] Preparing plan
Terraform v0.13.5
Configuring remote state backend...
Initializing Terraform configuration...
[...]
Why should I use TFCW
It is particularily useful when you work with ephemeral secrets which need to be renewed fairly often. However, you can also get massive benefit from it if you want to have a declarative way to manage your workspaces definitions as code.
You will most likely do not need to learn a new configuration syntax as TFCW is configured using HCL files. TFCW allows you to do all that whilst continuing to only write HCL files.
Without TFCW
Assuming you are using Terraform Cloud for managing this super simple stack:
// terraform.tf
terraform {
backend "remote" {
hostname = "app.terraform.io"
organization = "acme"
workspaces {
name = "foo"
}
}
}
provider "local" {
version = "~> 1.4.0"
}
variable "credentials" {}
resource "local_file" "credentials_file" {
filename = "./credentials"
file_permissons = "0600"
content = var.credentials
}
You then need to manage the value of the credentials somehow 🤷♂️ either through a .tfvars file or straight into the Terraform Cloud workspace definition. There are many different ways of achieving this but this can however quickly become a burden to maintain, specially at scale or when using different ways to store the sensitive values of the variables.
Once done, you can trigger a terraform run, manually, through the Terraform Cloud API or using a GitOps approach. eg:
~$ terraform plan
...
With TFCW
You do not have to change any of your Terraform code:
// terraform.tf
terraform {
backend "remote" {
hostname = "app.terraform.io"
organization = "acme"
workspaces {
name = "foo"
}
}
}
provider "local" {
version = "~> 1.4.0"
}
variable "credentials" {}
resource "local_file" "credentials_file" {
filename = "./credentials"
file_permission = "0600"
content = var.credentials
}
You need to add a new file within your Terraform folder (or anywhere you would like to store it) which can look like this at a bare minimum:
// tfcw.hcl
tfvar "credentials" {
// If you do not want to update the value on each run, you can optionally set a TTL
// to let TFCW aware that it should still be valid.
ttl = "1h"
vault {
address = "https://vault.acme.local"
path = "secret/very_sensitive"
key = "data"
}
}
That's it, you now have a declarative way to ensure that your variable is picked up from Vault whenever you trigger a Terraform run, even if it is a remote operation!
~$ tfcw run create
INFO[] Checking workspace configuration
INFO[] Processing variables and updating their values on TFC
INFO[] Preparing plan
Terraform v0.13.5
Configuring remote state backend...
Initializing Terraform configuration...
2020/04/06 17:23:13 [DEBUG] Using modified User-Agent: Terraform/0.12.24 TFC/d310d4ebb1
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
local_file.foo: Refreshing state... [id=376ee705fb211bc1d753477ba5607e0c3754009b]
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# local_file.credentials_file will be created
+ resource "local_file" "credentials_file" {
+ content = "secret_value"
+ directory_permission = "0777"
+ file_permission = "0600"
+ filename = "./credentials"
+ id = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
If you would prefer to keep your current way of triggering the Terraform runs, you can also simply use the render command which will only update the variables in Terraform Cloud or even locally:
~$ tfcw render --help
NAME:
tfcw render - render variables values
USAGE:
tfcw render [command options] [arguments...]
OPTIONS:
--render-type value, -r value where to render to values - options are : tfc, local or disabled (default: "tfc")
--ignore-ttls render all variables, unconditionnaly of their current expirations or configured TTLs
--dry-run simulate what TFCW would do onto the TFC API
You can also do dry runs if you want to get insights about what tfcw would actually do.
~$ tfcw render --dry-run
INFO[] Checking workspace configuration
INFO[] Processing variables and updating their values on TFC
INFO[] [DRY-RUN] Set variable credentials - (terraform) : x********x
Configuration syntax
The configuration syntax is maintained here.
Examples
Several examples are available in the docs/examples folder of this repository.
Supported sources for storing variable values
We currently support 6 sources as variable storage backends (2 natively and 4 others through s5 payloads):
Natively supported
- Vault (of course!)
- Environment variables
Through S5 payloads
- AES - GCM (using hexadecimal keys >= 128b)
- AWS - KMS
- GCP - KMS
- PGP
- (Vault is also supported through S5)
Usage
~$ tfcw --help
NAME:
tfcw - Terraform Cloud Wrapper
USAGE:
tfcw [global options] command [command options] [arguments...]
COMMANDS:
render render variables values
run manipulate runs
workspace, ws manipulate the workspace
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--address address, -a address address to access Terraform Cloud API [$TFCW_ADDRESS]
--config-file path, -c path path of a readable TFCW configuration file (.hcl or .json) (default: "<working-dir>/tfcw.hcl") [$TFCW_CONFIG_FILE]
--log-level level log level (debug,info,warn,fatal,panic) (default: "info") [$TFCW_LOG_LEVEL]
--log-format format log format (json,text) (default: "text") [$TFCW_LOG_FORMAT]
--organization organization, -o organization organization to use on Terraform Cloud API [$TFCW_ORGANIZATION]
--token token, -t token token to access Terraform Cloud API [$TFCW_TOKEN]
--working-dir path, -d path path of the directory containing your Terraform files (default: ".") [$TFCW_WORKING_DIR]
--workspace workspace, -w workspace workspace to use on Terraform Cloud API [$TFCW_WORKSPACE]
--help, -h show help
Install
Have a look onto the latest release page and pick your flavor.
Checksums are signed with the following GPG key: C09C A9F7 1C5C 988E 65E3 E5FC ADEA 38ED C46F 25BE
Go
~$ go install github.com/mvisonneau/tfcw/cmd/tfcw@latest
Homebrew
~$ brew install mvisonneau/tap/tfcw
Snapcraft
~$ snap install tfcw
Docker
~$ docker run -it --rm docker.io/mvisonneau/tfcw
~$ docker run -it --rm ghcr.io/mvisonneau/tfcw
~$ docker run -it --rm quay.io/mvisonneau/tfcw
Scoop
~$ scoop bucket add https://github.com/mvisonneau/scoops
~$ scoop install tfcw
Binaries, DEB and RPM packages
For the following ones, you need to know which version you want to install, to fetch the latest available :
~$ export TFCW_VERSION=$(curl -s "https://api.github.com/repos/mvisonneau/tfcw/releases/latest" | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')
# Binary (eg: freebsd/amd64)
~$ wget https://github.com/mvisonneau/tfcw/releases/download/${TFCW_VERSION}/tfcw_${TFCW_VERSION}_freebsd_amd64.tar.gz
~$ tar zxvf tfcw_${TFCW_VERSION}_freebsd_amd64.tar.gz -C /usr/local/bin
# DEB package (eg: linux/386)
~$ wget https://github.com/mvisonneau/tfcw/releases/download/${TFCW_VERSION}/tfcw_${TFCW_VERSION}_linux_386.deb
~$ dpkg -i tfcw_${TFCW_VERSION}_linux_386.deb
# RPM package (eg: linux/arm64)
~$ wget https://github.com/mvisonneau/tfcw/releases/download/${TFCW_VERSION}/tfcw_${TFCW_VERSION}_linux_arm64.rpm
~$ rpm -ivh tfcw_${TFCW_VERSION}_linux_arm64.rpm
Troubleshoot
You can use the --log-level debug flag in order to troubleshoot
~$ tfcw --log-level debug run create
INFO[] Checking workspace configuration
INFO[] Processing variables and updating their values on TFC
DEBU[] workspace id for foo: ws-wzzmTai00qifQAxB
INFO[] Set variable credentials (terraform)
INFO[] Preparing plan
DEBU[] Workspace id for foo: ws-wzzmTai00qifQAxB
DEBU[] Configured working directory:
DEBU[] Creating configuration version..
DEBU[] Configuration version ID: cv-6qwJz000vLCx5ktH
DEBU[] Uploading configuration version..
DEBU[] Uploaded configuration version!
INFO[] Run ID: run-Uo1C0000uvMcacBg
DEBU[] Plan ID: plan-xF1C0000EiFatd65
Terraform v0.13.5
Configuring remote state backend...
Initializing Terraform configuration...
[DEBUG] Using modified User-Agent: Terraform/0.12.24 TFC/d310d4ebb1
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
[...]
DEBU[2020-02-18T17:48:27Z] Discarding run ID: run-Uo1C0000uvMcacBg
DEBU[2020-02-18T17:48:27Z] Executed in 29.874019206s, exiting..
Perform local Terraform runs
Sometimes you can find it useful to be able to run Terraform locally instead of TFC in order to troubleshoot or plan/apply your changes in a quicker fashion. Adding the following function to your bashrc or zshrc could help you dynamically reprogramming your workspace in order to try your changes locally, using your regular terraform binaries.
tfcw-local () {
tfcw workspace operations disable > /dev/null
tfcw render --render-type local > /dev/null
[[ "$?" == "0" ]] && source tfcw.env
terraform "$@"
}
Of course, this config is quite opinionated and tailored to specific needs so feel free to amend it as you need!
Develop / Test
~$ make build-local
~$ ./tfcw
Build / Release
If you want to build and/or release your own version of tfcw, you need the following prerequisites :
~$ git clone git@github.com:mvisonneau/tfcw.git && cd tfcw
# Build the binaries locally
~$ make build-local
# Build the binaries and release them (you will need a GITHUB_TOKEN and to reconfigure .goreleaser.yml)
~$ make release
Contribute
Contributions are more than welcome! Feel free to submit a PR.
Directories