coffer

README

coffer

This command line tool is designed to simplify storage and retrieval of secrets in Amazon Web Services.

It uses the following services:

• Simple Storage Service (S3) to store secrets encrypted in files
• Key Management Service (KMS) to manage encryption keys which encrypt/decrypt your secrets

A typical use case for coffer is you have a docker container which needs to retrieve on startup some file based secrets and apply them prior to starting a service. This is quite common requirement with continuous integration agents running in docker containers.

coffer bundle format

coffer uses a a YAML file file to package a bunch of files together. The format of this file is illustrated below.

coffer has the ability to synchronise the files described in this bundle with the filesystem, creating/updating and changing the mode of the files.

files:
  "/home/user/myfile2" :
    mode: 0755
    content: |
      # this is my file
      # with content

environment

The command reads the following environment variables.

• AWS_REGION the AWS region
• AWS_PROFILE the AWS profile to use
• COFFER_ALIAS the alias name of the file in KMS
• S3_BUCKET the S3 bucket which the file will be uploaded

usage

Sub commands for this tool are:

• encrypt, this encrypts the coffer file.
• decrypt, this decrypts the coffer file, required at the moment if you want to edit it.
• upload, uploads the coffer to s3, ensuring that only encrypted data gets uploaded.
• download, pull down a coffer and validates it, file is only saved if it is decrypts and is valid.
• sync, sync a coffer with the file system, this creates/modifies/chmods files based on the information in the yaml.

example

Before you start.

• Create a bucket in S3, I suggest something like XXXX-coffers in the same region as your KMS key.
• Create a KMS key see Creating Keys with the alias coffer, note this needs to be in the same region as your S3 bucket.
• Make an IAM role in AWS for your servers permitting access to the S3 bucket and KMS key (see the IAM policy below).

Create a coffer file with some SSH keys in it.

cat > buildkite.coffer <<EOF
files:
  "/var/lib/buildkite-agent/.ssh/id_rsa":
    mode: 0600
    content: |
        -----BEGIN RSA PRIVATE KEY-----
        XXXX
        -----END RSA PRIVATE KEY-----
EOF

Encrypt and Upload the coffer file to S3.

AWS_PROFILE=XXXX AWS_REGION=us-west-2 coffer --coffer-file buildkite.coffer upload --bucket="XXXX-coffers"

IAM Role

If you want to give systems permission to access your coffer key in KMS use the following role. Note you will need to grab the ARN of your key from KMS.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": [
        "arn:aws:s3:::XXXX-coffers/*"
      ]
    },
    {
      "Sid": "Allow use of the key",
      "Effect": "Allow",
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "arn:aws:kms:us-west-2:XXXX:key/XXXX-XXXX-XXXX-XXXX-XXXX"
    }
  ]
}

KMS

You can list your key aliases using the AWS CLI.

aws --profile XXXX kms list-aliases

encryption

This now uses golang.org/x/crypto/nacl/secretbox which is a great little library designed to help people do message encryption correctly.

change log

2.0

• Changed file format, now uses YAML as a container for meta data and encrypted payload
• Added a version and name field
• Added support for KMS to remove the need for a secret

License

This code is released under the MIT license see the LICENSE.md file for more details.

Documentation

Index

• Variables
• func MustDecrypt(cofferFile, alias string) []byte
• func MustDownload(cofferFile, alias, bucket string)
• func MustDownloadSync(cofferFile, alias, bucket, base string)
• func MustEncrypt(cofferFile, alias string)
• func MustSync(cofferFile, alias, base string)
• func MustUpload(cofferFile, alias, bucket string)
• type Bundle
  • func (b *Bundle) MustValidate()
• type Coffer
  • func DecodeCoffer(data []byte) (coffer *Coffer, err error)
  • func (c *Coffer) Validate() bool
• type DataKey
• type FileData
  • func (f *FileData) MustValidate(name string)
• type KeyManagement
• type ObjectStorage

Variables

var (
	// CofferBlockSize size of the key
	CofferBlockSize = 32
	// OwnerRead is the default mode set for new coffer files, note the octal number
	OwnerRead = os.FileMode(0600)
	// Version the version of the coffer file
	Version = "2.0.0"
)

Functions

func MustDecrypt

func MustDecrypt(cofferFile, alias string) []byte

MustDecrypt decrypt the supplied file

func MustDownload

func MustDownload(cofferFile, alias, bucket string)

MustDownload download the file from the supplied s3 bucket

func MustDownloadSync

func MustDownloadSync(cofferFile, alias, bucket, base string)

MustDownloadSync download the file from the supplied s3 bucket and sync it to the filesystem

func MustEncrypt

func MustEncrypt(cofferFile, alias string)

MustEncrypt encrypt the supplied file

func MustSync

func MustSync(cofferFile, alias, base string)

MustSync sync the file up to S3

func MustUpload

func MustUpload(cofferFile, alias, bucket string)

MustUpload upload the file to the supplied s3 bucket

Types

type Bundle

type Bundle struct {
	Files map[string]*FileData `yaml:"files"`
}

Bundle bundle of files and their related information

func (*Bundle) MustValidate

func (b *Bundle) MustValidate()

MustValidate checks the validity of the bundle

type Coffer

type Coffer struct {
	Name       string `yaml:"name,omitempty"`
	Version    string `yaml:"version,omitempty"`
	Key        string `yaml:"key,omitempty"`
	CipherText string `yaml:"ct,omitempty"`
}

Coffer used as the container for an encrypted coffer

func DecodeCoffer

func DecodeCoffer(data []byte) (coffer *Coffer, err error)

DecodeCoffer decode the coffer file

func (*Coffer) Validate

func (c *Coffer) Validate() bool

Validate checks the validity of the coffer

type DataKey

type DataKey struct {
	CiphertextBlob []byte
	Plaintext      []byte
}

DataKey which contains the details of the KMS key

type FileData

type FileData struct {
	Mode    uint32 `yaml:"mode"`
	Owner   string `yaml:"owner"`
	Group   string `yaml:"group"`
	Content string `yaml:"content"`
}

FileData an encoded file with it's permissions

func (*FileData) MustValidate

func (f *FileData) MustValidate(name string)

MustValidate checks the validity of the file data structure.

type KeyManagement

type KeyManagement interface {
	GenerateDataKey(*kms.GenerateDataKeyInput) (*kms.GenerateDataKeyOutput, error)
	Decrypt(*kms.DecryptInput) (*kms.DecryptOutput, error)
}

KeyManagement is a sub-set of the capabilities of the KMS client.

type ObjectStorage

type ObjectStorage interface {
	ListObjects(*s3.ListObjectsInput) (*s3.ListObjectsOutput, error)
	DeleteObject(*s3.DeleteObjectInput) (*s3.DeleteObjectOutput, error)
	PutObject(*s3.PutObjectInput) (*s3.PutObjectOutput, error)
	GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error)
}

ObjectStorage is a sub-set of the capabilities of the S3 client.