cali

package module
v0.0.0-...-78327ce Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 27, 2018 License: MIT Imports: 23 Imported by: 0

README

Cali - ComposAble deveLopment and cI

IMPORTANT

This repository is no longer actively maintained. Please migrate over to https://github.com/skybet/cali.

Overview

Ever wanted to be able to ensure that all your developers are working using the same versions of the same tools?

Want to be free from dowloading huge Vagrant boxes?

Want to be able to update and distribute new versions of your tooling quickly and easily?

The answer to all these questions at our company was a resounding YES!

We needed a way to reduce the overheads associated with onboarding new starters, so instead of a document giving a list of software that needed to be downloaded, e.g. ChefDK, Vagrant, Terraform etc. etc. they could just download a single thing which gave them a ready made development environment whether they used Mac or Linux PCs. Vagrant was the obvious choice and for a while it worked, but as we scaled up as a tech team both in numbers and in diversity of tools, this quickly became unmanageable. Different teams started wanting different boxes with different thing and different versions of those things. We soon ended up with several huge box files to maintain, they were slow to update, slow to upload and slow to download and horrible to maintain. We spent a lot of time troubleshooting...

"Oh yes I have seen that error before, are you using Vagrant version X with VirtualBox version Y?

"Err... not sure this will work with Windows sorry!"

Then along came Docker for Mac and shortly after, Docker for Windows and this opened the world of containers up to non-Linux users and gave us the ability to distribute our tools using Docker across Mac, Linux and Windows for the first time. Being a Chef house, the first experiment was to run ChefDK out of a docker container and was distributed as some bash aliases (early days!).

alias buildtools="docker run --rm -it -v \$PWD:/root/build -v /var/run/docker.sock:/var/run/docker.sock -v $HOME/.aws:/root/.aws -e \"BUILD_ROOT=\$PWD\" -v ~/.gitconfig:/root/.gitconfig ourprivatedockerregistry.io/buildtools:stable"
alias kitchen="buildtools kitchen"

Typing kitchen would then execute Test Kitchen for Chef against your $PWD and no matter what laptop you were using, as long as you had docker, all our devs were now using the same version of ChefDK for developing Chef code against.

Clearly this was suboptimal and as soon as more tools came along such as Terraform which wanted to be able to obtain short leased AWS credentials from HashiCorp Vault, a new solution was needed to distribute and orchestrate these containers was needed. Also, would it not be cool if that same tool could be run directly, or as a container on CI? Then your developers and your CI is using exactly the same stuff?

So we broke out those Go ninja skills and started using the Docker API to programatically manage these containers, remembering at each step that at some point in the future, this tool would itself be containerised and and would need to be able to schedule its ephemeral job containers either directly on the host its running on, or tantalisingly, on a swarm. Each tool, whether it be ChefDK or Terraform or whatever would either need to be able to work with $PWD or be able to check out some git repo and do its thing within that clone, all within containers. The final pieces of the puzzle were including auth with Hashicorp Vault to allow us to connect our corporate AD to get on-demand AWS credentials and also an update command which would self update the code when a new version was available on our internal Artifact repository.

This has been a great success and we are really happy with how it works and how little day to day support we have to provide our developers to use it. Its been such a success that we are getting requests to put functionality into it which we feel does not really belong there. Rather than saying NO! to such feature requests, we are doing the only sane thing and open sourcing the guts of the system to allow anybody to go and create their own development tools to do whatever they want.

Building a CLI tool

package main

import "github.com/adampointer/cali"

func main() {
	cli := cali.Cli("cali")
	cli.SetShort("Example CLI tool")
	cli.SetLong("A nice long description of what your tool actually does")

	terraform := cli.Command("terraform [command]")
	terraform.SetShort("Run Terraform in an ephemeral container")
	terraform.SetLong(`Starts a container for Terraform and attempts to run it against your code. There are two choices for code source; a local mount, or directly from a git repo.

Examples:

  To build the contents of the current working directory using my_account as the AWS profile from the shared credentials file on this host.
  # cali terraform plan -p my_account

  Any addtional flags sent to the terraform command come after the --, e.g.
  # cali terraform plan -- -state=environments/test/terraform.tfstate -var-file=environments/test/terraform.tfvars
  # cali terraform -- plan -out plan.out
`)
	terraform.Flags().StringP("profile", "p", "default", "Profile to use from the AWS shared credentials file")
	terraform.BindFlags()

	terraformTask := terraform.Task("hashicorp/terraform:0.9.9")
	terraformTask.SetInitFunc(func(t *cali.Task, args []string) {
		t.AddEnv("AWS_PROFILE", cli.FlagValues().GetString("profile"))
	})

	cli.Start()
}

Now you can run containerised terraform :D

$ example terraform init
Terraform initialized in an empty directory!

The directory has no Terraform configuration files. You may begin working
with Terraform immediately by creating Terraform configuration files.

Or build from a git repo...

$ example terraform plan --git git@github.com:someone/terraform_code.git --git-branch master --git-path path/to/code

API

https://github.com/adampointer/cali/blob/master/API.md

Documentation

Index

Constants

View Source 
const (
	EXIT_CODE_RUNTIME_ERROR = 1
	EXIT_CODE_API_ERROR     = 2
)

Variables

This section is empty.

Functions

func Cli 

func Cli(n string) *cli

Cli returns a brand new cli

Types

type CreateResponse 

type CreateResponse struct {
	Id             string         `json:"id"`
	Status         string         `json:"status"`
	ProgressDetail ProgressDetail `json:"progressDetail"`
	Progress       string         `json:"progress,omitempty"`
}

CreateResponse is the response from Docker API when pulling an image

type DockerClient 

type DockerClient struct {
	Cli      *client.Client
	HostConf *container.HostConfig
	NetConf  *network.NetworkingConfig
	Conf     *container.Config
	// contains filtered or unexported fields
}

DockerClient is a slimmed down implementation of the docker cli

func NewDockerClient 

func NewDockerClient() *DockerClient

NewDockerClient returns a new DockerClient initialised with the API object

func (*DockerClient) AddBind 

func (c *DockerClient) AddBind(bnd string)

AddBind adds a bind mount to the HostConfig

func (*DockerClient) AddBinds 

func (c *DockerClient) AddBinds(bnds []string)

AddBinds adds multiple bind mounts to the HostConfig

func (*DockerClient) AddEnv 

func (c *DockerClient) AddEnv(key, value string)

AddEnvs adds an environment variable to the HostConfig

func (*DockerClient) AddEnvs 

func (c *DockerClient) AddEnvs(envs []string)

AddEnvs adds multiple envs to the HostConfig

func (*DockerClient) BindFromGit 

func (c *DockerClient) BindFromGit(cfg *GitCheckoutConfig, noGit func() error) error

BindFromGit creates a data container with a git clone inside and mounts its volumes inside your app container If there is no valid Git repo set in config, the noGit callback function will be executed instead

func (*DockerClient) ContainerExists 

func (c *DockerClient) ContainerExists(name string) bool

ContainerExists determines if the container with this name exist

func (*DockerClient) DeleteContainer 

func (c *DockerClient) DeleteContainer(id string) error

DeleteContainer - Delete a container

func (*DockerClient) Git 

func (c *DockerClient) Git() *Git

Git returns a new instance

func (*DockerClient) ImageExists 

func (c *DockerClient) ImageExists(image string) bool

ImageExists determines if an image exist locally

func (*DockerClient) InitDocker 

func (c *DockerClient) InitDocker() error

Init initialises the client

func (*DockerClient) Privileged 

func (c *DockerClient) Privileged(p bool)

Privileged sets whether the container should run as privileged

func (*DockerClient) PullImage 

func (c *DockerClient) PullImage(image string) error

PullImage - Pull an image locally

func (*DockerClient) SetBinds 

func (c *DockerClient) SetBinds(bnds []string)

SetBinds sets the bind mounts in the HostConfig

func (*DockerClient) SetCmd 

func (c *DockerClient) SetCmd(cmd []string)

SetCmd sets the command to run in the container

func (*DockerClient) SetConf 

func (c *DockerClient) SetConf(co *container.Config)

SetConf sets the container.Config struct for the new container

func (*DockerClient) SetDefaults 

func (c *DockerClient) SetDefaults()

SetDefaults sets container, host and net configs to defaults. Called when instantiating a new client or can be called manually at any time to reset API configs back to empty defaults

func (*DockerClient) SetEnvs 

func (c *DockerClient) SetEnvs(envs []string)

SetEnvs sets the environment variables in the Conf

func (*DockerClient) SetHostConf 

func (c *DockerClient) SetHostConf(h *container.HostConfig)

SetHostConf sets the container.HostConfig struct for the new container

func (*DockerClient) SetImage 

func (c *DockerClient) SetImage(img string)

SetImage sets the image in Conf

func (*DockerClient) SetNetConf 

func (c *DockerClient) SetNetConf(n *network.NetworkingConfig)

SetNetConf sets the network.NetworkingConfig struct for the new container

func (*DockerClient) SetWorkDir 

func (c *DockerClient) SetWorkDir(wd string)

SetWorkDir sets the working directory of the container

func (*DockerClient) StartContainer 

func (c *DockerClient) StartContainer(rm bool, name string) (string, error)

StartContainer will create and start a container with logs and optional cleanup

type Event 

type Event struct {
	Id     string `json:"id"`
	Status string `json:"status"`
}

Event holds the json structure for Docker API events

type Git 

type Git struct {
	Image string
	// contains filtered or unexported fields
}

Git is used to interact with containerised git

func (*Git) Checkout 

func (g *Git) Checkout(cfg *GitCheckoutConfig) (string, error)

GitCheckout will create and start a container, checkout repo and leave container stopped so volume can be imported

func (*Git) Pull 

func (g *Git) Pull(name string) (string, error)

type GitCheckoutConfig 

type GitCheckoutConfig struct {
	Repo, Branch, RelPath, Image string
}

GitCheckoutConfig is input for Git.Checkout

type ProgressDetail 

type ProgressDetail struct {
	Current int `json:"current,omitempty"`
	Total   int `json:"total,omitempty"`
}

ProgressDetail records the progress achieved downloading an image

type Task 

type Task struct {
	*DockerClient
	// contains filtered or unexported fields
}

Task is the action performed when it's parent command is run

func (*Task) Bind 

func (t *Task) Bind(src, dst string) (string, error)

Bind is a utility function which will return the correctly formatted string when given a source and destination directory

The ~ symbol and relative paths will be correctly expanded depending on the host OS

func (*Task) SetDefaults 

func (t *Task) SetDefaults(args []string) error

SetDefaults sets the default host config for a task container Mounts the PWD to /tmp/workspace Mounts your ~/.aws directory to /root - change this if your image runs as a non-root user Sets /tmp/workspace as the workdir Configures git

func (*Task) SetFunc 

func (t *Task) SetFunc(f TaskFunc)

SetFunc sets the TaskFunc which is run when the parent command is run if this is left unset, the defaultTaskFunc will be executed instead

func (*Task) SetInitFunc 

func (t *Task) SetInitFunc(f TaskFunc)

SetInitFunc sets the TaskFunc which is executed before the main TaskFunc. It's pupose is to do any setup of the DockerClient which depends on command line args for example

type TaskFunc 

type TaskFunc func(t *Task, args []string)

TaskFunc is a function executed by a Task when the command the Task belongs to is run

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.