Sillyproxy is a very simple proxy server that is configured by environment variables. It can be used to demonstrate rolling out edge configuration using Services in Docker Swarm Mode. This is useful for managing canary and blue-green deployments.
Similar production setups can be setup using bash scripts or other templating systems to leverage rolling updates in Services using your favorite proxy system.
Let's go through an example. In this example, we will setup a backend application, with multilpe versions and an edge proxy. The edge proxy will make different application version available on the same port, dividing traffic between two services based on configured weight.
All the examples here were done with Docker for Mac on a single node. These can all work without modification using Docker 1.12+ with any number nodes.
For the purposes of this example, we'll be hitting
localhost but that could
be replaced with the address of any node in your swarm.
For more complex setup, please consult the Docker documentation.
Initialize the swarm cluster
Make sure that your docker instance is in Swarm-mode. For a single node, the command is simply the following:
$ docker swarm init Swarm initialized: current node (2wr3rr195xt1bvr7cuwrw3mp8) is now a manager. To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-343oklx5y9zqwc129p4ttvgxeeexh49vpgslavcnqzipjmp2n4-0375hm0z4nto02l8jbt08ga1n \ 192.168.65.2:2377 To add a manager to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-343oklx5y9zqwc129p4ttvgxeeexh49vpgslavcnqzipjmp2n4-bnfuijeolp487mllvj7l9sjil \ 192.168.65.2:2377
Creating the Network
We'll need a backend network for our edge and backend services to communicate on. This traffic will run on an overlay network and only be accessible to other services attached to the same network.
$ docker network create --driver overlay backend 6u9nez4oadt63nadfxgbvwmss
Deploy our Service
We are now ready to deploy the first version of our service. First, we build
myapp from the directory
myapp is a simple web service
that displays the container's hostname. It also displays the contents of an
unknown, if the header is not set.
All commands are done from the sillyproxy root.
Let's start by building the image:
$ docker build -t myapp myapp/ Sending build context to Docker daemon 3.584 kB Step 1 : FROM golang:1.7-alpine ---> 52493611af1e Step 2 : COPY . /go/src/github.com/stevvooe/sillyproxy/myapp ---> 489c65a768e7 Removing intermediate container 3195d1e76f24 Step 3 : RUN go install github.com/stevvooe/sillyproxy/myapp ---> Running in 08b2e8a0bef0 ---> edf64a6a1894 Removing intermediate container 08b2e8a0bef0 Step 4 : EXPOSE 8080 ---> Running in 47c58faeb679 ---> 3f83e0bf1717 Removing intermediate container 47c58faeb679 Step 5 : ENTRYPOINT /go/bin/myapp ---> Running in 77b163b18105 ---> 34436cbe5b50 Removing intermediate container 77b163b18105 Successfully built 34436cbe5b50
Once the image is built, we are ready to run our first service. We are going to bind it to the external port 7999 for canary testing, but that is not necessary. We also attach it to the backend network so we can use it with our proxy later.
$ docker service create --name myapp-v1 --network backend -p7999:8080 myapp ekfaa2wfbxj2j22l6gay0iqoa
We can use
docker service ls to confirm it is running:
$ docker service ls ID NAME REPLICAS IMAGE COMMAND ekfaa2wfbxj2 myapp-v1 1/1 myapp
We can also see that we get the expected result by curling the service endpoint:
$ curl http://localhost:7999 unknown e99a62b52d1d
Using the Proxy
While the above could be used to scale and update the service, we'd like to have a little more control over our traffic. Specifically, we'd like to be able to serve up different running versions of the service and direct different amounts of traffic to make sure we are good.
Again, from the project root, let's build an image for our proxy.
$ docker build -t sillyproxy . Sending build context to Docker daemon 480.3 kB Step 1 : FROM golang:1.7-alpine ---> 52493611af1e Step 2 : COPY . /go/src/github.com/stevvooe/sillyproxy ---> f732e09a9d28 Removing intermediate container 57b0d3368d4a Step 3 : RUN go install github.com/stevvooe/sillyproxy ---> Running in 2cac4703e808 ---> 400ed84a617a Removing intermediate container 2cac4703e808 Step 4 : ENTRYPOINT /go/bin/sillyproxy ---> Running in a2194801b359 ---> c42d98b3b82c Removing intermediate container a2194801b359 Successfully built c42d98b3b82c
With that image, we will create our proxy, directed towards our service:
$ docker service create --name proxy --network backend -p8080:8080 -eBLUE=http://myapp-v1:8080 sillyproxy 6e4pqdozk8wh7s3zou1af7g1r
It's important to note that we've exposed the service on port 8080 across the
cluster and attached the proxy to the backend network. Anything attached to
the same backend can be accessed using the name of the service as a DNS value.
myapp-v1 is also on backend, we just use the service name to configure
the URL to use for the
We can confirm this is working with curl:
$ curl -v localhost:8080/ blue e99a62b52d1d
Notice that this is the same container id from the backend, which is available
on port 7999, except that the proxy has set the
$ curl localhost:7999 master ✱ unknown e99a62b52d1d
At this point, we could remove the export of 7999 if no longer need to get to the service directly.
Scaling the Service
At this point, we are looking great for production, except that we are running a single instance. If an instance goes down or we need to spread load across a set of nodes, we can add more instances to the backend. We do this by setting the number of replicas for a service.
To see the current number of replicas, we use the
docker service ls command:
$ docker service ls ID NAME REPLICAS IMAGE COMMAND 6e4pqdozk8wh proxy 1/1 sillyproxy ekfaa2wfbxj2 myapp-v1 1/1 myapp
For our use case, we'll need two proxy instances and four backends. We do this
docker service scale command:
$ docker service scale myapp-v1=4 proxy=2
After the new containers are started, you'll see the following:
$ docker service ls ID NAME REPLICAS IMAGE COMMAND 6e4pqdozk8wh proxy 2/2 sillyproxy ekfaa2wfbxj2 myapp-v1 4/4 myapp
Using curl to hit the proxy, we can see we get four different backends:
$ curl localhost:8080 blue 7f5d3eacb82d $ curl localhost:8080 blue 7f5d3eacb82d $ curl localhost:8080 blue 788e2a92dbce $ curl localhost:8080 blue 788e2a92dbce $ curl localhost:8080 blue 7f973b247aff $ curl localhost:8080 blue 7f973b247aff $ curl localhost:8080 blue e99a62b52d1d
By using the service name
myapp-v1, that we configured when we created the
proxy service, docker will route connections to all the backends available as
the service scales. This uses a linux kernel feature called IPVS and a gossip
network to notify peers of the locations for running replicas of a service. All
we've told the proxy to do is hit
http://localhost:8080 and docker is doing
the rest. Even though we are hitting
localhost:8080 for the proxy, those
connections are also being load balanced between the two instances of the
The same result can be achieved by hitting the service directly on port 7999,
expect that the
Color value will be unknown.
Deploying a new version
At this point, we'd like to deploy a new version of our service.
myapp has a
feature switch that will display the container id in HTML rather than plain
text. This is activated by the environment variable
V2 but this could just as
well be another image.
Let's create the new service and confirm it is running:
$ docker service create --name myapp-v2 --network backend -e V2=1 -p7998:8080 myapp f5l4xujsn904uhu2xv229rgez $ docker service ls ID NAME REPLICAS IMAGE COMMAND 6e4pqdozk8wh proxy 2/2 sillyproxy ekfaa2wfbxj2 myapp-v1 4/4 myapp f5l4xujsn904 myapp-v2 1/1 myapp
We've made the new service accessible on port 7998 for testing. Let's go ahead
and hit with
~/g/s/g/s/sillyproxy ❯❯❯ curl localhost:7998 master ✱ <h1>be0dc0efaba2 (unknown)</h1>
Clearly, production ready. Let's add this to
sillyproxy, but let's be
conservative and only route 20% of traffic to the new version. We do this using
docker service update command with additional environment variables:
docker service update --env-add GREEN=http://myapp-v2:8080/ --env-add GREEN_WEIGHT=1 --env-add BLUE_WEIGHT=4 proxy
GREEN, with a weight of 1, and modify
BLUE to have a weight of 4.
The above kicks off a rolling update of the service with new environment
Any proxy can be setup to coordinate this setup using their own weight system.
curl, we can see that certain requests receive HTML, rather than plain
$ curl localhost:8080 blue e99a62b52d1d $ curl localhost:8080 blue 7f5d3eacb82d $ curl localhost:8080 blue 788e2a92dbce $ curl localhost:8080 <h1>be0dc0efaba2 (green)</h1> $ curl localhost:8080 blue 7f973b247aff $ curl localhost:8080 <h1>be0dc0efaba2 (green)</h1> $ curl localhost:8080 blue e99a62b52d1d
Moving to Green
People are wild about HTML! UX testing has confirmed and we are ready to go
GREEN across the board.
Before we do this, let's scale the
GREEN backend, to
$ docker service scale myapp-v2=4 myapp-v2 scaled to 4
Now, that we are ready, we just kick off another rolling update to go full green:
$ docker service update --env-rm GREEN_WEIGHT --env-rm BLUE_WEIGHT --env-rm BLUE proxy proxy
docker service ls to monitor the deployment:
$ docker service ls ID NAME REPLICAS IMAGE COMMAND 6e4pqdozk8wh proxy 2/2 sillyproxy ekfaa2wfbxj2 myapp-v1 4/4 myapp f5l4xujsn904 myapp-v2 4/4 myapp
Hitting the endpoint, we can see that we now only hit the
GREEN backend, with
the HTML output, load balanced among the
$ curl localhost:8080 <h1>583ef5bfc936 (green)</h1> $ curl localhost:8080 <h1>be5901cf1337 (green)</h1> $ curl localhost:8080 <h1>bb907389b5ea (green)</h1>
Well, it turns out that HTML is costing a massive amount of bandwidth and we cannot it afford the extra traffic. We have to rollback.
Luckily, this is just another rolling update:
$ docker service update --env-rm GREEN --env-add BLUE=http://myapp-v1:8080 proxy proxy
And we are back to plain text:
$ curl localhost:8080 blue e99a62b52d1d
The above demonstrates the basics of blue-green deployment. There are many other things one may want in a production system that are not covered here, but hopefully on can extrapolate more complex setups from this.
The key takeaway from this is to use Docker Services with rolling updates to control the proxy configuration. Anything that can be updated on a service can drive a configuration push for the proxy. This is not limited to environment includes image, volumes, bind mounts or anything else.
This could also be combined with templating and key-value stores to employ your
favorite proxy, using the
docker service update command to control the point
at which your new service version goes live.
There is no documentation for this package.