gitlab-runner-container

README

CICD GitLab Runner - Base Container

[[TOC]]

Description

A (soon to be) multi-arch docker image to be used to coordinate CICD pipelines using the Nomad Parametised Job Custom Executor. This makes use of seperate one-time-use executors which are spun up within a Nomad cluster and all commnds executed within this environment. The executor environment can be found in the GitLab Executor - Base Container repository as a part of the overarching CarbonCollins - Cloud project.

Basic Usage

These images are intended to be used by platform specific jobs which are defined in seperate repositories. These can be found:

• Linux AMD64
• Linux ARM (Planned in the future)
• Linux ARM64v8
• Darwin AMD64 (Planned in the future)

Environment Variables

This application expects certain environment variables to be presented to the docker image when starting operation. An example of which can be found in the Linux AMD64 - job.template.nomad file. but a full list is as follows:

Area	Variable	Description	Required	Default
GitLab
	CONFIG_FILE	The location on disk where the gitlab-runner will save its config file	yes	-
	CI_SERVER_URL	The URL to the GitLab instance. For GitLab.com it would be https://gitlab.com/	yes	-
Runner
	RUNNER_NAME	The name of the runner that will displayed in the GitLab UI	yes	-
	RUNNER_USER	The user in which the gitlab-runner will operate as. This is normlly gitlab-runner	yes	-
	RUNNER_WORKING_DIR	The working directory for the gitlab-runner	yes	-
	RUNNER_PLATFORM	Added to the list of runner tags to denote what platform this runner is on e.g. linux-amd64	no	-
	RUNNER_TAGS	A comma seperated list of tags that will be added to the gitlab-runner which can be used to request specific runners in jobs	no	-
	RUNNER_SOCK_ADDR	A file path to where the Unix Socket will be created for use with communicating between the GitLab Runner and this code	no	/run/gitlab-nomad-runner/runner.sock
	RUNNER_REGISTRATION_TOKEN	The GitLab Runner Registration Token	yes	-
Executor
	EXECUTOR_NOMAD_REGION	The region of the Nomad cluster which contains the executor parametised job	no	global
	EXECUTOR_NOMAD_NAMESPACE	The namespace of the Nomad clister which contains the executor parametised job	no	default
	EXECUTOR_NOMAD_JOB_ID	The paramatised job id of the executor parametised job	yes	-
Authentication
	AUTH_TYPE	Determines which authentication method will be used to authenticate against Vault and or Nomad	no	approle+nomad-backend
	APPROLE_LOGIN_PATH	The path in Vault where AppRole is mounted	yes when approle+nomad-backend	approle
	APPROLE_NOMAD_BACKEND_PATH	The path in Vault where the Nomad Backend and selected role is mounted	yes when approle+nomad-backend	-
	APPROLE_ROLE_ID	The RoleId used when logging into AppRole	yes when approle+nomad-backend	-
	APPROLE_SECRET_ID	The SecretId used when logging into AppRole. This currently can not be wrapped	yes when approle+nomad-backend	-

You can also provide configuration variables for Nomad and Vault

• Vault - Environment Variables
• Nomad - Environment Variables

The most common that would be used is to specify VAULT_ADDR and NOMAD_ADDR so that the runner knows where the Vault and Nomad instances are.

Deployments

This job does not directly deploy to the cluster, however, this repo does trigger builds in all of the downstream runner repositories which in turn deploy the runners for specific platforms. See Basic Usage for a list of downstream executors

Authentication

The runners have configurable authentication which currently only supports one method:

• approle+nomad-backend This will authenticate to Vault using Vault - AppRole to get a Vault token and then use that token to obtain a Nomad token for each job using Vault - Nomad Backend.

If you want to use this code and would like a different authentication method raise an issue.

Executables & Entrypoints

There should be two final executables produced from this code which are as following:

runner

The runner is the main executable which should be called directly by the docker image, its esentially a wrapper arround the GitLab - Runner binary and is responsible for registering, running, and unregistering the runner from GitLab itself. This binary is also responsible for renewing any Vault tokens, listening for requests on a Unix Socket from the executor, and dispatching the downstream executor job from those incomming requests. The source for this is located under cmd/runner/main.go

executor

The executor is intended to be called by the gitlab-runner itself as an interface for the custom executor. This serves as a wrapper for the Unix socket created by the runner. It is done like this due to the limitations of custom executors which must call a script on disk to operate. The source for this is located under cmd/executor/main.go

Other Info

I am currently running this in my homelab as an experiment and as a learning exercise, so due to this I wouldn’t recommend it for any real production loads... It can be used as an example of one way of doing this :D