Cuegen
Cuegen is a tool to build kubernetes resources with CUE. It solves the
problems of composing and versioning of "charts", as well as easily importing
external data (scripts, certificates, keys, etc.) into CUE charts. For this
purpose, it extends the rich possibilities that CUE already provides
with the ability to "compose" charts from various sources and load resources,
controlled by attributes.
If CUE and creating k8s resources with CUE is new to you, the
CUE homepage, the k8s tutorial, as well as examples
in this repository (in that order) are good starting points.
Table of Contents
Features
- Compose manifests from various, versioned sources (local directories, git-repositories)
- Load file contents into CUE values, e.g. a script into a ConfigMap key
- Load whole directories as key/values data at once, e.g. into a ConfigMap
- Load structured data (JSON/YAML/env) into a CUE struct
- Automatically decrypt SOPS-encrypted data
- Load remote charts
Install
Download the latest release or build with go1.20rc3 or later:
go install github.com/noris-network/cuegen@latest
Usage
cuegen path/to/cuegen.yaml
# or
cuegen path/to/directory-containing-cuegen-dot-yaml
# or
cuegen https://git.example.com/deployments/myapp.git
Have a look at the examples for some ready-to-run examples.
Cuegen can be used as config management plugin in ArgoCD. A container
with the plugin.yaml
configuration
is available at docker hub. The cuegen-cmp plugin container
expects the cuegen config file to be named cuegen.yaml
.
Configuration
A configuration file (preferred name: cuegen.cue
, schema) is
required to run cuegen
. For backwards compatibility the yaml format will still
be supported in the future.
cuegen: {
objectsPath: "objects" // this will be dumped to YAML
secretDataPath: "secret.*.data" // values matching this path will be loaded as []byte and
// automatically be base64 encoded when dumped to YAML
checkPath: // this path is checked to contain only concrete values
checkPaths: [ // when more than one path needs to be checked, the
"values", // 'objectsPath' with always be checked.
"global",
]
components: [ // merge cue files from these soures into the main entrypoint
"https://$GITLAB_TOKEN@gitlab.noris.net/mcs/components/cuegen/mongodb.git?ref=v6.0.4-mcs.1",
"https://$GITLAB_TOKEN@gitlab.noris.net/mcs/components/cuegen/wekan.git?ref=v6.71-mcs.0",
]
debug: false // print some info useful for debugging
}
Environment Variables
Some environment variables can help working with cuegen:
CUEGEN_DEBUG turn on debug output with "true"
CUEGEN_HTTP_PASSWORD password for git authentication
CUEGEN_HTTP_USERNAME username for git authentication
DUMP_OVERLAYS_TO directory to dump overlays to (debug only)
SOPS_AGE_KEY age key for decryption
SOPS_AGE_KEY_FILE age key file for decryption
YQ_PRETTYPRINT !="": run yaml output thru `yq -P`
starting with `/`: use as path to yq
Components
Components can be
- directories outside the main chart
- zip files
- git repositories
Local paths are not restricted. As this could be a security problem,
this will change in a future release.
Environment variables in components are expanded, that can be used to
e.g. add authentication to git urls or prefix local paths. For cloning private
repositories via Http url, CUEGEN_HTTP_USERNAME
and CUEGEN_HTTP_PASSWORD
can
also be set in the environment.
Components Example:
cuegen: components: [
"../database-chart",
"../common-static-resources.zip",
"https://github.com/nxcc/cuegen-test-chart-one?ref=v0.1.0",
"https://github.com/noris-network/cuegen?ref=v0.2.1#examples/configmap",
]
Because of the way CUE and cuegen work, all *.cue
files need to be in the root
of the components directory.
No special files need to be present for a chart, and although cuegen does not
require or evaluate them, it is recommended to add metadata (see examples).
For git repositories, a branch or tag name can be specified with the ref
parameter.
If a zip file has only one directory in it's root, this level is skipped and
cuegen uses all files in that directory.
Attributes
The directory containing cuegen.yaml
is the base directory, all paths are relative
to this.
Attribute "@readfile"
Attribute @readfile
loads contents of given file(s) into a CUE value. An optional
suffix controlls whitespace (=nl
: ensure \n
at end, =trim
: trim whitespace
from begin and end).
Load version.txt
into Version
, trim any whitespace from begin and end:
Version: string @readfile(version.txt=trim)
Load my.crt
and chain.crt
concatenated into Certificate
. A \n
is ensured
to be placed after my.crt
:
Certificate: string @readfile(my.crt=nl, chain.crt)
Attribute "@readmap"
Attribute @readmap
loads contents of given file(s) into a CUE value as key/value
data. When the cue path matches secretDataPath
, values are inserted as []byte.
This will automatically base64 encode them when dumped to YAML. This is useful
for Secrets. This can also be forced by an =bytes
suffix.
Load values from data.json
as []bytes:
secret: foo: {
data: {} @readmap(data.json=bytes)
}
Attribute "@read"
with file Attribute value
Attribute @read(filename)
tries to load structured data (JSON/YAML/env) into a
CUE struct.
Load key/value data from env.env
and data.yaml
into configMap.myconfig.data
:
configMap: "myconfig": {
data: {} @read(env.env, data.yaml)
}
with directory Attribute value
Attribute @read(directory)
tries to load regular files from the given directory
as key/values into a CUE struct.
Load all files from directory scripts
as key/values into configMap.scripts.data
:
configMap: "scripts": {
data: {} @read(scripts)
}
Order Workaround
Until issue 2555 is resolved in CUE, there is a temporary workaround.
Changelog
v0.1.0
- Initial release
v0.1.1
- Improved attribute lookup
v0.2.0
- Added checks when reading cuegen.yaml
v0.2.1
- Improved error messages
v0.3.0
- Added ability to read subpaths from git repos
v0.3.1
- No code changes, trigger cmp build
v0.3.2
- No code changes, bump go version to 1.20
v0.4.0
- switch default config to cuegen.cue
, use cue v0.5.0-beta5
v0.4.1
- Make components & checkPaths optional
v0.4.2
- downgrade cue to v0.5.0-beta.2 (performance regression)
v0.4.3
- fix running as kustomize plugin
v0.4.4
- improve handling of git urls
v0.5.0
- add cuegen default config
v0.6.0
- add dumpOverlays option
v0.7.0
- upgrade cue to v0.5.0 (many fixes, rare performance regression still present)
v0.7.1
- fix secret handling of @readfile
v0.7.2
- internal cleanup
v0.8.0
- allow remote cuegen directories, rm kustomize plugin support
v0.9.0
- upgrade cue to v0.6.0
v0.10.0
- add YQ_PRETTYPRINT
to filter output thru yq -P
v0.11.0
- add DUMP_OVERLAYS_TO
to dump overlays to directory (debug)
v0.11.1
- upgrade sops
v0.12.0
- add workaround for file order bug
v0.13.0
- No code changes, bump cue version to 0.7.0
v0.13.1
- bump deps
v0.14.x
- "silent release" of v1alpha2 schema...
v0.14.4
- No code changes, bump cue version to 0.8.0