yaml

README

yaml CLI

A CLI tool for transforming YAML files: Grep objects, join files, get/add/edit/delete YAML nodes based on selectors etc.

[input.yml] => [apply transformations] => [output.yml]

Note: The input.yml file might contain multiple YAML documents/objects separated by ---.

• One-liner transformations
  • yaml set "selector: value"
  • yaml default "selector: value"
  • yaml delete "selector"
  • yaml cat file1.yml file2.yml fileN.yml
• yaml get "selector"
• yaml print "selector"
• Transformation files
  • yaml apply file1.yml file2.yml fileN.yml
    • Examples of transformation YAML files
• Print selected YAML nodes
  • yaml get "selector"
    • kubectl - print pod's main container image
• Grep documents/objects
  • yaml grep "selector: value" ...
    • Grep k8s deployment object by name
  • yaml grep -v "selector: value" ...
    • Grep all k8s objects that don't create Pods
    • Print first container's image of linkerd2 deployment objects
• Useful Kubernetes examples
  • Push all non-pod objects to k8s
  • Rollout k8s deployments from desired-state files sequentially
• Known issues
• Feedback
• License

One-liner transformations

yaml set "selector: value"

# Add/overwrite field's value
$ cat input.yml | yaml set "metadata.labels.environment: staging" > output.yml

yaml default "selector: value"

# Add default value (if no such value exists yet)
$ cat input.yml | yaml default "metadata.labels.environment: staging" > output.yml

yaml delete "selector"

# Delete specific field
$ cat input.yml | yaml delete "metadata.labels.environment" > output.yml

yaml cat file1.yml file2.yml fileN.yml

# Join multiple YAML files into one, where all documents/objects are separated by `---`
$ yaml cat k8s-apps/*.yml > output.yml

yaml get "selector"

Get value of a given YAML node.

$ kubectl get pod/nats-8576dfb67-vg6v7 -o yaml | yaml get spec.containers[0].image
nats-streaming:0.10.0

yaml print "selector"

Print node of a give YAML node.

$ kubectl get pod/nats-8576dfb67-vg6v7 -o yaml | yaml print kind apiVersion metadata.annotations
kind: Pod
apiVersion: v1
metadata.annotations:
  kubernetes.io/psp: eks.privileged

Transformation files

yaml apply file1.yml file2.yml fileN.yml

$ yaml cat k8s-apps/*.yml | yaml apply staging.yml enable-linkerd.yml > staging/desired-state.yml

staging.yml:

match:
    # all YAML objects
set:
    metadata.labels.environment: staging
---
match:
    kind: Deployment
    metadata.name: api
set:
    metadata.labels.first: updated-label
    spec.replicas: 3

enable-linkerd.yml:

match:
  kind: [Deployment, Pod]
default:
  metadata.annotations:
    linkerd.io/inject: enabled

Changes applied to the original object:

 apiVersion: apps/v1
 kind: Deployment
 metadata:
     name: api
     labels:
-        first: label
+        first: updated-label
         second: label
+        environment: staging
+    annotations:
+        linkerd.io/inject: enabled
 spec:
-    replicas: 1
+    replicas: 3
 ...

Examples of transformation YAML files

match:
    kind: Deployment
set:
    spec.template.spec.containers[*]:
        imagePullPolicy: IfNotPresent

match:
    kind: Deployment
set:
    spec.template.spec.nodeSelector:
        worker: generic

match:
    kind: Deployment
    metadata.name: api
set:
    spec.replicas: 3

match:
    kind: Ingress
set:
    metadata:
        annotations:
            kubernetes.io/ingress.class: nginx

match:
    kind: Deployment
delete: spec.replicas

match:
    kind: Deployment
    spec.template.spec.containers[0].image: nats-streaming
set:
    spec.template.spec.containers[0].image: nats-streaming:0.15.1

Print selected YAML nodes

yaml get "selector"

kubectl - print pod's main container image

$ kubectl get pods/nats-8576dfb67-vg6v7 -o yaml | yaml get spec.containers[0].image
nats-streaming:0.10.0

Grep documents/objects

Grep documents/objects matching all of the given selector: value pairs.

If a value is an array (ie. selector: [first, second]), the key must match at least one of the values (a logical "OR").

yaml grep "selector: value" ...

Grep k8s deployment object by name

$ cat desired-state.yml | yaml grep "kind: Deployment" "metadata.name: linkerd"

yaml grep -v "selector: value" ...

Grep all k8s objects that don't create Pods

$ cat desired-state.yml | yaml grep -v "kind: [Deployment, Pod, Job, ReplicaSet, ReplicationController]"

Print first container's image of linkerd2 deployment objects

$ cat linkerd.yml | yaml grep "kind: Deployment" | yaml get "spec.template.spec.containers[0].image"
gcr.io/linkerd-io/controller:stable-2.4.0
gcr.io/linkerd-io/controller:stable-2.4.0
gcr.io/linkerd-io/web:stable-2.4.0
prom/prometheus:v2.10.0
gcr.io/linkerd-io/grafana:stable-2.4.0
gcr.io/linkerd-io/controller:stable-2.4.0
gcr.io/linkerd-io/controller:stable-2.4.0
gcr.io/linkerd-io/controller:stable-2.4.0

Useful Kubernetes examples

Push all non-pod objects to k8s

$ cat desired-state.yml | yaml grep -v "kind: [Deployment, Pod, Job]"

Rollout k8s deployments from desired-state files sequentially

$ for file in *.yml; do
    out=$(cat $file | yaml grep "kind: Deployment" | kubectl apply -f -)

    for deploy in $(echo "$out" | cut -d' ' -f1); do
        kubectl rollout status --timeout 180s $deploy || {
            kubectl rollout undo $deploy
            exit 1
        }
    done
  done

Known issues

1. Merging complex nodes doesn't work well

set:
    metadata:
        we:
            cant:
                merge:
                    complex: objects
                    such:
                        as: this
                        properly:
                            just: yet

We'll want to fix this later. For now, use explicit paths to the final nodes:

set:
    metadata.we.cant.merge.complex: objects
    metadata.we.cant.merge.such.as: this
    metadata.we.cant.merge.such.properly.just: yet

1. Wildcard array[*] matching doesn't work yet

match:
    spec.template.spec.containers[*].name: prometheus

1. Selectors with . dots in the selector path, ie.

metadata:
    annotations:
        linkerd.io/inject: enabled

Since the match selectors are separated with . dots, we'll have to figure out how to support these selector keys with inner . dots.

We might wanna support

delete:
    - metadata.annotations."linkerd.io/inject"
set:
    metadata.annotations."rbac.authorization.kubernetes.io/autoupdate": true

Feedback

Any feedback welcome! Please open issues and feature requests..

License

Licensed under the MIT License.

Documentation

Index

• func Bytes(in *Node) []byte
• func Delete(doc *yaml.Node, path string) error
• func Get(doc *yaml.Node, path string) (*yaml.Node, error)
• func Set(doc *yaml.Node, path string, value *yaml.Node) error
• func SetDefault(doc *yaml.Node, path string, value *yaml.Node) error
• func Write(w io.Writer, in *Node) error
• type Node
  • func Parse(in []byte) (*Node, error)
• type Transformation
  • func Transformations(r io.Reader) ([]*Transformation, error)
  • func (t *Transformation) Apply(doc *yaml.Node) error
  • func (t *Transformation) ApplyDefault(doc *yaml.Node) error
  • func (t *Transformation) ApplyDeletes(doc *yaml.Node) error
  • func (t *Transformation) ApplySet(doc *yaml.Node) error
  • func (t *Transformation) MustMatchAll(doc *yaml.Node) (bool, error)

Functions

func Bytes

func Bytes(in *Node) []byte

func Delete

func Delete(doc *yaml.Node, path string) error

func Get

func Get(doc *yaml.Node, path string) (*yaml.Node, error)

func Set

func Set(doc *yaml.Node, path string, value *yaml.Node) error

func SetDefault

func SetDefault(doc *yaml.Node, path string, value *yaml.Node) error

func Write

func Write(w io.Writer, in *Node) error

Types

type Node

type Node = yaml.Node

func Parse

func Parse(in []byte) (*Node, error)

type Transformation

type Transformation struct {
	Matches  map[string]yaml.Node `yaml:"match"`
	Sets     yaml.Node            `yaml:"set"`
	Defaults yaml.Node            `yaml:"default"`
	Deletes  []string             `yaml:"delete"`
}

func Transformations

func Transformations(r io.Reader) ([]*Transformation, error)

func (*Transformation) Apply

func (t *Transformation) Apply(doc *yaml.Node) error

func (*Transformation) ApplyDefault

func (t *Transformation) ApplyDefault(doc *yaml.Node) error

func (*Transformation) ApplyDeletes

func (t *Transformation) ApplyDeletes(doc *yaml.Node) error

func (*Transformation) ApplySet

func (t *Transformation) ApplySet(doc *yaml.Node) error

func (*Transformation) MustMatchAll

func (t *Transformation) MustMatchAll(doc *yaml.Node) (bool, error)