mimic

package module
Version: v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 2, 2021 License: Apache-2.0 Imports: 9 Imported by: 6

README

Mimic - Define your Configuration, Infrastructure and Deployments as Go Code

Go Report Card GoDoc

mimic: A Go module for defining and generating config in Go:

  • Define your Configuration (e.g Envoy, Prometheus, Alertmanager, Nginx, Prometheus Alerts, Rules, Grafana Dashaboards etc.)
  • Define your Infrastructure (e.g Terraform, Ansible, Puppet, Chef, Kubernetes etc)
  • Define any other file that you can imagine

...using simple, typed and testable Go code!

mimic: Mimic is a super-human with the ability to copy the powers and abilities of others.

Getting Started

  1. Create a new go file for your config e.g config/example.go
  2. Import mimic using Go 1.12+ via import "github.com/bwplotka/mimic"
  3. Add configuration in your main package using the mimic module
  4. Run go run config/example.go generate
  5. You will now see the generated Kubernetes YAML file here: cat gen/config/some-statefulset.yaml

For example:

   package main
   
   import (
       "github.com/bwplotka/mimic"
       "github.com/bwplotka/mimic/encoding"
       "github.com/go-openapi/swag"
       appsv1 "k8s.io/api/apps/v1"
       corev1 "k8s.io/api/core/v1"
       metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
   )
   
   func main() {
       generator := mimic.New()
   
       // Defer Generate to ensure we generate the output.
       defer generator.Generate()
   
       // Hook in your config below.
       // As an example Kubernetes configuration!
       const name = "some-statefulset"
   
       // Create some containers ... (imagine for now).
       var container1, container2, container3 corev1.Container
       var volume1 corev1.Volume
       
       // Configure a statefulset using native Kubernetes structs.
       set := appsv1.StatefulSet{
           TypeMeta: metav1.TypeMeta{
               Kind:       "StatefulSet",
               APIVersion: "apps/v1",
           },
           ObjectMeta: metav1.ObjectMeta{
               Name: name,
               Labels: map[string]string{
                   "app": name,
               },
           },
           Spec: appsv1.StatefulSetSpec{
               Replicas:    swag.Int32(2),
               ServiceName: name,
               Template: corev1.PodTemplateSpec{
                   ObjectMeta: metav1.ObjectMeta{
                       Labels: map[string]string{
                           "app": name,
                       },
                   },
                   Spec: corev1.PodSpec{
                       InitContainers: []corev1.Container{container1},
                       Containers:     []corev1.Container{container2, container3},
                       Volumes:        []corev1.Volume{volume1},
                   },
               },
               Selector: &metav1.LabelSelector{
                   MatchLabels: map[string]string{
                       "app": name,
                   },
               },
           },
       }
       // Now Add some-statefulset.yaml to the config folder.
       generator.With("config").Add(name+".yaml", encoding.GhodssYAML(set))
   }

Now you are ready to start defining your own resources!

Other examples can be found in here.

What is mimic?

mimic is a package that allows you to define your configuration using Go and generate this into configuration files your application and tooling understands.

Why was mimic created?

mimic has been built from our past experience using this concept to configure our applications and infrastructure.

It offers not only to show the concept and an implementation example but also to share what we have learned, best practice and patterns that we believe are valuable for everyone.

But Why Go?

Why you should define your templates/infra/configs in Go?

  • Configuration as code ... like actual code, not json, yaml or tf.

  • Go is a strongly typed language. This means that compiler and IDE of your choice will massively help you find what config fields are allowed, what values enum expects and what is the type of each field.

  • Unit/Integration test your configuration, infrastructure and deployment.
    For example:

    • Test your PromQL queries in Prometheus alerts works as expected? Just write unit test for those using e.g this
    • Enforce conventions such as service naming conventions via tests.
  • Import configuration structs directly from the project you are configurating for example bring in Kubernestes, Prometheus or any other structs that are exported. Allowing you to ensure your config matches the project.

    No more blind searches and surprises. It cannot be safer or simpler than this.

  • Versioning and dependency management. Utilize go modules to ensure you are using the correct version of the configuration for the project version you are running.

  • Documentation of your config, Go recommends goDoc formatting, so you can leverage native comments for each struct's fields to document behaviour or details related to the config field. Giving you visibility in your config of exactly what your defining. See this great Kubernetes struct as an example.

  • Quick feedback loop. Catch most mistakes and incompatibilities in Go compile time, before you even deploy it further. As you probably know one of Go goal is to have very fas compilation time, which feels like you are running a script.

  • Keep the set of the languages used in your organization to a minimum - just one: Go, one of the cleanest, simplest and developer friendly languages around.

What mimic is NOT

  • It does NOT implement any deployment/distribution logic.
  • It is NOT intended to trigger any changes. Instead use the right tool for the job e.g. kubectl apply, ansible, puppet, chef, terraform
  • It is NOT (yet) a registry for reusable templates, however we encourage the community to create public repositories for those!

What does mimic include?

  • providers go structs representing configuration for applications, infrastructure and container management.
    • Included are a set of go providers that do not have native structs OR are not easily importable (yet).
  • encoding a way to transform your go config struts into specific file types.
    • Included are json, yaml and jsonpb.
  • abstractions a way to abstract concepts to a higher level if really needed (see best practice).
  • Examples:

Want to help us and Contribute?

Please do!

First start defining your configuration, infra and deployment as Go code!

Share with the community:

  • Share the Go templates you create.
  • Share the Go configuration structs for non-Go projects.
  • Share the Go unit/integration/acceptance tests against certain providers's definitions.
  • Share best practices and your experience!

Please use GitHub issues and our slack #mimic for feedback and questions.

As always pull requests are VERY welcome as well!

Have a project written in Go?

If you maintain your own project using Go it would be great to help the effort of making config as go a reality by exposing your configuration structs for users to import.

How: * Maintain and export your config structs like Kubernetes does (it is an API and well documented types) * Define configuration file via protobuf e.g like envoy here

Problems:

  • What if project has only json schema? or no schema at all, just project written in different language:

    Answer: Generate it yourself from YAML (e.g using this online tool). Answer2: At some point if this concept will be big enough anyone can maintain useful Go module with typed, documented and testable config for some providers like we have in providers package

  • Importing native Go structs is the dream, however:

    • Not all project's config are prepared to be imported (config tied to implementation, huge deps, secret masked, no marshaling etc): See: https://github.com/prometheus/alertmanager/pull/1804
    • This is where providers come in and we can define a set of structs to meet the config specified for your needs.

Documentation

Other solutions

  • Cue
  • mixins
  • jsonnet
  • Pullumi (Paid)

Documentation

Overview

`mimic`: A Go module for defining and generating config in Go:

* Define your Configuration (e.g Envoy, Prometheus, Alertmanager, Nginx, Prometheus Alerts, Rules, Grafana Dashaboards etc.) * Define your Infrastructure (e.g Terraform, Ansible, Puppet, Chef, etc) * Define your Container Management (e.g Docker compose, Kubernetes, etc) * Define any other file that you can imagine

... using simple, typed and testable Go code!

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func PanicErr

func PanicErr(err error)

PanicErr allows to panic because of certain error.

func PanicIfErr

func PanicIfErr(err error)

PanicIfErr allows to panic on error.

func Panicf

func Panicf(format string, a ...interface{})

Panicf allows panic error propagation using sprintf-like formatting.

Types

type FilePool

type FilePool struct {
	Logger log.Logger
	// contains filtered or unexported fields
}

FilePool is a struct for storing and managing files to be generated as part of generation.

func (*FilePool) Add

func (f *FilePool) Add(fileName string, r io.Reader)

Add adds a file to the file pool at the current path. The file is identified by filename. Content of the file is passed via an io.Reader.

If the file with the given name has already been added at this path the code will `panic`. NOTE: See mimic/encoding for different marshallers to use as io.Reader.

type Generator

type Generator struct {
	FilePool
	// contains filtered or unexported fields
}

Generator manages a pool of generated files.

func New

func New(injs ...func(cmd *kingpin.CmdClause)) *Generator

New returns a new Generator that parses os.Args as command line arguments. It allows passing closure BEFORE parsing the flags to allow defining additional flags.

NOTE: Read README.md before using. This is intentionally NOT following Go library patterns like: * It uses panics as the main error handling way. * It creates CLI command inside constructor. * It does not allow custom loggers etc

func (*Generator) Generate

func (g *Generator) Generate()

Generate generates the configuration files that have been defined and added to a generator.

func (*Generator) With

func (g *Generator) With(parts ...string) *Generator

With behaves like linux `cd` command. It allows to "walk" & organize output files in a desired way for ease of use. Example:

“`

gen := gen.With("mycompany.com", "production", "eu1", "kubernetes", "thanos")

“` Giving the path `mycompany.com/production/eu1/kubernetes/thanos`.

With return a Generator pointing at the specified path which can be specified even further: Example: “`

gen := mimic.New()
// gen/
...
gen = gen.With('foo')
// gen/foo
...
{
  gen := gen.With('bar')
  // gen/foo/bar
}
// gen/foo

“`

Directories

Path Synopsis
encoding is a package of default encodings supplied with mimic.
encoding is a package of default encodings supplied with mimic.
lib
abstr
Abstractions are used to (as the name suggests) abstract away the underlying config and structs to the caller allowing complex configurations or concepts to be created with minimal code.
Abstractions are used to (as the name suggests) abstract away the underlying config and structs to the caller allowing complex configurations or concepts to be created with minimal code.
schemas
Providers are a way of defining a configuration specification in Go structs for projects that either do not currently expose their configuration structs or are difficult to work with (due to deps, complexity etc.).
Providers are a way of defining a configuration specification in Go structs for projects that either do not currently expose their configuration structs or are difficult to work with (due to deps, complexity etc.).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL