cli

package module
v0.0.0-...-8635785 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 19, 2018 License: MIT Imports: 16 Imported by: 0

README

GoDoc

Streamline CLI and configuration code in Go applications using static analysis and code generation. Commands and config are described by convention, allowing automatic generation of flags, command handlers, docs, config loaders, etc.

Status

Beta quality: relatively new and hasn't been fully proven yet.

Usage

Install:

go get github.com/buchanae/cli/cmd/cli

In main_cli.go:

package main

import (
  "fmt"
  "net/http"
  "github.com/buchanae/cli"
)

//go:generate cli .

type ServerOpt struct {
  // Server name.
  Name string
  // Address to listen on.
  Addr string
}

func DefaultServerOpt() ServerOpt {
  return ServerOpt{
    Name: "cli-example",
    Addr: ":8080",
  }
}

// Run a simple echo server.
// Example: ./server run --name "my-server" "Hello, world!"
func Run(opt ServerOpt, msg string) {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "from server %q: %s\n", opt.Name, msg)
  })
  cli.Check(http.ListenAndServe(opt.Addr, nil))
}

func main() {
  cli.AutoCobra("server", specs())
}

Generate the CLI command handlers and build it:

go generate .
found cli "Run"
generated file generated_specs.go

go build -o server .

Try it out:

./server run 'Hello, world!'

curl localhost:8080
from server "cli-example": Hello, world!

Try it with some different options:

./server run --name "my-server" --addr :8081 'Hello, world!'

curl localhost:8081
from server "my-server": Hello, world!

Try it with a config file:

cat config.yaml
addr: ":8081"
name: "config-server"

./server run 'Hi!'

curl localhost:8081
from server "config-server": Hi!

This example code is in ./examples/server.

How it works

The cli utility parses the source code, looks for exported functions and their options, then generates Go code containing this metadata (a cli.Spec). At runtime, this metadata is used to generate commands, flags, docs, loaders, etc.

Conventions used:

  • Only files with the suffix _cli.go are analyzed.
  • Exported functions are turned into CLI commands.
  • If a function has a struct-type argument named opt, the fields of that type are used to generate flags, docs, file loaders, etc.
  • If an option type MyOpt has a matching function DefaultMyOpt() MyOpt, that function will provide default values for the options.
  • Command function arguments are coerced from CLI positional arguments,
    e.g. Age(name string, age int) maps to ./app age "Alex" 33

I'm most familiar with cobra and YAML config files, so I wrote AutoCobra(appName string, specs []Spec) to handle my common usecase, but hopefully cli is flexible enough to handle a wide variety of preferences.

Why?

Building powerful configuration and commandline interfaces is important, yet writing the code can be tedious, error-prone, and tricky.

Issues I frequently encounter:

  • Loading and merging config files, defaults, and CLI flags is error-prone and tricky. It's easy to get the order wrong, and correct code gets complex.

  • Config key names and CLI flag names can have inconsistent naming/casing (http.port vs --server_port).

  • Config files and CLI flags can get out of sync, e.g. add a new config option but forget to add a CLI flag.

  • A subset of config is available via CLI flags, a different subset via env. vars, and the rest requires a config file.

  • Names can be misspelled or incorrectly formatted, but the error passes silently, leading to subtle behavior that is difficult to debug.

  • time.Duration (and friends) is not handled well by common marshalers, such as YAML.

  • Wrappers for time.Duration are needed for proper (un)marshling from config files, but the wrappers then invade the whole codebase, leading to code like time.NewTimer(time.Duration(config.TimeoutDuration)).

  • Config docs easily get out of sync with the actual types.

  • Evolving config leads to broken systems when upgrading to newer versions.

  • Writing CLI and config code is often tedious, verbose, and covered in boilerplate. This is especially annoying when you build a CLI with lots of commands.

  • Unit testing is tricky because the CLI/config code usually interacts with the entire application. Organizing mocks or other tricks gets messy.

  • Unit testing is tricky because, again, you want to replace the usual stdin/out/err with something you can test.

  • Every new project develops a new pattern for loading, validation, testing, etc.

Design Goals

  1. Configuration should feel natural when created and used in code. Config should be based on struct types, defaults should be provided by instances, or functions that return instances, of those types.

  2. Documentation should be written in code, as with nearly all Go documentation. Tools should be provided to generate other forms of documentation.

  3. Flags, environment variables, config files, and other types of data sources should use struct types as the source of truth while loading.

  4. Common config errors, such as misspelling or non-existent keys, should be caught by the core library.

  5. Merging multiple config sources should be handled simply and robustly by the core library.

  6. Projects should be consistent in their config, CLI, and test code. The pattern and tools developed here should be robust enough to support many projects.

  7. The pattern should allow for easily removing code that is duplicated amongst multiple commands, such as database init code.

  8. The pattern should help reduce boilerplate and other sources of verbose code.

Other wants:

  • Generate YAML config with dynamic values and docs.
    Useful for doc generation, debugging, and bug reports.

Design Decisions

  1. Use cli.Fatal in top-level CLI code to minimize error checks. I totally agree with Go's approach to error handling by value, but panic/recover is useful too. A CLI command is top-level; it isn't expected to be called from other code, so the error values aren't being checked by anything, and panics aren't escaping into general code. CLI commands aren't being called in loops, so panic/recover performance isn't an issue. When an error occurs in a CLI command, you usually want the whole program to stop with an error, which seems like a good fit for Fatal/panic.

  2. Use code generation to inspect commands and config. The best way to keep config and docs up to date is to have it written right next to the code in the form of code comments.

  3. Keep code generation minimal; generate just enough information for libraries to do the rest at runtime. Details such as cobra command building, doc parsing, flag building, etc. could all happen during code generation, but it feels slightly less flexible and more likely to become complex. Also, more strings/data being generated as code means larger binaries for projects with lots of commands. Honestly, I'm on the fence here though.

  4. Allow an alternative to struct tags. Sometimes you don't have access to the struct type, or you don't want to modify it (maybe it's generated by protoc).

To do / Known Issues

  • be able to hide/ignore fields without using a struct tag, for fields which you don't have access to or don't want to modify with cli tags.
  • provide sensitive tag for passwords and other sensitive fields.
  • properly marshal yaml/json slices/maps/etc.
  • GCE metadata, etcd, consul, openstack provider
  • dump json, env, flags
  • handle map[string]string via "key=value" flag value
  • pull fieldname from json tag
  • ignore/alias fields via struct tag
  • recognize misspelled env var
  • case sensitivity
  • manage editing config file

Complex:

  • reloading
  • multiple config files with merging

Questions:

  • how to handle pointers? cycles?
  • how are slices handled in env vars?
  • how are slices of structs handled in flags?

Documentation

Overview

package cli provides utilities for streamlining CLI and configuration code. See github.com/buchanae/cli for a full example.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// DotKey joins key parts with a "." and converts to lowercase.
	DotKey KeyFunc = lowerJoin(".")

	// UnderscoreKey joins key parts with a "_" and converts to lowercase.
	UnderscoreKey = lowerJoin("_")

	// DashKey joins key parts with a "-" and converts to lowercase.
	DashKey = lowerJoin("-")
)
View Source 
var DefaultJSON = FileOpts{
	Paths:  []string{"config.json"},
	OptKey: []string{"config"},
}

DefaultJSON contains the most common configuration for loading options from a JSON config file.

View Source 
var DefaultTOML = FileOpts{
	Paths:  []string{"config.toml"},
	OptKey: []string{"config"},
}

DefaultTOML contains the most common configuration for loading options from a TOML config file.

View Source 
var DefaultYAML = FileOpts{
	Paths:  []string{"config.yaml", "config.yml"},
	OptKey: []string{"config"},
}

DefaultYAML contains the most common configuration for loading options from a YAML config file.

Functions

func AutoCobra 

func AutoCobra(appname string, specs []Spec) error

AutoCobra sets up a common pattern for apps: commands are built into a tree of cobra subcommands, flags are created using pflag, options are loaded from env. vars, flags, and YAML config files.

func Check 

func Check(err error)

Check panics with an instance of ErrFatal if err != nil.

func Coerce 

func Coerce(dest interface{}, val interface{}) error

Coerce attempts to coerce "val" to the type of "dest".

func Enrich 

func Enrich(cmd *Cmd)

Enrich fills in Cmd and Opt fields runtime by parsing names and doc strings, looking for annotations such as Synopsis, Deprecated, Example, etc.

Example 
cmd := &Cmd{
	RawName: "WordCount",
	RawDoc: `Count the number of words in a file.

word-count counts the number of words in a file and writes a single number to stdout.
This is a second line of docs.

Deprecated: please don't use word-count, it will be removed.
Hidden
Aliases: count-words count-word wc
Example: word-count <file.txt>
`,
	Opts: []*Opt{
		{
			RawDoc: `Opt detail synopsis.`,
		},
		{
			DefaultValue: os.Stderr,
		},
	},
}

Enrich(cmd)

fmt.Println("NAME: ", cmd.Name)
fmt.Println("PATH: ", cmd.Path)
fmt.Println("SYN:  ", cmd.Synopsis)
fmt.Println("EX:   ", cmd.Example)
fmt.Println("DEP:  ", cmd.Deprecated)
fmt.Println("HIDE: ", cmd.Hidden)
fmt.Println("ALIAS:", cmd.Aliases)
fmt.Println("SYN:  ", cmd.Opts[0].Synopsis)
fmt.Println("DEF:  ", cmd.Opts[1].DefaultString)

Output:

NAME:  count
PATH:  [word count]
SYN:   Count the number of words in a file.
EX:    word-count <file.txt>
DEP:   please don't use word-count, it will be removed.
HIDE:  true
ALIAS: [count-words count-word wc]
SYN:   Opt detail synopsis.
DEF:   os.Stderr

func Fatal 

func Fatal(msg string, args ...interface{})

Fatal panics with an instance of ErrFatal with a formatted message.

func Run 

func Run(spec Spec, l *Loader, raw []string) (err error)

Run runs the Spec with the given args. The loader is used to load option values from multiple sources (flags, env, yaml, etc). Panics of type ErrFatal and ErrUsage are recovered and returned as an error, all other panics are passed through.

Types

type Arg 

type Arg struct {
	// Name is the name of this argument.
	Name string
	// Type contains a string describing the type of this field,
	// e.g. "[]string".
	Type string
	// Variadic is true if this function argument is variadic,
	// e.g. func example(a string, b ...string)
	// "b" is variadic.
	Variadic bool
	// Value contains a pointer to the value for this argument.
	// Used by `cli` machinery to set the value of this argument.
	Value interface{}
}

Arg defines a positional argument of a Cmd.

Usually generated by the `cli` code generator.

type Cmd 

type Cmd struct {
	// RawName is the raw, unprocessed  name of the function.
	RawName string
	// RawDoc is the raw, unprocessed doc string attached to the function.
	RawDoc string
	// Args describes metadata about the function arguments.
	Args []*Arg
	// Opts describes metadata about the function options
	// (the `opt` argument, by convention).
	Opts []*Opt

	// Name is the command name,
	// normally determined by parsing the raw function name and/or doc.
	Name string
	// Path is the path to this subcommand,
	// normally determined by parsing the raw function name and/or docs.
	Path []string
	// Doc is the function doc after processing,
	// where annotations such as  "Deprecated" and "Hidden" are removed.
	Doc string
	// Synopsis is a short description of the command.
	Synopsis string
	// Example describes an example of how to use the command.
	Example string
	// Deprecated marks this command as deprecated and contains
	// a message describing why.
	Deprecated string
	// Hidden marks this command as hidden.
	Hidden bool
	// Aliases contains aliases for this command.
	Aliases []string
}

Cmd holds metadata related to a CLI command.

Usually generated by the `cli` code generator.

type Cobra 

type Cobra struct {
	cobra.Command
	KeyFunc
}

Cobra helps build a set of cobra commands. Commands are built into a tree of subcommands based on their common subpaths.

func (*Cobra) Add 

func (cb *Cobra) Add(spec Spec) *cobra.Command

Add adds a command to the tree.

func (*Cobra) SetRunner 

func (cb *Cobra) SetRunner(cmd *cobra.Command, spec Spec, l *Loader)

SetRunner sets `cobra.Command.RunE` to use the loader and runner from this package.

type ErrFatal 

type ErrFatal struct {
	// contains filtered or unexported fields
}

ErrFatal is used to signal fatal errors coming from `Fatal`.

type ErrUsage 

type ErrUsage struct {
	// contains filtered or unexported fields
}

ErrUsage is used to signal fatal errors caused by invalid commandline usage.

type FileOpts 

type FileOpts struct {
	// Paths is a list of paths to look for a config file.
	// Environment variables will be expanded using `os.ExpandEnv`.
	// Loading will stop on the first path that exists.
	Paths []string
	// OptKey is used to look for a config file path set by
	// an option (e.g. by a flag or env. var). For example,
	// an OptKey of ["config", "file"] could load the config
	// file from a "--config.file" flag. OptKey is prepended
	// to the Paths list, so it takes priority.
	OptKey []string
}

FileOpts describes options related to loading options from a file.

type KeyFunc 

type KeyFunc func([]string) string

KeyFunc defines a function which transforms a key.

Example 
path := []string{"Foo", "bar", "BAZ"}
fmt.Println(DotKey(path))
fmt.Println(UnderscoreKey(path))
fmt.Println(DashKey(path))

Output:

foo.bar.baz
foo_bar_baz
foo-bar-baz

type Loader 

type Loader struct {

	// Coerce can be used to override the type coercion
	// needed when setting an option value. A coerce function
	// must set the value. "dst" is always a pointer to the
	// value which needs to be set, e.g. *int for an option
	// value of type int. See coerce.go for an example.
	Coerce func(dst, src interface{}) error
	// contains filtered or unexported fields
}

Loader is used to load, coerce, and set option values at command run time. Loader.Load runs the providers in order. Once an option has been set, it is not overridden by later values.

func NewLoader 

func NewLoader(opts []*Opt, providers ...Provider) *Loader

NewLoader returns a Loader instance which is configured to load option values from the given providers.

func (*Loader) Errors 

func (l *Loader) Errors() []error

Errors returns a list of errors encountered during loading.

func (*Loader) Get 

func (l *Loader) Get(key []string) interface{}

Get gets the current option value for the given key.

func (*Loader) GetString 

func (l *Loader) GetString(key []string) string

GetString returns the option value as a string, or else an empty string.

func (*Loader) Keys 

func (l *Loader) Keys() [][]string

Keys returns a list of keys for all options.

func (*Loader) Load 

func (l *Loader) Load()

Load runs the providers, loading and setting option values. Load is meant to be called only once. TODO this is awkward. The code should make it clear that 

it only gets called once, and/or make it safe to load
multiple times.

func (*Loader) Set 

func (l *Loader) Set(key []string, val interface{})

Set sets an option value for the option at the given key. Once an option is set, it will not be overridden by future calls to Set. Set uses Loader.Coerce to set the value.

TODO this is ugly. This checks IsSet, but doesn't set it to true, 

that happens somewhere else. The whole set/coerce interface
needs cleanup

type Opt 

type Opt struct {
	// Key is the path of struct fields names from the root to this option.
	// e.g. ["Server", "TLS", "KeyPath"]
	Key []string
	// RawDoc is the raw, unprocessed doc string attached to this field.
	RawDoc string

	// Doc is the field doc after processing,
	// where annotations such as  "Deprecated" and "Hidden" are removed.
	Doc string
	// Synopsis is a short description of this option.
	Synopsis string
	// Hidden marks this option as hidden.
	Hidden bool
	// Deprecated marks this option as deprecated and contains
	// a message describing why.
	Deprecated string
	// Type contains a string describing the type of this field,
	// e.g. "[]string".
	Type string
	// Short is the name of the short version of a flag for this option.
	Short string
	// Value contains a pointer to the value for this option.
	// Used by Loader machinery to set the value of this option.
	Value interface{}
	// DefaultValue contains the default value of this option.
	DefaultValue interface{}
	// DefaultString contains a more human-friendly description
	// of the default value, e.g. os.Stderr instead of io.Writer.
	DefaultString string
	// IsSet is true if this value has been set.
	// Used by Loader machinery.
	// TODO ugly.
	IsSet bool
}

Opt holds metadata related to an option of a Cmd.

Usually generated by the `cli` code generator.

type Provider 

type Provider interface {
	Provide(*Loader) error
}

Provider is implemented by types which provide option values, such as from environment variables, config files, CLI flags, etc.

func Consul 

func Consul() Provider

func Env 

func Env(prefix string) Provider

Env loads option values from environment variables with the given prefix. The prefix and option keys are converted to uppercase.

Example 
// Note: Env converts keys to uppercase + underscore.
os.Setenv("CLI_FOO_BAR", "baz")

// TODO gross
val := ""
key := []string{"foo", "bar"}
opts := []*Opt{
	{Key: key, Value: &val},
}

l := NewLoader(opts, Env("cli"))
l.Load()
fmt.Println(val)

Output:

baz

func JSON 

func JSON(opts FileOpts) Provider

JSON loads options from a JSON file.

func PFlags 

func PFlags(fs *pflag.FlagSet, opts []*Opt, kf KeyFunc) Provider

PFlags loads option values from a pflag.FlagSet. The given KeyFunc is used to format the flag names: DotKey will create flags like "server.address", DashKey will create "server-address", etc.

func TOML 

func TOML(opts FileOpts) Provider

TOML loads options from a TOML file.

func YAML 

func YAML(opts FileOpts) Provider

YAML loads options from a YAML file.

type Spec 

type Spec interface {
	Cmd() *Cmd
	Run()
}

Spec is implemented by the generated by the `cli` code generator.

Source Files

View all Source files

Directories

Path Synopsis
cmd
cli
examples
mailer/foo
old
old/dynamo
old/logger
old/roger-example
old/scheduler
old/server
old/simple
old/storage
old/worker
server
main_cli.go
 main_cli.go
simple
todo
todo/db

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.