procs

README

Procs

Procs is a library to make working with command line applications a little nicer.

The primary use case is when you have to use a command line client in place of an API. Often times you want to do things like output stdout within your own logs or ensure that every time the command is called, there are a standard set of flags that are used.

Basic Usage

The majority of this functionality is intended to be included the procs.Process.

Defining a Command

A command can be defined by a string rather than a []string. Normally, this also implies that the library will run the command in a shell, exposing a potential man in the middle attack. Rather than using a shell, procs lexically parses the command for the different arguments. It also allows for pipes in order to string commands together.

p := procs.NewProcess("kubectl get events | grep dev")

You can also define a new Process by passing in predefined commands.

cmds := []*exec.Cmd{
	exec.Command("kubectl", "get", "events"),
	exec.Command("grep", "dev"),
}

p := procs.Process{Cmds: cmds}

Output Handling

One use case that is cumbersome is using the piped output from a command. For example, lets say we wanted to start a couple commands and have each command have its own prefix in stdout, while still capturing the output of the command as-is.

p := procs.NewProcess("cmd1")
p.OutputHandler = func(line string) string {
	fmt.Printf("cmd1 | %s\n")
	return line
}
out, _ := p.Run()
fmt.Println(out)

Whatever is returned from the OutputHandler will be in the buffered output. In this way you can choose to filter or skip output buffering completely.

You can also define a ErrHandler using the same signature to get the same filtering for stderr.

Environment Variables

Rather than use the exec.Cmd []string environment variables, a procs.Process uses a map[string]string for environment variables.

p := procs.NewProcess("echo $FOO")
p.Env = map[string]string{"FOO": "foo"}

Also, environment variables defined by the Process.Env can be expanded automatically using the os.Expand semantics and the provided environment.

There is a ParseEnv function that can help to merge the parent processes' environment with any new values.

env := ParseEnv(os.Environ())
env["USER"] = "foo"

Finally, if you are building commands manually, the Env function can take a map[string]string and convert it to a []string for use with an exec.Cmd. The Env function also accepts a useEnv bool to help include the parent process environment.

cmd := exec.Command("knife", "cookbook", "show", cb)
cmd.Env = Env(map[string]string{"USER": "knife-user"}, true)

Example Applications

Take a look in the cmd dir for some simple applications that use the library. You can also make all to build them. The examples below assume you've built them locally.

Prelog

The prelog command allows running a command and prefixing the output with a value.

$ ./prelog -prefix foo -- echo 'hello world!'
Running the command
foo | hello world!
Accessing the output without a prefix.
hello world!
Running the command with Start / Wait
foo | hello world!

Cmdtmpl

The cmdtmpl command uses the procs.Builder to create a command based on some paramters. It will take a data.yml file and template.yml file to create a command.

$ cat example/data.json
{
  "source": "https://my.example.org",
  "user": "foo",
  "model": "widget",
  "action": "create",
  "args": "-f new -i improved"
}
$ cat example/template.json
[
  "mysvc ${model} ${action} ${args}",
  "--endpoint ${source}",
  "--username ${user}"
]
$ ./cmdtmpl -data example/data.json -template example/template.json
Command: mysvc foo widget create -f new -i imporoved --endpoint https://my.example.org --username foo
$ ./cmdtmpl -data example/data.json -template example/template.json -field user=bar
Command: mysvc foo widget create -f new -i imporoved --endpoint https://my.example.org --username bar

Procmon

The procmon command acts like foreman with the difference being it uses a JSON file with key value pairs instead of a Procfile. This example uses the procs.Manager to manage a set of procs.Processes.

$ cat example/procfile.json
{
  "web": "python -m SimpleHTTPServer"
}
$ ./procmon -procfile example/procfile.json
web | Starting web with python -m SimpleHTTPServer

You can then access http://localhost:8000 to see the logs. You can also kill the child process and see procmon recognizing it has exited and exit itself.

Documentation

Overview

Procs is a library to make working with command line applications a little nicer.

The goal is to expand on the os/exec package by providing some features usually accomplished in a shell, without having to resort to a shell. Procs also tries to make working with output simpler by providing a simple line handler API over working with io pipes.

Finally, while the hope is that procs provides some convenience, it is also a goal to help make it easier to write more secure code. For example, avoiding a shell and the ability to manage the environment as a map[string]string are both measures that intend to make it easier to accomplish things like avoiding outputting secrets and opening the door for MITM attacks. With that said, it is always important to consider the security implications, especially when you are working with untrusted input or sensitive data.

Example

package main

import (
	"fmt"

	"github.com/ionrock/procs"
)

func main() {

	b := procs.Builder{
		Context: map[string]string{
			"NAME": "eric",
		},
		Templates: []string{
			"echo $NAME |",
			"grep $NAME",
		},
	}

	cmd := b.Command()

	fmt.Println(cmd)

	p := procs.NewProcess(cmd)

	p.Run()
	out, _ := p.Output()
	fmt.Println(string(out))
}

Output:
echo eric | grep eric
eric

Example (PredefinedCmds)

package main

import (
	"fmt"
	"os/exec"

	"github.com/ionrock/procs"
)

func main() {
	p := procs.Process{
		Cmds: []*exec.Cmd{
			exec.Command("echo", "foo"),
			exec.Command("grep", "foo"),
		},
	}

	p.Run()
	out, _ := p.Output()
	fmt.Println(string(out))
}

Output:
foo

Index

• func Env(env map[string]string, useEnv bool) []string
• func ParseEnv(environ []string) map[string]string
• func SplitCommand(cmd string) []string
• func SplitCommandEnv(cmd string, getenv func(key string) string) []string
• type Builder
  • func (b *Builder) Command() string
  • func (b *Builder) CommandContext(ctx map[string]string) string
• type Manager
  • func NewManager() *Manager
  • func (m *Manager) Remove(name string) error
  • func (m *Manager) Start(name, cmd string) error
  • func (m *Manager) StartProcess(name string, p *Process) error
  • func (m *Manager) StderrHandler(name string) OutHandler
  • func (m *Manager) StdoutHandler(name string) OutHandler
  • func (m *Manager) Stop(name string) error
  • func (m *Manager) Wait() error
• type OutHandler
• type Process
  • func NewProcess(command string) *Process
  • func (p *Process) ErrOutput() ([]byte, error)
  • func (p *Process) Output() ([]byte, error)
  • func (p *Process) Run() error
  • func (p *Process) Start() error
  • func (p *Process) Stop() error
  • func (p *Process) Wait() error

Examples

• Package
• Package (PredefinedCmds)
• SplitCommand
• SplitCommandEnv

Functions

func Env

func Env(env map[string]string, useEnv bool) []string

Env takes a map[string]string and converts it to a []string that can be used with exec.Cmd. The useEnv boolean flag will include the current process environment, overlaying the provided env map[string]string.

func ParseEnv

func ParseEnv(environ []string) map[string]string

ParseEnv takes an environment []string and converts it to a map[string]string.

func SplitCommand

func SplitCommand(cmd string) []string

SplitCommand parses a command and splits it into lexical arguments like a shell, returning a []string that can be used as arguments to exec.Command.

Example

package main

import (
	"fmt"

	"github.com/ionrock/procs"
)

func main() {
	parts := procs.SplitCommand("echo 'hello world'")
	for i, p := range parts {
		fmt.Printf("%d %s\n", i+1, p)
	}

}

Output:
1 echo
2 hello world

func SplitCommandEnv

func SplitCommandEnv(cmd string, getenv func(key string) string) []string

SplitCommandEnv parses a command and splits it into lexical arguments like a shell, returning a []string that can be used as arguments to exec.Command. It also allows providing an expansion function that will be used when expanding values within the parsed arguments.

Example

package main

import (
	"fmt"

	"github.com/ionrock/procs"
)

func main() {
	env := map[string]string{
		"GREETING": "hello",
		"NAME":     "world!",
		"PASSWORD": "secret",
	}

	getenv := func(key string) string {
		if v, ok := env[key]; ok && key != "PASSWORD" {
			return v
		}
		return ""
	}

	parts := procs.SplitCommandEnv("echo '$GREETING $NAME $PASSWORD'", getenv)

	for i, p := range parts {
		fmt.Printf("%d %s\n", i+1, p)
	}

}

Output:
1 echo
2 hello world!

Types

type Builder

type Builder struct {
	Context   map[string]string
	Templates []string
}

Builder helps construct commands using templates.

func (*Builder) Command

func (b *Builder) Command() string

Command returns the result of the templates as a single string.

func (*Builder) CommandContext

func (b *Builder) CommandContext(ctx map[string]string) string

CommandContext returns the result of the templates as a single string, but allows providing an environment context as a map[string]string for expansions.

type Manager

type Manager struct {
	Processes map[string]*Process
	// contains filtered or unexported fields
}

Manager manages a set of Processes.

func NewManager

func NewManager() *Manager

NewManager creates a new *Manager.

func (*Manager) Remove

func (m *Manager) Remove(name string) error

Remove will try to stop and remove a managed process.

func (*Manager) Start

func (m *Manager) Start(name, cmd string) error

Start and managed a new process using the default handlers from a string.

func (*Manager) StartProcess

func (m *Manager) StartProcess(name string, p *Process) error

StartProcess starts and manages a predifined process.

func (*Manager) StderrHandler

func (m *Manager) StderrHandler(name string) OutHandler

StderrHandler returns an OutHandler that will ensure the underlying process has an empty stderr buffer and logs to stdout a prefixed value of "$name | $line".

func (*Manager) StdoutHandler

func (m *Manager) StdoutHandler(name string) OutHandler

StdoutHandler returns an OutHandler that will ensure the underlying process has an empty stdout buffer and logs to stdout a prefixed value of "$name | $line".

func (*Manager) Stop

func (m *Manager) Stop(name string) error

Stop will try to stop a managed process. If the process does not exist, no error is returned.

func (*Manager) Wait

func (m *Manager) Wait() error

Wait will block until all managed processes have finished.

type OutHandler

type OutHandler func(string) string

OutHandler defines the interface for writing output handlers for Process objects.

type Process

type Process struct {
	// CmdString takes a string and parses it into the relevant cmds
	CmdString string

	// Cmds is the list of command delmited by pipes.
	Cmds []*exec.Cmd

	// Env provides a map[string]string that can mutated before
	// running a command.
	Env map[string]string

	// Dir defines the directory the command should run in. The
	// Default is the current dir.
	Dir string

	// OutputHandler can be defined to perform any sort of processing
	// on the output. The simple interface is to accept a string (a
	// line of output) and return a string that will be included in the
	// buffered output and/or output written to stdout.'
	//
	// For example defining the Process as:
	//
	//     prefix := "myapp"
	//     p := &procs.Process{
	//         OutputHandler: func(line string) string {
	//             return fmt.Sprintf("%s | %s", prefix, line)
	//         },
	//     }
	//
	// This would prefix the stdout lines with a "myapp | ".
	//
	// By the default, this function is nil and will be skipped, with
	// the unchanged line being added to the respective output buffer.
	OutputHandler OutHandler

	// ErrHandler is a OutputHandler for stderr.
	ErrHandler OutHandler
	// contains filtered or unexported fields
}

Process is intended to be used like exec.Cmd where possible.

func NewProcess

func NewProcess(command string) *Process

NewProcess creates a new *Process from a command string.

It is assumed that the user will mutate the resulting *Process by setting the necessary attributes.

func (*Process) ErrOutput

func (p *Process) ErrOutput() ([]byte, error)

ErrOutput returns the buffered stderr as []byte

func (*Process) Output

func (p *Process) Output() ([]byte, error)

Output returns the buffered output as []byte.

func (*Process) Run

func (p *Process) Run() error

Run executes the cmds and returns the output as a string and any error.

func (*Process) Start

func (p *Process) Start() error

Start will start the list of cmds.

func (*Process) Stop

func (p *Process) Stop() error

Stop tries to stop the process.

func (*Process) Wait

func (p *Process) Wait() error

Wait will block, waiting for the commands to finish.