herd

package module
v0.12.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 9, 2023 License: MIT Imports: 36 Imported by: 0

README

Herd - Lightning-fast parallel ssh client

Herd is a massively parallel ssh client that will replace all your hacky for loops, pssh, xargs and gnu parallel oneliners with something that is faster, more flexible and simply more awesome!

Herd can find your hosts in you known_hosts file, in consul and many other places. Using a powerful query syntax, you select the hosts you want and run any command in parallel on all of them. Output can be streamed as it happens, or shown at the end for easy inspection. On top of that, Herd is chock full of helpful features:

  • It can query your inventory systems and display information and statistics about your hosts
  • Timeouts, parallelism and delays are all configurable for very flexible ways of running commands
  • A variety of output modes show exactly what you need. Be it timestamps at the start of all lines, output separated by host or one big river of output. It can do all that and more.
  • Output is also stored in a machine-readable way so you can post process it as much as you want
  • An interactive shell is available that even lets you create host lists based on the result of running commands on hosts

Examples

Here are some examples to get you going. The full documentation can be found on https://herd.seveas.net

  • List all hosts in a specific domain

    herd list *.example.com

  • Run a command on all of them

    herd run *.example.com -- uptime

  • Run a long-running command, with output appearing immediately and reduced parallelism

    herd run *.example.com --parallel 10 --host-timeout 5m --timeout 1h -- sudo puppet agent -t

  • Show some information about hosts in an inventory system

    herd list site=site-5 --attributes ip_address,os,memory

  • Rolling restart of a service registered in consul

    herd run --splay 1m --parallel 1 consul_service=smtpd -- sudo systemctl restart postfix

Installing

There are pre-built binaries on the download page. If you prefer to install from source, install go, clone this repository and just run make.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Providers 

func Providers() []string

func RegisterProvider 

func RegisterProvider(name string, constructor func(string) HostProvider, magic func() HostProvider)

func Version 

func Version() string

func VersionTuple added in v0.11.0 

func VersionTuple() (major, minor, patch int)

Types

type Cache 

type Cache interface {
	Source() HostProvider
	Invalidate()
	Keep()
	SetCacheDir(string)
}

type DataLoader 

type DataLoader interface {
	SetDataDir(string) error
}

type Executor 

type Executor interface {
	Run(ctx context.Context, host *Host, cmd string, oc chan OutputLine) *Result
	SetConnectTimeout(time.Duration)
}

type History 

type History []*HistoryItem

func (History) Save 

func (h History) Save(path string) error

type HistoryItem 

type HistoryItem struct {
	Command string
	Results []*Result
	Summary struct {
		Ok   int
		Fail int
		Err  int
	}
	StartTime   time.Time
	EndTime     time.Time
	ElapsedTime float64
	// contains filtered or unexported fields
}

func (*HistoryItem) MarshalJSON 

func (h *HistoryItem) MarshalJSON() ([]byte, error)

type Host 

type Host struct {
	Name       string
	Address    string
	Attributes HostAttributes
	Connection io.Closer
	// contains filtered or unexported fields
}

A host represents a remote host. It can be instantiated manually, but is usually fetched from one or more Providers, which can all contribute to the hosts attributes.

func NewHost 

func NewHost(name, address string, attributes HostAttributes) *Host

Hosts should be initialized with this function, which also initializes any internal data, without which SSH connections will not be possible.

func (*Host) AddPublicKey 

func (h *Host) AddPublicKey(k ssh.PublicKey)

Adds a public key to a host. Used by the ssh know hosts provider, but can be used by any other code as well.

func (*Host) Amend 

func (h *Host) Amend(h2 *Host)

func (*Host) GetAttribute 

func (h *Host) GetAttribute(key string) (interface{}, bool)

func (*Host) MarshalJSON added in v0.11.0 

func (h *Host) MarshalJSON() ([]byte, error)

func (*Host) Match 

func (h *Host) Match(hostnameGlob string, attributes MatchAttributes) bool

func (*Host) PublicKeys 

func (h *Host) PublicKeys() []ssh.PublicKey

func (Host) String 

func (h Host) String() string

func (*Host) UnmarshalJSON 

func (h *Host) UnmarshalJSON(data []byte) error

type HostAttributes 

type HostAttributes map[string]interface{}

Hosts can have attributes of any types, but querying is limited to strings, booleans, numbers, nil and slices of these values.

type HostKeyProvider added in v0.11.0 

type HostKeyProvider interface {
	LoadHostKeys(ctx context.Context, l LoadingMessage) (map[string][]ssh.PublicKey, error)
}

type HostListOptions 

type HostListOptions struct {
	OneLine       bool
	Separator     string
	Csv           bool
	Attributes    []string
	AllAttributes bool
	Align         bool
	Header        bool
	Template      string
	Count         []string
	CountAll      bool
	SortByCount   bool
	Group         string
}

type HostProvider 

type HostProvider interface {
	Name() string
	Prefix() string
	ParseViper(v *viper.Viper) error
	Load(ctx context.Context, l LoadingMessage) (*HostSet, error)
	Equivalent(p HostProvider) bool
}

func NewProvider 

func NewProvider(pname, name string) (HostProvider, error)

type HostSet added in v0.10.0 

type HostSet struct {
	// contains filtered or unexported fields
}

func MergeHostSets added in v0.10.0 

func MergeHostSets(sets []*HostSet) *HostSet

func NewHostSet added in v0.10.0 

func NewHostSet() *HostSet

func (*HostSet) AddHost added in v0.10.0 

func (s *HostSet) AddHost(host *Host)

func (*HostSet) AddHosts added in v0.10.0 

func (s *HostSet) AddHosts(hosts *HostSet)

func (*HostSet) Get added in v0.10.0 

func (s *HostSet) Get(i int) *Host

func (*HostSet) Len added in v0.10.0 

func (s *HostSet) Len() int

func (*HostSet) MarshalJSON added in v0.10.0 

func (s *HostSet) MarshalJSON() ([]byte, error)

func (*HostSet) Remove added in v0.10.0 

func (s *HostSet) Remove(glob string, attrs MatchAttributes)

func (*HostSet) Sample added in v0.10.0 

func (s *HostSet) Sample(attributes []string, count int)

func (*HostSet) Search added in v0.10.0 

func (s *HostSet) Search(hostnameGlob string, attributes MatchAttributes) *HostSet

func (*HostSet) SetSortFields added in v0.10.0 

func (s *HostSet) SetSortFields(attributes []string)

func (*HostSet) Sort added in v0.10.0 

func (s *HostSet) Sort()

func (*HostSet) String added in v0.10.0 

func (s *HostSet) String() string

func (*HostSet) Uniq added in v0.10.0 

func (s *HostSet) Uniq()

func (*HostSet) UnmarshalJSON added in v0.10.0 

func (s *HostSet) UnmarshalJSON(b []byte) error

type LoadingMessage 

type LoadingMessage func(string, bool, error)

type MatchAttribute 

type MatchAttribute struct {
	Name        string
	FuzzyTyping bool
	Negate      bool
	Regex       bool
	Value       interface{}
}

func (MatchAttribute) Match 

func (m MatchAttribute) Match(value interface{}) (matches bool)

func (MatchAttribute) String 

func (m MatchAttribute) String() string

type MatchAttributes 

type MatchAttributes []MatchAttribute

type MultiError 

type MultiError struct {
	Subject string
	// contains filtered or unexported fields
}

func (*MultiError) Add 

func (m *MultiError) Add(e error)

func (*MultiError) Error 

func (m *MultiError) Error() string

func (*MultiError) HasErrors 

func (m *MultiError) HasErrors() bool

type OutputLine 

type OutputLine struct {
	Host   *Host
	Stderr bool
	Data   []byte
}

type OutputMode 

type OutputMode int
const (
	OutputTail OutputMode = iota
	OutputPerhost
	OutputInline
	OutputAll
)

type ProgressMessage 

type ProgressMessage struct {
	Host   *Host
	State  ProgressState
	Result *Result
}

type ProgressState 

type ProgressState int
const (
	Queued ProgressState = iota
	Waiting
	Running
	Finished
)

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

func NewRegistry 

func NewRegistry(dataDir, cacheDir string) *Registry

func (*Registry) AddMagicProvider 

func (r *Registry) AddMagicProvider(p HostProvider)

func (*Registry) AddProvider 

func (r *Registry) AddProvider(p HostProvider)

func (*Registry) InvalidateCache 

func (r *Registry) InvalidateCache()

func (*Registry) KeepCaches added in v0.10.0 

func (r *Registry) KeepCaches()

func (*Registry) LoadHostKeys added in v0.11.0 

func (r *Registry) LoadHostKeys(ctx context.Context, lm LoadingMessage) error

func (*Registry) LoadHosts 

func (r *Registry) LoadHosts(ctx context.Context, lm LoadingMessage) error

func (*Registry) LoadMagicProviders 

func (r *Registry) LoadMagicProviders()

func (*Registry) LoadProviders 

func (r *Registry) LoadProviders(c *viper.Viper) error

func (*Registry) Search added in v0.10.0 

func (r *Registry) Search(hostnameGlob string, attributes MatchAttributes, sampled []string, count int) *HostSet

func (*Registry) Settings 

func (r *Registry) Settings() (string, map[string]interface{})

type Result 

type Result struct {
	Host        string
	ExitStatus  int
	Stdout      []byte
	Stderr      []byte
	Err         error
	StartTime   time.Time
	EndTime     time.Time
	ElapsedTime float64
	// contains filtered or unexported fields
}

func (Result) MarshalJSON 

func (r Result) MarshalJSON() ([]byte, error)

func (Result) String 

func (r Result) String() string

type Runner 

type Runner struct {
	// contains filtered or unexported fields
}

func NewRunner 

func NewRunner(hosts *HostSet, executor Executor) *Runner

func (*Runner) End 

func (r *Runner) End()

func (*Runner) GetTimeout added in v0.10.0 

func (r *Runner) GetTimeout() time.Duration

func (*Runner) Interrupt added in v0.11.0 

func (r *Runner) Interrupt()

func (*Runner) OnSignal added in v0.11.0 

func (r *Runner) OnSignal(s os.Signal, f func())

func (*Runner) Run 

func (r *Runner) Run(command string, pc chan ProgressMessage, oc chan OutputLine) (*HistoryItem, error)

func (*Runner) SetConnectTimeout 

func (r *Runner) SetConnectTimeout(t time.Duration)

func (*Runner) SetHostTimeout 

func (r *Runner) SetHostTimeout(t time.Duration)

func (*Runner) SetParallel 

func (r *Runner) SetParallel(p int)

func (*Runner) SetSplay 

func (r *Runner) SetSplay(t time.Duration)

func (*Runner) SetTimeout 

func (r *Runner) SetTimeout(t time.Duration)

func (*Runner) Settings 

func (r *Runner) Settings() (string, map[string]interface{})

type SettingsFunc 

type SettingsFunc func() (string, map[string]interface{})

type SimpleUI 

type SimpleUI struct {
	// contains filtered or unexported fields
}

func NewSimpleUI 

func NewSimpleUI(hosts *HostSet) *SimpleUI

func (*SimpleUI) BindLogrus 

func (ui *SimpleUI) BindLogrus()

func (*SimpleUI) End 

func (ui *SimpleUI) End()

func (*SimpleUI) LoadingMessage 

func (ui *SimpleUI) LoadingMessage(what string, done bool, err error)

func (*SimpleUI) OutputChannel 

func (ui *SimpleUI) OutputChannel() chan OutputLine

func (*SimpleUI) PrintHistoryItem 

func (ui *SimpleUI) PrintHistoryItem(hi *HistoryItem)

func (*SimpleUI) PrintHostList 

func (ui *SimpleUI) PrintHostList(opts HostListOptions)

func (*SimpleUI) PrintSettings 

func (ui *SimpleUI) PrintSettings(funcs ...SettingsFunc)

func (*SimpleUI) ProgressChannel 

func (ui *SimpleUI) ProgressChannel(deadline time.Time) chan ProgressMessage

func (*SimpleUI) SetOutputMode 

func (ui *SimpleUI) SetOutputMode(o OutputMode)

func (*SimpleUI) SetOutputTimestamp 

func (ui *SimpleUI) SetOutputTimestamp(e bool)

func (*SimpleUI) SetPagerEnabled 

func (ui *SimpleUI) SetPagerEnabled(e bool)

func (*SimpleUI) Settings 

func (ui *SimpleUI) Settings() (string, map[string]interface{})

func (*SimpleUI) Sync 

func (ui *SimpleUI) Sync()

func (*SimpleUI) Write 

func (ui *SimpleUI) Write(msg []byte) (int, error)

type TimeoutError 

type TimeoutError struct {
	Message string
}

func (TimeoutError) Error 

func (e TimeoutError) Error() string

type UI 

type UI interface {
	PrintHistoryItem(hi *HistoryItem)
	PrintHostList(opts HostListOptions)
	PrintSettings(...SettingsFunc)
	SetOutputMode(OutputMode)
	SetOutputTimestamp(bool)
	SetPagerEnabled(bool)
	Write([]byte) (int, error)
	Sync()
	End()
	LoadingMessage(what string, done bool, err error)
	OutputChannel() chan OutputLine
	ProgressChannel(deadline time.Time) chan ProgressMessage
	BindLogrus()
	Settings() (string, map[string]interface{})
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
herd
herd-provider-aws
herd-provider-azure
herd-provider-consul
herd-provider-example
herd-provider-google
herd-provider-prometheus
herd-provider-puppet
herd-provider-tailscale
herd-provider-transip
provider
aws
azure
cache
consul
example
google
http
json
known_hosts
plain
plugin
plugin/common
plugin/server
prometheus
puppet
putty
tailscale
transip
parser

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.