bichme

README

bichme

bichme (/biːtʃˈmɛ/) is utility for quick and dirty command execution on multiple machines at once.

Yes, you have Ansible for configuration management. Loki and Kibana for logs. Grafana for metrics. Prometheus alerts for when things go sideways. A perfectly crafted CI/CD pipeline. Infrastructure as code. GitOps. The works.

And yet here you are, mass-grepping 947 servers for that one config line you're not sure actually got deployed. Sometimes you just need to run uptime or check free space with df -h / on everything and call it monitoring. Or maybe you want to do a mass-upgrade for the latest CVE with a domain and logo.

bichme connects to multiple servers via SSH in parallel, executes commands or scripts, transfer files, and aggregates the output. No YAML. No inventory files. No plugins. Just a list of hosts and a command.

Installation

go install vld.bg/bichme/cmd/bichme@latest

Quick Start

Create a file with your target hosts (one per line):

# servers.txt
web01.example.com
web02.example.com
db01.example.com:2222  # custom port

Run a command on all of them:

bichme shell servers.txt uptime

Or upload and execute a script:

bichme exec servers.txt ./deploy.sh

Commands

shell

Run a shell command on multiple machines.

bichme shell <servers-file> <command> [flags]

Example:

bichme shell servers.txt df -h /
bichme shell servers.txt 'systemctl status nginx | head -5'

exec

Upload and execute a file on multiple machines. The file is transferred via SFTP and then executed.

bichme exec <servers-file> <file> [flags]

Example:

bichme exec servers.txt ./scripts/health-check.sh
bichme exec servers.txt ./deploy.sh -f config.yaml -f secrets.env

The -f flag uploads additional files alongside the main executable. Useful for configs, data files, or dependencies your script needs.

upload

Upload files to multiple machines via SFTP.

bichme upload <servers-file> <pattern>... [flags]

Example:

bichme upload servers.txt migrations/*.sql
bichme upload servers.txt package.tar.gz -o ~/deploy

download

Download files from multiple machines via SFTP. Files are stored in per-host subdirectories.

bichme download <servers-file> <pattern>... [flags]

Example:

bichme download servers.txt /var/log/*.log -o ~/logs
bichme download servers.txt '/etc/nginx/*.conf'

ping

Test SSH connectivity to multiple machines without executing any commands.

bichme ping <servers-file> [flags]

When a host's key is not yet in ~/.ssh/known_hosts, bichme collects it during the connection attempt. After all hosts have been contacted, any unknown keys are printed with their fingerprints and you are prompted once to add them all:

3 unknown host key(s) were encountered:

  web01.example.com:22  ssh-ed25519  SHA256:abc123...
  web02.example.com:22  ssh-ed25519  SHA256:def456...
  db01.example.com:22   ssh-ed25519  SHA256:ghi789...

Add these keys to ~/.ssh/known_hosts? [y/N]:

If ~/.ssh/known_hosts does not exist it is created automatically. Use --insecure to skip host key verification entirely.

Example:

bichme ping servers.txt
bichme ping servers.txt -w 50
bichme ping servers.txt --conn-timeout 5s

history

View and manage execution history.

bichme history           # list recent executions
bichme history show 1    # show details of execution #1
bichme history purge --keep 10        # keep only last 10
bichme history purge --older-than 24h # delete older than 24h
bichme history purge --all            # delete everything

Hosts File Format

One host per line. Empty lines and comments (starting with #) are ignored.

# Production web servers
web01.example.com
web02.example.com

# Database servers (custom port)
db01.example.com:5022
db02.example.com:5022

# Can also specify user inline
admin@legacy.example.com

Duplicate hosts are automatically removed.

SSH Authentication

bichme uses your SSH agent for authentication and whatever unencrypted keys it could find by reading identity files from ~/.ssh/ (id_rsa, id_ed25519, etc.).

eval $(ssh-agent)
ssh-add

Host keys are verified against ~/.ssh/known_hosts and /etc/ssh/ssh_known_hosts. Unknown hosts are handled interactively by bichme ping — see the ping section above. Use --insecure to skip verification entirely (not recommended).

Runtime Signals

Send SIGUSR1 to a running bichme process to print current execution statistics:

kill -USR1 $(pgrep bichme)

Output

Each line of output is prefixed with the hostname:

web01:  15:42:01 up 42 days,  3:21,  0 users,  load average: 0.08, 0.03, 0.01
web02:  15:42:01 up 38 days,  1:15,  0 users,  load average: 0.12, 0.08, 0.02
db01:   15:42:01 up 99 days,  8:44,  0 users,  load average: 0.45, 0.32, 0.28

At the end of execution, a summary is printed:

============== 6 ==============
 Connection failed:     1
 Execution failed:      3
 Done:                  38
===============================
 Total: 42
===============================

Examples

Basic Operations

# Check uptime across all servers
bichme shell servers.txt uptime

# Check disk usage on root partition
bichme shell servers.txt 'df -h /'

# Get memory info from all hosts
bichme shell servers.txt 'free -m'

Commands with Pipes and Quotes

When your command includes pipes, redirects, or special characters, wrap it in quotes:

# Check nginx status (first 5 lines only)
bichme shell servers.txt 'systemctl status nginx | head -5'

# Find large log files
bichme shell servers.txt 'find /var/log -size +100M -exec ls -lh {} \;'

# Count active connections
bichme shell servers.txt 'ss -tuln | grep LISTEN | wc -l'

Controlling Parallelism

# Rolling restart: one server at a time
bichme shell servers.txt 'systemctl restart myapp' -w 1

# Aggressive parallelism for quick checks
bichme shell servers.txt hostname -w 5000

Different Users and Ports

# Combine as root on non-standard SSH port
bichme shell servers.txt 'whoami' -u admin -p 2222

Timeouts and Retries

# Quick timeout for health checks
bichme shell servers.txt 'curl -s localhost:8080/health' -t 10s

# Long-running database backup
bichme exec servers.txt ./backup.sh -t 4h

# Unreliable network? More retries
bichme shell servers.txt uptime -r 10 --conn-timeout 60s

Uploading and Executing Scripts

# Simple script execution
bichme exec servers.txt ./scripts/health-check.sh

# Deploy script with config file
bichme exec servers.txt ./deploy.sh -f config.yaml

# Multiple support files
bichme exec servers.txt ./install.sh -f package.tar.gz -f settings.json

Filtering and Processing Output

bichme output can be piped through standard Unix tools:

# Sort servers by load average
bichme shell servers.txt uptime 2>/dev/null| sort -t: -k2 -rn

# Find servers with high disk usage
bichme shell servers.txt 'df -h /' | grep -E '[89][0-9]%|100%'

# Only show failures (non-zero exit)
bichme shell servers.txt 'systemctl is-active myapp' 2>&1 | grep -v '^.*: active'

# Save output for later analysis
bichme shell servers.txt 'cat /etc/os-release' > os-versions.txt

Debugging and Troubleshooting

# Verbose output shows connection details
bichme shell servers.txt uptime --verbose

# Check progress on long-running job (in another terminal)
kill -USR1 $(pgrep bichme)

# Review what you ran yesterday
bichme history
bichme history show 42

Skipping History

# Don't record sensitive operations
bichme shell servers.txt 'echo $SECRET_KEY' --history=false

License

MIT License. See LICENSE for details.

Documentation

Index

• Variables
• func Prompt(keys []PendingKey, w io.Writer, r io.Reader) (bool, error)
• func Run(ctx context.Context, servers []string, cmd string, opts Opts) error
• func Version() string
• func WriteStats(w io.Writer, archive map[*Job]error) error
• func WriteToKnownHosts(keys []PendingKey) error
• type HistoryItem
  • func ListHistory(root string) ([]HistoryItem, error)
  • func (hi HistoryItem) Delete() error
  • func (hi HistoryItem) Read(p []byte) (n int, err error)
  • func (hi HistoryItem) Summary() (succeeded, failed int)
  • func (hi HistoryItem) WriteTo(w io.Writer) (n int64, err error)
• type HostConfig
• type HostResult
• type Job
  • func (j *Job) Cleanup(ctx context.Context) error
  • func (j *Job) Close() error
  • func (j *Job) Dial(ctx context.Context) error
  • func (j *Job) Download(ctx context.Context) error
  • func (j *Job) Exec(ctx context.Context) error
  • func (j *Job) Start(ctx context.Context) error
  • func (j *Job) Upload(ctx context.Context) error
• type KeyCollector
  • func (c *KeyCollector) Callback(base ssh.HostKeyCallback) ssh.HostKeyCallback
  • func (c *KeyCollector) Keys() []PendingKey
• type Opts
• type Output
  • func NewOutput(prefix string) *Output
  • func (o *Output) Close() error
  • func (o *Output) Flush() error
  • func (o *Output) SetFile(f io.WriteCloser)
  • func (o *Output) SetStdout(w io.Writer)
  • func (o *Output) Write(p []byte) (n int, err error)
• type PendingKey
• type SSHConfigResolver
• type Tasks
  • func (t *Tasks) Done() bool
  • func (t Tasks) Has(flag Tasks) bool
  • func (t *Tasks) Set(flag Tasks)
  • func (t *Tasks) Unset(flag Tasks)

Variables

var (
	ErrConnection   = errors.New("connection failed")
	ErrFileTransfer = errors.New("file transfer failed")
	ErrExecution    = errors.New("execution failed")
)

Functions

func Prompt

func Prompt(keys []PendingKey, w io.Writer, r io.Reader) (bool, error)

Prompt prints the pending keys to w and asks the user to confirm via r. Returns true if the user accepted.

func Run

func Run(ctx context.Context, servers []string, cmd string, opts Opts) error

Run parallel SSH command executions across multiple servers with automatic retry, output aggregation, and optional execution history.

func Version

func Version() string

Version of the release.

func WriteStats

func WriteStats(w io.Writer, archive map[*Job]error) error

WriteStats summary by reading the jobs' errors and performing some deductions.

func WriteToKnownHosts

func WriteToKnownHosts(keys []PendingKey) error

WriteToKnownHosts appends keys to ~/.ssh/known_hosts, creating the file (and ~/.ssh/ directory) if they do not yet exist.

Types

type HistoryItem

type HistoryItem struct {
	Path     string
	Time     time.Time
	Duration time.Duration
	Hosts    map[string]HostResult
	Files    []string
	Logs     []string
	Command  string
}

func ListHistory

func ListHistory(root string) ([]HistoryItem, error)

func (HistoryItem) Delete

func (hi HistoryItem) Delete() error

Delete the underlying state directory.

func (HistoryItem) Read

func (hi HistoryItem) Read(p []byte) (n int, err error)

Read implements io.Reader.

func (HistoryItem) Summary

func (hi HistoryItem) Summary() (succeeded, failed int)

Summary returns counts of succeeded and failed hosts.

func (HistoryItem) WriteTo

func (hi HistoryItem) WriteTo(w io.Writer) (n int64, err error)

WriteTo implements io.WriterTo.

type HostConfig

type HostConfig struct {
	Hostname      string
	Port          int
	User          string
	IdentityFiles []string
}

HostConfig holds parsed SSH settings for a single host.

type HostResult

type HostResult struct {
	Error    string        `json:"error,omitempty"`
	Tries    int           `json:"tries"`
	Duration time.Duration `json:"duration"`
}

HostResult captures the final execution state for a single host.

type Job

type Job struct {
	// contains filtered or unexported fields
}

Job represents a single task to be executed on a single host. A job holds its information while going through retries until completion or exhaustion.

func (*Job) Cleanup

func (j *Job) Cleanup(ctx context.Context) error

Cleanup removes uploaded files from the remote host.

func (*Job) Close

func (j *Job) Close() error

Close implements io.Closer. Close is idempotent; calling it multiple times returns nil after the first call.

func (*Job) Dial

func (j *Job) Dial(ctx context.Context) error

Dial connects to the remote host.

func (*Job) Download

func (j *Job) Download(ctx context.Context) error

Download files from the remote host to local directory.

func (*Job) Exec

func (j *Job) Exec(ctx context.Context) error

Exec executes the job's command, but teeing output to the history and stdout.

func (*Job) Start

func (j *Job) Start(ctx context.Context) error

Start a job to do its remaining tasks.

func (*Job) Upload

func (j *Job) Upload(ctx context.Context) error

Upload files and make sure the first one will be executable.

type KeyCollector

type KeyCollector struct {
	// contains filtered or unexported fields
}

KeyCollector accumulates unknown host keys encountered during SSH dials. It is safe for concurrent use.

func (*KeyCollector) Callback

func (c *KeyCollector) Callback(base ssh.HostKeyCallback) ssh.HostKeyCallback

Callback wraps an existing ssh.HostKeyCallback. The returned callback:

• passes through nil (key is known and matches)
• records unknown hosts in the collector and returns nil so the dial succeeds
• returns the error unchanged for changed keys (potential MITM)

func (*KeyCollector) Keys

func (c *KeyCollector) Keys() []PendingKey

Keys returns all collected unknown-host keys. Safe to call after Run returns.

type Opts

type Opts struct {
	User         string
	Port         int
	Retries      int
	Workers      int
	Files        []string
	ConnTimeout  time.Duration
	ExecTimeout  time.Duration
	History      bool
	HistoryPath  string
	UploadPath   string
	Insecure     bool
	DownloadPath string
	Tasks        Tasks
	KeyCollector *KeyCollector
}

Opts carries CLI arguments from ./cmd into Run(). Values are copied into each Job at creation time - jobs don't share this struct.

type Output

type Output struct {
	// contains filtered or unexported fields
}

Output is used to relay remove execution's output into the history (if enabled), while also teeing the output to stdout.

func NewOutput

func NewOutput(prefix string) *Output

NewOutput creates new output with given prefix, without underlaying file, printing to os.Stdout.

func (*Output) Close

func (o *Output) Close() error

Close the underlaying file (if any).

func (*Output) Flush

func (o *Output) Flush() error

Flush writes any buffered data to stdout with a trailing newline.

func (*Output) SetFile

func (o *Output) SetFile(f io.WriteCloser)

SetFile sets f as underlaying file.

func (*Output) SetStdout

func (o *Output) SetStdout(w io.Writer)

SetStdout changes where to tee the output, instead of os.Stdout. w MUST NOT be nil, otherwise Output will eventually panic on Write or Flush.

func (*Output) Write

func (o *Output) Write(p []byte) (n int, err error)

Write to underlaying file (if any) while buffering p until a newline is received in order to print to standard output. Returns the result from file's Write.

type PendingKey

type PendingKey struct {
	Host   string // "host:port"
	Remote net.Addr
	Key    ssh.PublicKey
}

PendingKey holds an unknown host key encountered during an SSH dial.

type SSHConfigResolver

type SSHConfigResolver func(alias string) HostConfig

SSHConfigResolver looks up per-host SSH config settings.

type Tasks

type Tasks int8

Tasks is a bit mask that can hold all the things a job should do.

const (
	KeepHistoryTask Tasks = 1 << iota
	ExecTask
	DownloadTask
	UploadTask
	CleanupTask
	PingTask
)

func (*Tasks) Done

func (t *Tasks) Done() bool

Done reports whether all flags from t are unset.

func (Tasks) Has

func (t Tasks) Has(flag Tasks) bool

Has reports whether flag is set in t.

func (*Tasks) Set

func (t *Tasks) Set(flag Tasks)

Set given flag into t.

func (*Tasks) Unset

func (t *Tasks) Unset(flag Tasks)

Unset given flag from t.