Documentation
¶
Overview ¶
Package rexec runs external commands locally or remotely. It wraps os/exec and golang.org/x/crypto/ssh with a simple interface.
The key types are:
- Command: a struct that represents a command to run.
- Executor: an interface that runs a Command. available executors are LocalExecutor, ShellExecutor, ImmediateSshExecutor, and KeepAliveSshExecutor.
- ExecutorFactory: a struct that creates an Executor. This is not necessary, you can create a literal Executor directly.
Everything is designed to be friendly to marshal and unmarshal to/from JSON or other formats. Thus, basically all types are created with struct literals.
Index ¶
- Variables
- func DialSsh(config *SshClientConfig) (*ssh.Client, error)
- type Command
- type ExecuteCloser
- type Executor
- type ExecutorFactory
- type ImmediateSshExecutor
- type KeepAliveSshExecutor
- type LocalExecutor
- type ManagedIO
- type ShellExecutor
- type SshAuth
- type SshClientConfig
- type SshHostKeyCheckConfig
- type SshKeepAliveConfig
Examples ¶
- ExecutorFactory.Executor
- ExecutorFactory.Executor (FromJson)
- ImmediateSshExecutor.Execute
- ImmediateSshExecutor.Execute (Cancel)
- ImmediateSshExecutor.Execute (Timeout)
- KeepAliveSshExecutor.Execute
- LocalExecutor.Execute
- NewManagedIO
- NewSshAuth
- ShellExecutor.Execute (Bash)
- ShellExecutor.Execute (Ssh)
- SshAuth.AuthMethod
Constants ¶
This section is empty.
Variables ¶
var ( WorkdirDangerous = []string{"\n", "\t", "\r", "\b", " ", ";", "&", "|", "<", ">", "`", "(", ")", "{", "}", "[", "]", "$", "~"} EnvDangerous = WorkdirDangerous CommandDangerous = []string{":(){ :|:& };:"} )
Dangerous substrings that should not be present in the command, workdir, or env. These are used to prevent injection attacks.
var ( ErrEmptyCommand = fmt.Errorf("command is empty") ErrContainsDangerous = fmt.Errorf("contains dangerous string") )
shellCmd Validate() errors.
var ( ErrNilCommand = errors.New("nil command") ErrParseCommand = errors.New("failed to parse command") ErrInvalidCommand = errors.New("invalid command") ErrStartedCommand = errors.New("command has already been executed") ErrBadSshConfig = errors.New("bad SSH client configuration") ErrInternalError = errors.New("internal error") // should not happen, means a bug of code logic )
errors that Executor.Execute may return.
var ( ErrExecutorNotSet = fmt.Errorf("no executor is properly set") ErrMultipleExecutors = fmt.Errorf("multiple executors are set") ErrNilExecutor = fmt.Errorf("executor is nil") ErrExecutorBadConfig = fmt.Errorf("executor has bad configuration") )
ExecutorFactory errors
var ( ErrSshAuthMutex = fmt.Errorf("exactly one of Password, PrivateKey, PrivateKeyPath must be set or use NewSshAuth() to set a custom auth method") ErrSshAuthEmptyPassword = fmt.Errorf("password is empty") ErrSshAuthEmptyPrivateKey = fmt.Errorf("private key is empty") )
SshAuth errors that can be returned by Prepare().
var ( // ErrAlreadyClosed is returned when calling Close() on an already closed client. ErrAlreadyClosed = fmt.Errorf("already closed") )
keep-alive ssh client errors
var Logger *slog.Logger
Logger is the logger used by the rexec package.
The logging is disabled by default by setting it to a null logger that discards all logs.
Callers can assign a different logger to this variable to enable logging:
rexec.Logger = slog.Default().With("pkg", "rexec")
var MinSshKeepAliveInterval = 1 * time.Second
MinSshKeepAliveInterval is the minimum interval between keep-alive. This is used as the minimum return value for the interval() function.
Functions ¶
Types ¶
type Command ¶
type Command struct {
// command to run on the remote host. with arguments joined by space.
Command string
// workdir is the working directory to run the command in.
Workdir string
// env is the environment variables to set for the command.
Env map[string]string
Stdin io.Reader
Stdout io.Writer
Stderr io.Writer
Status int
// contains filtered or unexported fields
}
Command is a command to run.
It includes a command (with arguments joined by space), and optional workdir and env variables to set before running the command.
Stdin, Stdout, and Stderr are the standard input, output, and error of the command. It is recommended to set all of them before running the command; otherwise, the behavior depends on the executor. The Validate() method will set the default values if they are nil.
It is designed to do the same thing as following shell command:
cd Workdir && \ export Env.key=Env.value && \ Command < Stdin > Stdout 2> Stderr
However, the actual behavior may vary depending on the executor.
func (*Command) ShellString ¶
ShellString returns a combined command line to run on a shell, which cd to the workdir, sets the env variables, and runs the command:
"cd <workdir> && export <env_key>=<env_val> && export ... && <command>"
It is recommended to call Validate() before calling this function to ensure the command is not injected.
type ExecuteCloser ¶
ExecuteCloser is an interface that combines Executor and Closer.
ExecutorFactory will create executors that implement this interface.
type Executor ¶
type Executor interface {
// Execute implements the execution of a command.
//
// A typical implementation of Execute should:
// 0. Fast fail if the context is done.
// 1. Fast fail if the command is nil.
// 2. Fast fail if the command has already been executed.
// 3. Set the command status to -1.
// 4. Validate the command.
// 5. Prepare the command, make the proc/client/session/... to execute the command.
// 6. Start the command in another goroutine.
// 7. Wait for the command to finish in the main goroutine.
// 8. set status (exit code) of the command. (prefer to do this in a defer statement placing at 3~5 as early as possible)
// 9. return the error.
Execute(ctx context.Context, cmd *Command) error
}
Executor executes given command.
type ExecutorFactory ¶
type ExecutorFactory struct {
Local *LocalExecutor
Shell *ShellExecutor
ImmediateSsh *ImmediateSshExecutor
KeepAliveSsh *KeepAliveSshExecutor
}
ExecutorFactory is a factory for creating executors. It helps caller to create an Executor without programming the exact type.
It includes the configuration for the Local, Shell, ImmediateSsh, and KeepAliveSsh executors. Exactly one of these fields must be set to a non-nil value. ExecutorFactory.Executor() will create a new corresponding Executor based on this non-nil fields.
Executor literals are not flexible to the executor type (e.g. ShellExecutor here):
executor := &rexec.ShellExecutor{ShellPath: "/bin/sh", ShellArgs: []string{"-c"}}
With ExecutorFactory, the type of executor is determined by the non-nil field, it is easier to change (by configuration file, for example).
executor, _ := rexec.ExecutorFactory{
Shell: &ShellExecutor{ShellPath: "/bin/sh", ShellArgs: []string{"-c"}},
}.Executor()
func (ExecutorFactory) Executor ¶
func (f ExecutorFactory) Executor() (ExecuteCloser, error)
Executor creates the corresponding Executor.
It returns an error if no executor is properly set, or multiple executors are set.
Example ¶
// Create an ExecutorFactory with LocalExecutor
f := ExecutorFactory{
Local: &LocalExecutor{},
}
// Get the Executor
executor, err := f.Executor()
if err != nil {
panic(err)
}
defer executor.Close()
// Use the Executor
err = executor.Execute(context.Background(), &Command{
Command: "echo hello",
Stdout: os.Stdout,
})
if err != nil {
panic(err)
}
Output: hello
Example (FromJson) ¶
// Create an ExecutorFactory with ShellExecutor
jsonConfig := []byte(`{
"Shell": {
"ShellPath": "/bin/sh",
"ShellArgs": ["-c"]
}
}`)
var f ExecutorFactory
if err := json.Unmarshal(jsonConfig, &f); err != nil {
panic(err)
}
// Get the Executor
executor, err := f.Executor()
if err != nil {
panic(err)
}
defer executor.Close()
// Use the Executor
err = executor.Execute(context.Background(), &Command{
Command: "echo hello",
Stdout: os.Stdout,
})
if err != nil {
panic(err)
}
Output: hello
type ImmediateSshExecutor ¶
type ImmediateSshExecutor struct {
Config *SshClientConfig
}
ImmediateSshExecutor is an SSH Executor based on golang.org/x/crypto/ssh that dials the remote host immediately each time it is called to Execute(cmd) and closes the connection immediately after the command is finished.
It's safe to reuse the same ImmediateSshExecutor for multiple commands concurrently. But keep in mind that the connections won't be reused between commands.
func (*ImmediateSshExecutor) Close ¶
func (e *ImmediateSshExecutor) Close() error
func (*ImmediateSshExecutor) Execute ¶
func (e *ImmediateSshExecutor) Execute(ctx context.Context, cmd *Command) error
Example ¶
executor := &ImmediateSshExecutor{Config: &SshClientConfig{
Addr: "localhost:24622",
User: "root",
Auth: []SshAuth{
{PrivateKeyPath: "./testsshd/testsshd.id_rsa"},
},
TimeoutSeconds: 5,
HostKeyCheck: ignoreHostKeyCheck,
}}
ctx := context.Background()
var stdin = bytes.NewReader([]byte("stdin"))
var stdout bytes.Buffer
cmd := &Command{
Command: "echo $ENV1 $ENV2 from $(pwd) and $(cat -)",
Workdir: "/usr",
Env: map[string]string{
"ENV1": "hello",
"ENV2": "world",
},
Stdin: stdin,
Stdout: &stdout,
}
err := executor.Execute(ctx, cmd)
if err != nil {
fmt.Printf("error: %v", err)
}
fmt.Printf("stdout: %q", stdout.String())
Output: stdout: "hello world from /usr and stdin\n"
Example (Cancel) ¶
executor := &ImmediateSshExecutor{Config: &SshClientConfig{
Addr: "localhost:24622",
User: "root",
Auth: []SshAuth{
{PrivateKeyPath: "./testsshd/testsshd.id_rsa"},
},
TimeoutSeconds: 5,
HostKeyCheck: ignoreHostKeyCheck,
}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
var stdout bytes.Buffer
var stderr bytes.Buffer
cmd := &Command{
Command: "sleep 10; echo hello",
Stdout: &stdout,
Stderr: &stderr,
}
time.AfterFunc(2*time.Second, cancel)
err := executor.Execute(ctx, cmd)
fmt.Printf("error: %v\n", err)
fmt.Printf("stdout: %q\n", stdout.String())
fmt.Printf("stderr: %q\n", stderr.String())
Output: error: context canceled stdout: "" stderr: ""
Example (Timeout) ¶
executor := &ImmediateSshExecutor{Config: &SshClientConfig{
Addr: "localhost:24622",
User: "root",
Auth: []SshAuth{
{PrivateKeyPath: "./testsshd/testsshd.id_rsa"},
},
TimeoutSeconds: 5,
HostKeyCheck: ignoreHostKeyCheck,
}}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
var stdout bytes.Buffer
var stderr bytes.Buffer
cmd := &Command{
Command: "sleep 10; echo hello",
Stdout: &stdout,
Stderr: &stderr,
}
err := executor.Execute(ctx, cmd)
fmt.Printf("error: %v\n", err)
fmt.Printf("stdout: %q\n", stdout.String())
fmt.Printf("stderr: %q\n", stderr.String())
Output: error: context deadline exceeded stdout: "" stderr: ""
type KeepAliveSshExecutor ¶
type KeepAliveSshExecutor struct {
Config *SshClientConfig
// contains filtered or unexported fields
}
KeepAliveSshExecutor is an SSH Executor based on golang.org/x/crypto/ssh that dials the remote host once and keeps the connection alive until the executor is Closed.
It creates a new session for each command to execute. It's safe to reuse the same KeepAliveSshExecutor for multiple commands concurrently.
func (*KeepAliveSshExecutor) Close ¶
func (e *KeepAliveSshExecutor) Close() error
Close the SSH client and stops the keep-alive loop.
func (*KeepAliveSshExecutor) Execute ¶
func (e *KeepAliveSshExecutor) Execute(ctx context.Context, cmd *Command) error
Execute the command on the SSH client.
It will dial the remote host if the connection is not established yet. Or it will reuse the existing keeping-alive connection. New session will be created (within the same connection) for each command.
The connection will be kept alive until Close() is called.
Example ¶
This example demonstrates using KeepAliveSshExecutor for periodic tasks.
executor := &KeepAliveSshExecutor{Config: &SshClientConfig{
Addr: "localhost:24622",
User: "root",
Auth: []SshAuth{
{PrivateKeyPath: "./testsshd/testsshd.id_rsa"},
},
TimeoutSeconds: 5,
HostKeyCheck: ignoreHostKeyCheck,
KeepAlive: SshKeepAliveConfig{
IntervalSeconds: 10,
IncrementSeconds: 3,
},
}}
defer executor.Close() // remember to close the KeepAliveSshExecutor
// for demonstration, we only run the loop for 3 times.
ctx, cancel := context.WithTimeout(context.Background(), 16*time.Second)
defer cancel()
ticker := time.NewTicker(5 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done(): // cancel at the 16th second
return
case <-ticker.C: // tick at the 5th, 10th, and 15th seconds
cmd := &Command{
Command: "echo T",
}
managedIO := NewManagedIO()
managedIO.Hijack(cmd)
err := executor.Execute(ctx, cmd)
if err != nil {
panic(err)
}
fmt.Printf("stdout: %q\n", managedIO.Stdout.String())
}
}
Output: stdout: "T\n" stdout: "T\n" stdout: "T\n"
type LocalExecutor ¶
type LocalExecutor struct{}
LocalExecutor runs command with os/exec on the local machine.
func (*LocalExecutor) Close ¶
func (e *LocalExecutor) Close() error
func (*LocalExecutor) Execute ¶
func (e *LocalExecutor) Execute(ctx context.Context, cmd *Command) error
Example ¶
executor := &LocalExecutor{}
ctx := context.Background()
var stdout bytes.Buffer
cmd := &Command{
Command: "echo hello",
Stdout: &stdout,
}
err := executor.Execute(ctx, cmd)
if err != nil {
fmt.Printf("error: %v", err)
}
fmt.Printf("stdout: %q", stdout.String())
Output: stdout: "hello\n"
type ManagedIO ¶
ManagedIO is a bundle of bytes.Buffer that can be used as the standard input, output, and error of a Command.
The zero value for ManagedIO is NOT ready to use. Use NewManagedIO or NewCombinedOutputManagedIO to create a correct instance, or assign the buffers manually (never nil) before using it.
func NewCombinedOutputManagedIO
deprecated
func NewCombinedOutputManagedIO() *ManagedIO
Deprecated: this is buggy. The output maybe lost. Do not use it.
NewCombinedOutputManagedIO creates a new ManagedIO with a single buffer for both Stdout and Stderr.
func NewManagedIO ¶
func NewManagedIO() *ManagedIO
NewManagedIO creates a new ManagedIO with empty buffers for Stdin, Stdout, and Stderr respectively.
Example ¶
// create a new Command
cmd := &Command{
Command: "cat -", // this command reads from stdin and writes to stdout
}
// hijack the command's IO
m := NewManagedIO()
m.Hijack(cmd)
// write to the hijacked stdin
m.Stdin.Write([]byte("hello"))
// execute the command
executor := &LocalExecutor{}
err := executor.Execute(context.Background(), cmd)
if err != nil {
panic(err)
}
// read from the hijacked stdout
out, err := io.ReadAll(m.Stdout)
if err != nil {
panic(err)
}
fmt.Println(string(out))
Output: hello
func (*ManagedIO) Hijack ¶
Hijack replaces the Stdin, Stdout, and Stderr of the Command with the ManagedIO's buffers.
Writing to the Stdin buffer of ManagedIO will write to the Stdin of the Command. Reading from the Stdout and Stderr buffer of the ManagedIO will get the Stdout and Stderr of the Command.
It also starts goroutines to copy the old std IO (if exists) from/to the buffers so that the caller can still read/write to the original reader/writer.
type ShellExecutor ¶
ShellExecutor is an Executor that runs commands on a local `shell -c` or ssh command.
- sh <sh-args> -c "command args..."
- ssh <ssh-args> "command args..."
The shell will be run with os/exec.
func (*ShellExecutor) Close ¶
func (e *ShellExecutor) Close() error
func (*ShellExecutor) Execute ¶
func (e *ShellExecutor) Execute(ctx context.Context, cmd *Command) error
Example (Bash) ¶
executor := &ShellExecutor{
ShellPath: "/bin/bash",
ShellArgs: []string{"-c"},
}
ctx := context.Background()
var stdout bytes.Buffer
cmd := &Command{
Command: "echo $TEST_ENV from $(pwd)",
Workdir: "/usr",
Env: map[string]string{
"TEST_ENV": "hello",
},
Stdout: &stdout,
}
err := executor.Execute(ctx, cmd)
if err != nil {
fmt.Printf("error: %v", err)
}
fmt.Printf("stdout: %q", stdout.String())
Output: stdout: "hello from /usr\n"
Example (Ssh) ¶
executor := &ShellExecutor{
ShellPath: "ssh",
ShellArgs: []string{
"-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", "-q",
"-o", "PasswordAuthentication=no",
"-i", "./testsshd/testsshd.id_rsa",
"-p", "24622", "root@localhost",
},
}
ctx := context.Background()
var stdin = bytes.NewReader([]byte("hello from stdin"))
var stdout bytes.Buffer
cmd := &Command{
Command: "cat -", // read from stdin
Stdin: stdin,
Stdout: &stdout,
}
err := executor.Execute(ctx, cmd)
if err != nil {
fmt.Printf("error: %v", err)
}
fmt.Printf("stdout: %q", stdout.String())
Output: stdout: "hello from stdin"
type SshAuth ¶
type SshAuth struct {
// Password is the password to use for authentication.
Password string
// PrivateKey is the private key to use for authentication.
PrivateKey string
// PrivateKeyPath is the path to the private key to use for authentication.
PrivateKeyPath string
// Retries is the number of times to retry the connection for this auth method.
// If Retries < 0, will retry indefinitely.
Retries int
// contains filtered or unexported fields
}
SshAuth wraps the ssh.AuthMethod to make it easier to bind values from configuration files or databases.
It's OK to construct it manually, by
auth := &SshAuth{Password: "password"}
Set exactly one of Password, PrivateKey, PrivateKeyPath field to authenticate with RFC 4252 password or public key authentication.
For other authentication methods, use NewSshAuth() to set a custom auth method.
func NewSshAuth ¶
func NewSshAuth(authMethod ssh.AuthMethod) *SshAuth
NewSshAuth returns a new SshAuth wrapping the given underlying ssh.AuthMethod. It is useful to set a custom auth method that is not covered by Password, PrivateKey, or PrivateKeyPath.
Example:
auth := NewSshAuth(ssh.PasswordCallback(func() (string, error) {
return "password", nil
})
Example ¶
auth := NewSshAuth(ssh.Password("root"))
// Prepare the auth method
if err := auth.Prepare(); err != nil {
log.Fatalf("unable to prepare auth: %v", err)
}
cli, err := ssh.Dial("tcp", "localhost:24622", &ssh.ClientConfig{
User: "root",
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Auth: []ssh.AuthMethod{
auth.AuthMethod(), // AuthMethod is ready to call after Prepare()
},
})
if err != nil {
log.Fatalf("unable to dial: %v", err)
}
s, err := cli.NewSession()
if err != nil {
log.Fatalf("unable to create session: %v", err)
}
r, err := s.Output("echo hello")
if err != nil {
log.Fatalf("unable to run command: %v", err)
}
fmt.Println(string(r))
Output: hello
func (*SshAuth) AuthMethod ¶
func (a *SshAuth) AuthMethod() ssh.AuthMethod
AuthMethod returns the prepared ssh.AuthMethod. It panics if Prepare() was not called before.
Example ¶
An example to use SshAuth.AuthMethod with golang.org/x/crypto/ssh.Dial().
Prerequisites:
cd ./testsshd && docker compose -f testsshd-docker-compose.yml up
To start a sshd server on localhost:24622 (see testsshd/README.md for more details).
auth := &SshAuth{
PrivateKeyPath: "./testsshd/testsshd.id_rsa",
}
// Prepare the auth method
if err := auth.Prepare(); err != nil {
log.Fatalf("unable to prepare auth: %v", err)
}
cli, err := ssh.Dial("tcp", "localhost:24622", &ssh.ClientConfig{
User: "root",
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Auth: []ssh.AuthMethod{
auth.AuthMethod(), // AuthMethod is ready to call after Prepare()
},
})
if err != nil {
log.Fatalf("unable to dial: %v", err)
}
s, err := cli.NewSession()
if err != nil {
log.Fatalf("unable to create session: %v", err)
}
r, err := s.Output("echo hello")
if err != nil {
log.Fatalf("unable to run command: %v", err)
}
fmt.Println(string(r))
Output: hello
type SshClientConfig ¶
type SshClientConfig struct {
// Addr is the address of the remote host: "host:port".
Addr string
// User contains the username to authenticate as.
User string
// Auth contains the authentication methods to use.
Auth []SshAuth
// TimeoutSeconds is the maximum amount of time for the TCP connection to
// establish. A Timeout of zero means no timeout.
TimeoutSeconds int
// KeepAlive contains the configuration for the SSH client to keep the
// connection alive.
// As for now, only KeepAliveSshExecutor supports this.
KeepAlive SshKeepAliveConfig
// HostKeyCheck is the configuration for host key checking.
// If nil, host key checking is disabled (insecure, do not use in production).
// If not nil, host key checking is enabled according to the configuration.
HostKeyCheck *SshHostKeyCheckConfig
}
SshClientConfig contains the configuration for the SSH client.
It is a wrapper around ssh.ClientConfig plus the address of the remote host to make it easier to bind values from sources like configuration files.
func (SshClientConfig) Timeout ¶
func (c SshClientConfig) Timeout() time.Duration
Timeout converts the TimeoutSeconds to time.Duration.
type SshHostKeyCheckConfig ¶
type SshHostKeyCheckConfig struct {
// FixedHostKey is an "ssh-ed25519 ..." you got from
// `ssh-keyscan <server-ip>` (excluding the IP address part)
FixedHostKey string
// KnownHostsPath is a list of paths to the known_hosts files,
// usually ~/.ssh/known_hosts and /etc/ssh/ssh_known_hosts
KnownHostsPath []string
// InsecureIgnore can be set to true to disable host key checking.
// Insecure, do not use in production.
InsecureIgnore bool
}
SshHostKeyCheckConfig contains the configuration for host key checking.
One of FixedHostKey or KnownHostsPath should be set to enable host key checking.
A nil/zero config means using the default known_hosts, which trys to read from ~/.ssh/known_hosts and /etc/ssh/ssh_known_hosts if exist, or it denies all host keys (which makes all connections fail).
If multiple fields are set, the priority is:
FixedHostKey > KnownHostsPath
That is, the first non-empty field will be used for host key checking, and the rest will be ignored.
type SshKeepAliveConfig ¶
type SshKeepAliveConfig struct {
IntervalSeconds int // the initial interval between keep-alive, in seconds
IncrementSeconds int // the increment of interval between keep-alive, in seconds
}
SshKeepAliveConfig contains the configuration for the SSH client to keep the connection alive.
The final interval between keep-alive will be:
max(IntervalSeconds + IncrementSeconds * retries, MinSshKeepAliveInterval)
Special cases:
- If IntervalSeconds < 0, it will be defaulted to 0.
- If IncrementSeconds == 0, the interval will be fixed.
- If IncrementSeconds < 0, the interval will be decreased.
- If the calculated interval is less than MinSshKeepAliveInterval, it will be defaulted to MinSshKeepAliveInterval.