script

package module

v0.3.1 Latest Latest Go to latest Published: Mar 17, 2023 License: MIT Imports: 21 Imported by: 0

README ¶

This is a fork of github.com/bitfield/script

The original version combines both stdout and stderr into stdout. I needed to write a program that ignored the stderr and only captured the stdout. The program I was writing used script to scp files from a source directory to a destination inside my company, my company specifies in their /etc/ssh/sshd_conf: "Banner corporateMumboJumbo.txt" -- SSH puts this to stderr. So each scp command you'd see the corporateMumboJumbo.txt -- I just want to know if the file copied or not. So I modifed script.

To filter out stderr (Optionally you can write it to a file) you'll need to open the pipe first, then tell the pipe where to put stderr, then exec. the command:

p:= script.NewPipe()
p.SetStderr(io.Discard)
p.Exec("scp /path/to/myfile mycorpserver:/path/to/destination").Stdout()

import "github.com/rmasci/script"

What is `script`?

script is a Go library for doing the kind of tasks that shell scripts are good at: reading files, executing subprocesses, counting lines, matching strings, and so on.

Why shouldn't it be as easy to write system administration programs in Go as it is in a typical shell? script aims to make it just that easy.

Shell scripts often compose a sequence of operations on a stream of data (a pipeline). This is how script works, too.

This is one absolutely superb API design. Taking inspiration from shell pipes and turning it into a Go library with syntax this clean is really impressive.
—Simon Willison

Quick start: Unix equivalents

If you're already familiar with shell scripting and the Unix toolset, here is a rough guide to the equivalent script operation for each listed Unix command.

Unix / shell	`script` equivalent
(any program name)	`Exec()`
`[ -f FILE ]`	`IfExists()`
`>`	`WriteFile()`
`>>`	`AppendFile()`
`$*`	`Args()`
`basename`	`Basename()`
`cat`	`File()` / `Concat()`
`cut`	`Column()`
`dirname`	`Dirname()`
`echo`	`Echo()`
`grep`	`Match()` / `MatchRegexp()`
`grep -v`	`Reject()` / `RejectRegexp()`
`head`	`First()`
`find -type f`	`FindFiles`
`jq`	`JQ`
`ls`	`ListFiles()`
`sed`	`Replace()` / `ReplaceRegexp()`
`sha256sum`	`SHA256Sum()` / `SHA256Sums()`
`tail`	`Last()`
`uniq -c`	`Freq()`
`wc -l`	`CountLines()`
`xargs`	`ExecForEach()`

Some examples

Let's see some simple examples. Suppose you want to read the contents of a file as a string:

contents, err := script.File("test.txt").String()

That looks straightforward enough, but suppose you now want to count the lines in that file.

numLines, err := script.File("test.txt").CountLines()

For something a bit more challenging, let's try counting the number of lines in the file that match the string "Error":

numErrors, err := script.File("test.txt").Match("Error").CountLines()

But what if, instead of reading a specific file, we want to simply pipe input into this program, and have it output only matching lines (like grep)?

script.Stdin().Match("Error").Stdout()

Just for fun, let's filter all the results through some arbitrary Go function:

script.Stdin().Match("Error").FilterLine(strings.ToUpper).Stdout()

That was almost too easy! So let's pass in a list of files on the command line, and have our program read them all in sequence and output the matching lines:

script.Args().Concat().Match("Error").Stdout()

Maybe we're only interested in the first 10 matches. No problem:

script.Args().Concat().Match("Error").First(10).Stdout()

What's that? You want to append that output to a file instead of printing it to the terminal? You've got some attitude, mister.

script.Args().Concat().Match("Error").First(10).AppendFile("/var/log/errors.txt")

If the data is JSON, we can do better than simple string-matching. We can use JQ queries:

script.File("commits.json").JQ(".[0] | {message: .commit.message, name: .commit.committer.name}").Stdout()

Suppose we want to execute some external program instead of doing the work ourselves. We can do that too:

script.Exec("ping 127.0.0.1").Stdout()

But maybe we don't know the arguments yet; we might get them from the user, for example. We'd like to be able to run the external command repeatedly, each time passing it the next line of input. No worries:

script.Args().ExecForEach("ping -c 1 {{.}}").Stdout()

If there isn't a built-in operation that does what we want, we can just write our own:

script.Echo("hello world").Filter(func (r io.Reader, w io.Writer) error {
	n, err := io.Copy(w, r)
	fmt.Fprintf(w, "\nfiltered %d bytes\n", n)
	return err
}).Stdout()
// Output:
// hello world
// filtered 11 bytes

Notice that the "hello world" appeared before the "filtered n bytes". Filters run concurrently, so the pipeline can start producing output before the input has been fully read.

If we want to scan input line by line, we could do that with a Filter function that creates a bufio.Scanner on its input, but we don't need to:

script.Echo("a\nb\nc").FilterScan(func(line string, w io.Writer) {
	fmt.Fprintf(w, "scanned line: %q\n", line)
}).Stdout()
// Output:
// scanned line: "a"
// scanned line: "b"
// scanned line: "c"

And there's more. Much more. Read the docs for full details, and more examples.

A realistic use case

Let's use script to write a program that system administrators might actually need. One thing I often find myself doing is counting the most frequent visitors to a website over a given period of time. Given an Apache log in the Common Log Format like this:

212.205.21.11 - - [30/Jun/2019:17:06:15 +0000] "GET / HTTP/1.1" 200 2028 "https://example.com/ "Mozilla/5.0 (Linux; Android 8.0.0; FIG-LX1 Build/HUAWEIFIG-LX1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.156 Mobile Safari/537.36"

we would like to extract the visitor's IP address (the first column in the logfile), and count the number of times this IP address occurs in the file. Finally, we might like to list the top 10 visitors by frequency. In a shell script we might do something like:

cut -d' ' -f 1 access.log |sort |uniq -c |sort -rn |head

There's a lot going on there, and it's pleasing to find that the equivalent script program is quite brief:

package main

import (
	"github.com/bitfield/script"
)

func main() {
	script.Stdin().Column(1).Freq().First(10).Stdout()
}

Let's try it out with some sample data:

16 176.182.2.191
 7 212.205.21.11
 1 190.253.121.1
 1 90.53.111.17

Documentation

See pkg.go.dev for the full documentation, or read on for a summary.

Sources

These are functions that create a pipe with a given contents:

Source	Contents
`Args`	command-line arguments
`Echo`	a string
`Exec`	command output
`File`	file contents
`FindFiles`	recursive file listing
`IfExists`	do something only if some file exists
`ListFiles`	file listing (including wildcards)
`Slice`	slice elements, one per line
`Stdin`	standard input

Filters

Filters are methods on an existing pipe that also return a pipe, allowing you to chain filters indefinitely. The filters modify each line of their input according to the following rules:

Filter	Results
`Basename`	removes leading path components from each line, leaving only the filename
`Column`	Nth column of input
`Concat`	contents of multiple files
`Dirname`	removes filename from each line, leaving only leading path components
`Echo`	all input replaced by given string
`Exec`	filtered through external command
`ExecForEach`	execute given command template for each line of input
`Filter`	user-supplied function filtering a reader to a writer
`FilterLine`	user-supplied function filtering each line to a string
`FilterScan`	user-supplied function filtering each line to a writer
`First`	first N lines of input
`Freq`	frequency count of unique input lines, most frequent first
`Join`	replace all newlines with spaces
`JQ`	result of `jq` query
`Last`	last N lines of input
`Match`	lines matching given string
`MatchRegexp`	lines matching given regexp
`Reject`	lines not matching given string
`RejectRegexp`	lines not matching given regexp
`Replace`	matching text replaced with given string
`ReplaceRegexp`	matching text replaced with given string
`SHA256Sums`	SHA-256 hashes of each listed file

Note that filters run concurrently, rather than producing nothing until each stage has fully read its input. This is convenient for executing long-running comands, for example. If you do need to wait for the pipeline to complete, call Wait.

Sinks

Sinks are methods that return some data from a pipe, ending the pipeline and extracting its full contents in a specified way:

Sink	Destination	Results
`AppendFile`	appended to file, creating if it doesn't exist	bytes written, error
`Bytes`		data as `[]byte`, error
`CountLines`		number of lines, error
`Read`	given `[]byte`	bytes read, error
`SHA256Sum`		SHA-256 hash, error
`Slice`		data as `[]string`, error
`Stdout`	standard output	bytes written, error
`String`		data as `string`, error
`Wait`		none
`WriteFile`	specified file, truncating if it exists	bytes written, error

What's new

Version	New
v0.20.0	`JQ`

Contributing

See the contributor's guide for some helpful tips if you'd like to contribute to the script project.

Links

Gopher image by MariaLetta

Documentation ¶

Rendered for

Overview ¶

Package script aims to make it easy to write shell-type scripts in Go, for general system administration purposes: reading files, counting lines, matching strings, and so on.

Index ¶

type Pipe
type ReadAutoCloser
- func NewReadAutoCloser(r io.Reader) ReadAutoCloser
- func (a ReadAutoCloser) Close() error
- func (a ReadAutoCloser) Read(buf []byte) (n int, err error)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Pipe ¶

type Pipe struct {
	Reader ReadAutoCloser

	Timeout time.Duration
	// contains filtered or unexported fields
}

Pipe represents a pipe object with an associated ReadAutoCloser.

func Args ¶

func Args() *Pipe

Args creates a pipe containing the program's command-line arguments, one per line.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Args().Stdout()
	// prints command-line arguments
}

func Echo ¶

func Echo(s string) *Pipe

Echo creates a pipe containing the supplied string.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("Hello, world!").Stdout()
}

Output:

Hello, world!

func Exec ¶

func Exec(s string) *Pipe

Exec runs an external command and creates a pipe containing its combined output (stdout and stderr).

If the command had a non-zero exit status, the pipe's error status will also be set to the string "exit status X", where X is the integer exit status.

For convenience, you can get this value directly as an integer by calling ExitStatus on the pipe.

Even in the event of a non-zero exit status, the command's output will still be available in the pipe. This is often helpful for debugging. However, because String is a no-op if the pipe's error status is set, if you want output you will need to reset the error status before calling String.

Note that Exec can also be used as a filter, in which case the given command will read from the pipe as its standard input.

Example (Exitstatus) ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	p := script.Exec("echo")
	fmt.Println(p.ExitStatus())
}

Output:

0

Example (Ok) ¶

package main

import (
	"github.com/bitfield/script"
)

func main() {
	script.Exec("echo Hello, world!").Stdout()
}

Output:

Hello, world!

func File ¶

func File(name string) *Pipe

File creates a pipe that reads from the file at the specified path.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.File("testdata/hello.txt").Stdout()
}

Output:

hello world

func FindFiles ¶

func FindFiles(path string) *Pipe

FindFiles takes a directory path and creates a pipe listing all the files in the directory and its subdirectories recursively, one per line, like Unix `find -type f`. If the path doesn't exist or can't be read, the pipe's error status will be set.

Each line of the output consists of a slash-separated pathname, starting with the initial directory. For example, if the starting directory is "test", and it contains 1.txt and 2.txt:

test/1.txt test/2.txt

Example ¶

package main

import (
	"github.com/bitfield/script"
)

func main() {
	script.FindFiles("testdata/multiple_files_with_subdirectory").Stdout()
}

Output:

testdata/multiple_files_with_subdirectory/1.txt
testdata/multiple_files_with_subdirectory/2.txt
testdata/multiple_files_with_subdirectory/3.tar.zip
testdata/multiple_files_with_subdirectory/dir/.hidden
testdata/multiple_files_with_subdirectory/dir/1.txt
testdata/multiple_files_with_subdirectory/dir/2.txt

func IfExists ¶

func IfExists(filename string) *Pipe

IfExists tests whether the specified file exists, and creates a pipe whose error status reflects the result. If the file doesn't exist, the pipe's error status will be set, and if the file does exist, the pipe will have no error status. This can be used to do some operation only if a given file exists:

IfExists("/foo/bar").Exec("/usr/bin/something")

Example (False) ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.IfExists("doesntexist").Echo("found it").Stdout()
}

Example (True) ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.IfExists("./testdata/hello.txt").Echo("found it").Stdout()
}

Output:

found it

func ListFiles ¶

func ListFiles(path string) *Pipe

ListFiles creates a pipe containing the files and directories matching the supplied path, one per line. The path can be the name of a directory (`/path/to/dir`), the name of a file (`/path/to/file`), or a glob (wildcard expression) conforming to the syntax accepted by filepath.Match (for example `/path/to/*`).

ListFiles does not recurse into subdirectories (use FindFiles for this).

Example ¶

package main

import (
	"github.com/bitfield/script"
)

func main() {
	script.ListFiles("testdata/multiple_files_with_subdirectory").Stdout()
}

Output:

testdata/multiple_files_with_subdirectory/1.txt
testdata/multiple_files_with_subdirectory/2.txt
testdata/multiple_files_with_subdirectory/3.tar.zip
testdata/multiple_files_with_subdirectory/dir

func NewPipe ¶

func NewPipe() *Pipe

NewPipe returns a pointer to a new empty pipe.

func Slice ¶

func Slice(s []string) *Pipe

Slice creates a pipe containing each element of the supplied slice of strings, one per line.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	input := []string{"1", "2", "3"}
	script.Slice(input).Stdout()
}

Output:

1
2
3

func Stdin ¶

func Stdin() *Pipe

Stdin creates a pipe that reads from os.Stdin.

func (*Pipe) AppendFile ¶

func (p *Pipe) AppendFile(fileName string) (int64, error)

AppendFile appends the contents of the pipe to the specified file, and returns the number of bytes successfully written, or an error. If the file does not exist, it is created.

func (*Pipe) Basename ¶

func (p *Pipe) Basename() *Pipe

Basename reads a list of filepaths from the pipe, one per line, and removes any leading directory components from each line. So, for example, `/usr/local/bin/foo` would become just `foo`. This is the complementary operation to Dirname.

If a line is empty, Basename will produce '.'. Trailing slashes are removed. The behaviour of Basename is the same as filepath.Base (not by coincidence).

Example ¶

package main

import (
	"github.com/bitfield/script"
)

func main() {
	input := []string{
		"",
		"/",
		"/root",
		"/tmp/example.php",
		"/var/tmp/",
		"./src/filters",
		"C:/Program Files",
	}
	script.Slice(input).Basename().Stdout()
}

Output:

.
/
root
example.php
tmp
filters
Program Files

func (*Pipe) Bytes ¶

func (p *Pipe) Bytes() ([]byte, error)

Bytes returns the contents of the pipe as a []]byte, or an error.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	data, err := script.Echo("hello").Bytes()
	if err != nil {
		panic(err)
	}
	fmt.Println(data)
}

Output:

[104 101 108 108 111]

func (*Pipe) Close ¶

func (p *Pipe) Close() error

Close closes the pipe's associated reader. This is a no-op if the reader is not also a Closer.

func (*Pipe) Column ¶

func (p *Pipe) Column(col int) *Pipe

Column produces only the Nth column of each line of input, where '1' is the first column, and columns are delimited by whitespace. Specifically, whatever Unicode defines as whitespace ('WSpace=yes').

Lines containing less than N columns will be dropped altogether.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	input := []string{
		"PID   TT  STAT      TIME COMMAND",
		"  1   ??  Ss   873:17.62 /sbin/launchd",
		" 50   ??  Ss    13:18.13 /usr/libexec/UserEventAgent (System)",
		" 51   ??  Ss    22:56.75 /usr/sbin/syslogd",
	}
	script.Slice(input).Column(1).Stdout()
}

Output:

PID
1
50
51

func (*Pipe) Concat ¶

func (p *Pipe) Concat() *Pipe

Concat reads a list of file paths from the pipe, one per line, and produces the contents of all those files in sequence. If there are any errors (for example, non-existent files), these will be ignored, execution will continue, and the pipe's error status will not be set.

This makes it convenient to write programs that take a list of input files on the command line. For example:

script.Args().Concat().Stdout()

The list of files could also come from a file:

script.File("filelist.txt").Concat()

...or from the output of a command:

script.Exec("ls /var/app/config/").Concat().Stdout()

Each input file will be closed once it has been fully read. If any of the files can't be opened or read, `Concat` will simply skip these and carry on, without setting the pipe's error status. This mimics the behaviour of Unix `cat`.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	input := []string{
		"testdata/test.txt",
		"testdata/doesntexist.txt",
		"testdata/hello.txt",
	}
	script.Slice(input).Concat().Stdout()
}

Output:

This is the first line in the file.
Hello, world.
This is another line in the file.
hello world

func (*Pipe) CountLines ¶

func (p *Pipe) CountLines() (int, error)

CountLines returns the number of lines of input, or an error.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	n, err := script.Echo("a\nb\nc\n").CountLines()
	if err != nil {
		panic(err)
	}
	fmt.Println(n)
}

Output:

3

func (*Pipe) Dirname ¶

func (p *Pipe) Dirname() *Pipe

Dirname reads a list of pathnames from the pipe, one per line, and produces only the parent directories of each pathname. For example, `/usr/local/bin/foo` would become just `/usr/local/bin`. This is the complementary operation to Basename.

If a line is empty, Dirname will produce a '.'. Trailing slashes are removed, unless Dirname returns the root folder. Otherwise, the behaviour of Dirname is the same as filepath.Dir (not by coincidence).

Example ¶

package main

import (
	"github.com/bitfield/script"
)

func main() {
	input := []string{
		"",
		"/",
		"/root",
		"/tmp/example.php",
		"/var/tmp/",
		"./src/filters",
		"C:/Program Files",
	}
	script.Slice(input).Dirname().Stdout()
}

Output:

.
/
/
/tmp
/var
./src
C:

func (*Pipe) EachLine deprecated

func (p *Pipe) EachLine(process func(string, *strings.Builder)) *Pipe

EachLine calls the specified function for each line of input, passing it the line as a string, and a *strings.Builder to write its output to.

Deprecated: use FilterLine or FilterScan instead, which run concurrently and don't do unnecessary reads on the input.

Example ¶

package main

import (
	"strings"

	"github.com/rmasci/script"
)

func main() {
	script.File("testdata/test.txt").EachLine(func(line string, out *strings.Builder) {
		out.WriteString("> " + line + "\n")
	}).Stdout()
}

Output:

> This is the first line in the file.
> Hello, world.
> This is another line in the file.

func (*Pipe) Echo ¶

func (p *Pipe) Echo(s string) *Pipe

Echo produces the supplied string.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.NewPipe().Echo("Hello, world!").Stdout()
}

Output:

Hello, world!

func (*Pipe) Error ¶

func (p *Pipe) Error() error

Error returns any error present on the pipe, or nil otherwise.

func (*Pipe) Exec ¶

func (p *Pipe) Exec(command string) *Pipe

Exec runs an external command, sending it the contents of the pipe as input, and produces the command's combined output (`stdout` and `stderr`). The effect of this is to filter the contents of the pipe through the external command.

If the command had a non-zero exit status, the pipe's error status will also be set to the string "exit status X", where X is the integer exit status.

Example ¶

package main

import (
	"github.com/bitfield/script"
)

func main() {
	script.Echo("Hello, world!").Exec("tr a-z A-Z").Stdout()
}

Output:

HELLO, WORLD!

func (*Pipe) ExecForEach ¶

func (p *Pipe) ExecForEach(command string) *Pipe

ExecForEach runs the supplied command once for each line of input, and produces its combined output. The command string is interpreted as a Go template, so `{{.}}` will be replaced with the input value, for example.

If any command resulted in a non-zero exit status, the pipe's error status will also be set to the string "exit status X", where X is the integer exit status.

Example ¶

package main

import (
	"github.com/bitfield/script"
)

func main() {
	script.Echo("a\nb\nc\n").ExecForEach("echo {{.}}").Stdout()
}

Output:

a
b
c

func (*Pipe) ExitStatus ¶

func (p *Pipe) ExitStatus() int

ExitStatus returns the integer exit status of a previous command, if the pipe's error status is set, and if the error matches the pattern "exit status %d". Otherwise, it returns zero.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	p := script.Exec("echo")
	fmt.Println(p.ExitStatus())
}

Output:

0

func (*Pipe) Filter ¶

func (p *Pipe) Filter(filter func(io.Reader, io.Writer) error) *Pipe

Filter filters the contents of the pipe through the supplied function, which takes an io.Reader (the filter input) and an io.Writer (the filter output), and returns an error, which will be set on the pipe.

The filter function runs concurrently, so its goroutine will not complete until the pipe has been fully read. If you just need to make sure all concurrent filters have completed, call Wait on the end of the pipe.

Example ¶

package main

import (
	"fmt"
	"io"

	"github.com/rmasci/script"
)

func main() {
	script.Echo("hello world").Filter(func(r io.Reader, w io.Writer) error {
		n, err := io.Copy(w, r)
		fmt.Fprintf(w, "\nfiltered %d bytes\n", n)
		return err
	}).Stdout()
}

Output:

hello world
filtered 11 bytes

func (*Pipe) FilterLine ¶

func (p *Pipe) FilterLine(filter func(string) string) *Pipe

FilterLine filters the contents of the pipe, a line at a time, through the supplied function, which takes the line as a string and returns a string (the filter output). The filter function runs concurrently.

Example (Stdlib) ¶

package main

import (
	"strings"

	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc").FilterLine(strings.ToUpper).Stdout()
}

Output:

A
B
C

Example (User) ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc").FilterLine(func(line string) string {
		return "> " + line
	}).Stdout()
}

Output:

> a
> b
> c

func (*Pipe) FilterScan ¶

func (p *Pipe) FilterScan(filter func(string, io.Writer)) *Pipe

FilterScan filters the contents of the pipe, a line at a time, through the supplied function, which takes the line as a string and an io.Writer (the filtero output). The filter function runs concurrently.

Example ¶

package main

import (
	"fmt"
	"io"

	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc").FilterScan(func(line string, w io.Writer) {
		fmt.Fprintf(w, "scanned line: %q\n", line)
	}).Stdout()
}

Output:

scanned line: "a"
scanned line: "b"
scanned line: "c"

func (*Pipe) First ¶

func (p *Pipe) First(n int) *Pipe

First produces only the first N lines of input, or the whole input if there are less than N lines. If N is zero or negative, there is no output at all.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc\n").First(2).Stdout()
}

Output:

a
b

func (*Pipe) Freq ¶

func (p *Pipe) Freq() *Pipe

Freq produces only unique lines from the input, prefixed with a frequency count, in descending numerical order (most frequent lines first). Lines with equal frequency will be sorted alphabetically.

This is a common pattern in shell scripts to find the most frequently-occurring lines in a file:

sort testdata/freq.input.txt |uniq -c |sort -rn

Freq's behaviour is like the combination of Unix `sort`, `uniq -c`, and `sort -rn` used here. You can use Freq in combination with First to get, for example, the ten most common lines in a file:

script.Stdin().Freq().First(10).Stdout()

Like `uniq -c`, Freq left-pads its count values if necessary to make them easier to read:

10 apple

4 banana
2 orange
1 kumquat

Example ¶

package main

import (
	"strings"

	"github.com/rmasci/script"
)

func main() {
	input := strings.Join([]string{
		"apple",
		"orange",
		"banana",
		"banana",
		"apple",
		"orange",
		"kumquat",
		"apple",
		"orange",
		"apple",
		"banana",
		"banana",
		"apple",
		"apple",
		"orange",
		"apple",
		"apple",
		"apple",
		"apple",
	}, "\n")
	script.Echo(input).Freq().Stdout()
}

Output:

10 apple
 4 banana
 4 orange
 1 kumquat

func (*Pipe) JQ ¶ added in v0.2.0

func (p *Pipe) JQ(query string) *Pipe

JQ takes a query in the 'jq' language and applies it to the input (presumed to be JSON), producing the result. An invalid query will set the appropriate error on the pipe.

The exact dialect of JQ supported is that provided by github.com/itchyny/gojq, whose documentation explains the differences between it and 'standard' JQ.

func (*Pipe) Join ¶

func (p *Pipe) Join() *Pipe

Join produces its input as a single space-separated string, which will always end with a newline.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("hello\nworld\n").Join().Stdout()
}

Output:

hello world

func (*Pipe) Last ¶

func (p *Pipe) Last(n int) *Pipe

Last produces only the last N lines of input, or the whole input if there are less than N lines. If N is zero or negative, there is no output at all.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc\n").Last(2).Stdout()
}

Output:

b
c

func (*Pipe) Match ¶

func (p *Pipe) Match(s string) *Pipe

Match produces only lines that contain the specified string.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc\n").Match("b").Stdout()
}

Output:

b

func (*Pipe) MatchRegexp ¶

func (p *Pipe) MatchRegexp(re *regexp.Regexp) *Pipe

MatchRegexp produces only lines that match the specified compiled regular expression.

Example ¶

package main

import (
	"regexp"

	"github.com/rmasci/script"
)

func main() {
	re := regexp.MustCompile("w.*d")
	script.Echo("hello\nworld\n").MatchRegexp(re).Stdout()
}

Output:

world

func (*Pipe) Read ¶

func (p *Pipe) Read(b []byte) (int, error)

Read reads up to len(b) bytes from the data source into b. It returns the number of bytes read and any error encountered. At end of file, or on a nil pipe, Read returns 0, io.EOF.

Unlike most sinks, Read does not necessarily read the whole contents of the pipe. It will read as many bytes as it takes to fill the slice.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	buf := make([]byte, 12)
	n, err := script.Echo("hello world\n").Read(buf)
	if err != nil {
		panic(err)
	}
	fmt.Println(n)
}

Output:

12

func (*Pipe) Reject ¶

func (p *Pipe) Reject(s string) *Pipe

Reject produces only lines that do not contain the specified string.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc\n").Reject("b").Stdout()
}

Output:

a
c

func (*Pipe) RejectRegexp ¶

func (p *Pipe) RejectRegexp(re *regexp.Regexp) *Pipe

RejectRegexp produces only lines that don't match the specified compiled regular expression.

Example ¶

package main

import (
	"regexp"

	"github.com/rmasci/script"
)

func main() {
	re := regexp.MustCompile("w.*d")
	script.Echo("hello\nworld\n").RejectRegexp(re).Stdout()
}

Output:

hello

func (*Pipe) Replace ¶

func (p *Pipe) Replace(search, replace string) *Pipe

Replace replaces all occurrences of the 'search' string with the 'replace' string.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("a\nb\nc\n").Replace("b", "replacement").Stdout()
}

Output:

a
replacement
c

func (*Pipe) ReplaceRegexp ¶

func (p *Pipe) ReplaceRegexp(re *regexp.Regexp, replace string) *Pipe

ReplaceRegexp replaces all matches of the specified compiled regular expression with the 'replace' string. '$' characters in the replace string are interpreted as in regexp.Expand; for example, "$1" represents the text of the first submatch.

Example ¶

package main

import (
	"regexp"

	"github.com/rmasci/script"
)

func main() {
	re := regexp.MustCompile("w.*d")
	script.Echo("hello\nworld\n").ReplaceRegexp(re, "replacement").Stdout()
}

Output:

hello
replacement

func (*Pipe) SHA256Sum ¶

func (p *Pipe) SHA256Sum() (string, error)

SHA256Sum returns the hex-encoded SHA-256 hash of its input, or an error.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	sum, err := script.Echo("hello world").SHA256Sum()
	if err != nil {
		panic(err)
	}
	fmt.Println(sum)
}

Output:

b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

func (*Pipe) SHA256Sums ¶

func (p *Pipe) SHA256Sums() *Pipe

SHA256Sums reads a list of file paths from the pipe, one per line, and produces the hex-encoded SHA-256 hash of each file. Any files that cannot be opened or read will be ignored.

Example ¶

package main

import (
	"github.com/rmasci/script"
)

func main() {
	script.Echo("testdata/test.txt").SHA256Sums().Stdout()
}

Output:

a562c9c95e2ff3403e7ffcd8508c6b54d47d5f251387758d3e63dbaaa8296341

func (*Pipe) SetError ¶

func (p *Pipe) SetError(err error)

SetError sets the specified error on the pipe.

func (*Pipe) SetStderr ¶ added in v0.2.0

func (p *Pipe) SetStderr(w io.Writer)

SetStderr stderr sets the pipe's standard error to the specified reader. If this is not set, stdout / stderr are combined. You can then write the stderr to a log file or just open /dev/null and write the output there.

func (*Pipe) Slice ¶

func (p *Pipe) Slice() ([]string, error)

Slice returns the input as a slice of strings, one element per line, or an error.

An empty pipe will produce an empty slice. A pipe containing a single empty line (that is, a single `\n` character) will produce a slice containing the empty string.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	s, err := script.Echo("a\nb\nc\n").Slice()
	if err != nil {
		panic(err)
	}
	fmt.Println(s)
}

Output:

[a b c]

func (*Pipe) Stdout ¶

func (p *Pipe) Stdout() (int, error)

Stdout writes the input to the pipe's configured standard output, and returns the number of bytes successfully written, or an error.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	n, err := script.Echo("a\nb\nc\n").Stdout()
	if err != nil {
		panic(err)
	}
	fmt.Println(n)
}

Output:

a
b
c
6

func (*Pipe) String ¶

func (p *Pipe) String() (string, error)

String returns the input as a string, or an error.

Example ¶

package main

import (
	"fmt"

	"github.com/rmasci/script"
)

func main() {
	s, err := script.Echo("hello\nworld").String()
	if err != nil {
		panic(err)
	}
	fmt.Println(s)
}

Output:

hello
world

func (*Pipe) Tee ¶ added in v0.3.0

func (p *Pipe) Tee(outFile io.Writer) *Pipe

Tee works like Unix's tee and you could output to a file, and output to Standard out. ex:

outFile,err := os.Openfile("/path/to/file", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644)
....handle error other code....
p:=script.Exec(`sqlrun -e "select * from appdata"`).Tee(&outFile)

func (*Pipe) Wait ¶

func (p *Pipe) Wait()

Wait reads the input to completion and discards it. This is mostly useful for waiting until all concurrent filter stages have finished.

func (*Pipe) WithError ¶

func (p *Pipe) WithError(err error) *Pipe

WithError sets the specified error on the pipe and returns the modified pipe.

func (*Pipe) WithReader ¶

func (p *Pipe) WithReader(r io.Reader) *Pipe

WithReader sets the pipe's input to the specified reader. If necessary, the reader will be automatically closed once it has been completely read.

func (*Pipe) WithStdout ¶

func (p *Pipe) WithStdout(w io.Writer) *Pipe

WithStdout sets the pipe's standard output to the specified reader, instead of the default os.Stdout.

func (*Pipe) WriteFile ¶

func (p *Pipe) WriteFile(fileName string) (int64, error)

WriteFile writes the input to the specified file, and returns the number of bytes successfully written, or an error. If the file already exists, it is truncated and the new data will replace the old.

type ReadAutoCloser ¶

type ReadAutoCloser struct {
	// contains filtered or unexported fields
}

ReadAutoCloser represents a pipe source that will be automatically closed once it has been fully read.

func NewReadAutoCloser ¶

func NewReadAutoCloser(r io.Reader) ReadAutoCloser

NewReadAutoCloser returns an ReadAutoCloser wrapping the supplied Reader. If the Reader is not a Closer, it will be wrapped in a NopCloser to make it closable.

func (ReadAutoCloser) Close ¶

func (a ReadAutoCloser) Close() error

Close closes the data source associated with a, and returns the result of that close operation.

func (ReadAutoCloser) Read ¶

func (a ReadAutoCloser) Read(buf []byte) (n int, err error)

Read reads up to len(buf) bytes from the data source into buf. It returns the number of bytes read and any error encountered. At end of file, Read returns 0, io.EOF. In the EOF case, the data source will be closed.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
example

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

README ¶

This is a fork of github.com/bitfield/script

What is script?

Quick start: Unix equivalents

Some examples

A realistic use case

Documentation

Sources

Filters

Sinks

What's new

Contributing

Links

Documentation ¶

Overview ¶

Index ¶

Examples ¶

Constants ¶

Variables ¶

Functions ¶

Types ¶

type Pipe ¶

func Args ¶

func Echo ¶

func Exec ¶

func File ¶

func FindFiles ¶

func IfExists ¶

func ListFiles ¶

func NewPipe ¶

func Slice ¶

func Stdin ¶

func (*Pipe) AppendFile ¶

func (*Pipe) Basename ¶

func (*Pipe) Bytes ¶

func (*Pipe) Close ¶

func (*Pipe) Column ¶

func (*Pipe) Concat ¶

func (*Pipe) CountLines ¶

func (*Pipe) Dirname ¶

func (*Pipe) EachLine deprecated

func (*Pipe) Echo ¶

func (*Pipe) Error ¶

func (*Pipe) Exec ¶

func (*Pipe) ExecForEach ¶

func (*Pipe) ExitStatus ¶

func (*Pipe) Filter ¶

func (*Pipe) FilterLine ¶

func (*Pipe) FilterScan ¶

func (*Pipe) First ¶

func (*Pipe) Freq ¶

func (*Pipe) JQ ¶ added in v0.2.0

func (*Pipe) Join ¶

func (*Pipe) Last ¶

func (*Pipe) Match ¶

func (*Pipe) MatchRegexp ¶

func (*Pipe) Read ¶

func (*Pipe) Reject ¶

func (*Pipe) RejectRegexp ¶

func (*Pipe) Replace ¶

func (*Pipe) ReplaceRegexp ¶

func (*Pipe) SHA256Sum ¶

func (*Pipe) SHA256Sums ¶

func (*Pipe) SetError ¶

func (*Pipe) SetStderr ¶ added in v0.2.0

func (*Pipe) Slice ¶

func (*Pipe) Stdout ¶

func (*Pipe) String ¶

func (*Pipe) Tee ¶ added in v0.3.0

func (*Pipe) Wait ¶

func (*Pipe) WithError ¶

func (*Pipe) WithReader ¶

func (*Pipe) WithStdout ¶

func (*Pipe) WriteFile ¶

type ReadAutoCloser ¶

func NewReadAutoCloser ¶

func (ReadAutoCloser) Close ¶

func (ReadAutoCloser) Read ¶

Source Files ¶

Directories ¶

What is `script`?