script

package module
v1.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 27, 2020 License: Apache-2.0 Imports: 13 Imported by: 12

README

script

codecov GoDoc

Package script provides helper functions to write scripts.

Inspired by https://github.com/bitfield/script, with some improvements:

  • Output between streamed commands is a stream and not loaded to memory.

  • Better representation and handling of errors.

  • Proper incocation, usage and handling of stderr of custom commands.

The script chain is represented by a Stream object. While each command in the stream is abstracted by the Command struct. This library provides basic functionality, but can be extended freely.

Examples

HelloWorld

A simple "hello world" example that creats a stream and pipe it to the stdout.

// Create an "hello world" stream and use the ToStdout method to write it to stdout.
Echo("hello world").ToStdout()

Output:

hello world
Iterate

An example that shows how to iterate scanned lines.

// Stream can be any stream, in this case we have echoed 3 lines.
stream := Echo("first\nsecond\nthird")

// To iterate over the stream lines, it is better not to read it into memory and split over the
// lines, but use the `bufio.Scanner`:
defer stream.Close()
scanner := bufio.NewScanner(stream)
for scanner.Scan() {
    fmt.Println(scanner.Text())
}

Output:

first
second
third
Through

An example that shows how to create custom commands using the Through method with a PipeFn function.

Echo("1\n2\n3").Through(PipeFn(func(r io.Reader) (io.Reader, error) {
    // Create a command that sums up all numbers in input.
    //
    // In this example we create a reader function such that the whole code will fit into the
    // example function body. A more proper and readable way to do it was to create a new
    // type with a state that implements the `io.Reader` interface.

    // Use buffered reader to read lines from input.
    buf := bufio.NewReader(r)

    // Store the sum of all numbers.
    sum := 0

    // Read function reads the next line and adds it to the sum. If it gets and EOF error, it
    // writes the sum to the output and returns an EOF.
    read := func(b []byte) (int, error) {
        // Read next line from input.
        line, _, err := buf.ReadLine()

        // if EOF write sum to output.
        if err == io.EOF {
            return copy(b, append([]byte(strconv.Itoa(sum)), '\n')), io.EOF
        }
        if err != nil {
            return 0, err
        }

        // Convert the line to a number and add it to the sum.
        if i, err := strconv.Atoi(string(line)); err == nil {
            sum += i
        }

        // We don't write anything to output, so we return 0 bytes with no error.
        return 0, nil
    }

    return readerFn(read), nil
})).ToStdout()

Output:

6

Readme created from Go doc with goreadme

Documentation

Overview

Package script provides helper functions to write scripts.

Inspired by https://github.com/bitfield/script, with some improvements:

* Output between streamed commands is a stream and not loaded to memory.

* Better representation and handling of errors.

* Proper incocation, usage and handling of stderr of custom commands.

The script chain is represented by a (`Stream`) https://godoc.org/github.com/posener/script#Stream object. While each command in the stream is abstracted by the (`Command`) https://godoc.org/github.com/posener/script#Command struct. This library provides basic functionality, but can be extended freely.

Example (HelloWorld)

A simple "hello world" example that creats a stream and pipe it to the stdout. 

// Create an "hello world" stream and use the ToStdout method to write it to stdout.
Echo("hello world").ToStdout()

Output:

hello world
Example (Iterate)

An example that shows how to iterate scanned lines. 

// Stream can be any stream, in this case we have echoed 3 lines.
stream := Echo("first\nsecond\nthird")

// To iterate over the stream lines, it is better not to read it into memory and split over the
// lines, but use the `bufio.Scanner`:
defer stream.Close()
scanner := bufio.NewScanner(stream)
for scanner.Scan() {
	fmt.Println(scanner.Text())
}

Output:

first
second
third
Example (Through)

An example that shows how to create custom commands using the `Through` method with a `PipeFn` function. 

Echo("1\n2\n3").Through(PipeFn(func(r io.Reader) (io.Reader, error) {
	// Create a command that sums up all numbers in input.
	//
	// In this example we create a reader function such that the whole code will fit into the
	// example function body. A more proper and readable way to do it was to create a new
	// type with a state that implements the `io.Reader` interface.

	// Use buffered reader to read lines from input.
	buf := bufio.NewReader(r)

	// Store the sum of all numbers.
	sum := 0

	// Read function reads the next line and adds it to the sum. If it gets and EOF error, it
	// writes the sum to the output and returns an EOF.
	read := func(b []byte) (int, error) {
		// Read next line from input.
		line, _, err := buf.ReadLine()

		// if EOF write sum to output.
		if err == io.EOF {
			return copy(b, append([]byte(strconv.Itoa(sum)), '\n')), io.EOF
		}
		if err != nil {
			return 0, err
		}

		// Convert the line to a number and add it to the sum.
		if i, err := strconv.Atoi(string(line)); err == nil {
			sum += i
		}

		// We don't write anything to output, so we return 0 bytes with no error.
		return 0, nil
	}

	return readerFn(read), nil
})).ToStdout()

Output:

6

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func AppendFile added in v1.0.3 

func AppendFile(path string) (io.WriteCloser, error)

func File 

func File(path string) (io.WriteCloser, error)

Types

type Count 

type Count struct {
	// Stream can be used to pipe the output of wc.
	Stream
	// Count the number of lines, words and chars in the input.
	Lines, Words, Chars int
}

Count represents the output of `wc` shell command.

func (Count) String 

func (c Count) String() string

type Cut 

type Cut struct {
	// Fields defines which fields will be collected to the output of the command. The fields are 1
	// based (first field is 1).
	Fields []int
	// Delim is the delimited by which the fields of each line are sparated.
	Delim []byte
}

Cut is a `Modifier` that takes selected fields from each line according to a given delimiter. The default delimiter is tab.

Shell command: `cut -d<Delim> -f<Fields>`.

func (Cut) Modify 

func (c Cut) Modify(line []byte) (modifed []byte, err error)

func (Cut) Name 

func (c Cut) Name() string

type FileInfo added in v1.0.3 

type FileInfo struct {
	// FileInfo contains information about the file.
	os.FileInfo
	// Path is the path of the file. It may be relative or absolute, depending on how the `Ls`
	// command was invoked.
	Path string
}

File contains information about a file.

type Files 

type Files struct {
	Stream
	Files []FileInfo
}

Files is a stream of a list of files. A user can either use the file list directly or the the created stream. In the stream, each line contains a path to a file.

func Ls 

func Ls(paths ...string) Files

Ls returns a stream of a list files. In the returned stream, each line will contain a path to a single file.

If the provided paths list is empty, the local directory will be listed.

The provided paths may be relative to the local directory or absolute - this will influence the format of the returned paths in the output.

If some provided paths correlate to the arguments correlate to the same file, it will also appear multiple times in the output.

If any of the paths fails to be listed, it will result in an error in the output, but the stream will still conain all paths that were successfully listed.

Shell command: `ls`.

type Grep 

type Grep struct {
	Re      *regexp.Regexp
	Inverse bool
}

Grep is a modifier that filters only line that match `Re`. If Invert was set only line that did not match the regex will be returned.

Usage: 

(<Stream object>).Modify(script.Grep{Re: <re>})

Shell command: `grep [-v <Invert>] <Re>`.

func (Grep) Modify 

func (g Grep) Modify(line []byte) (modifed []byte, err error)

func (Grep) Name 

func (g Grep) Name() string

type Modifier added in v1.0.3 

type Modifier interface {
	// Modify a line. The input of this function will always be a single line from the input of the
	// stream, without the trailing '\n'. It should return the output of the stream and should
	// append a trailing '\n' if it want it to be a line in the output.
	//
	// When EOF of input stream is met, the function will be called once more with a nil line value
	// to enable output any buffered data.
	//
	// When the return modified value is nil, the line will be discarded.
	//
	// When the returned eof value is true, the Read will return that error.
	Modify(line []byte) (modifed []byte, err error)
	// Name returns the name of the command that will represent this modifier.
	Name() string
}

Modifier modifies input lines to output. On each line of the input the Modify method is called, and the modifier can change it, omit it, or break the iteration.

type ModifyFn added in v1.0.3 

type ModifyFn func(line []byte) (modifed []byte, err error)

ModifyFn is a function for modifying input lines.

func (ModifyFn) Modify added in v1.0.3 

func (m ModifyFn) Modify(line []byte) (modifed []byte, err error)

func (ModifyFn) Name added in v1.0.3 

func (m ModifyFn) Name() string

type Pipe added in v1.0.3 

type Pipe interface {
	// Pipe gets a reader and returns another reader. A pipe may return an error and a reader
	// together.
	Pipe(stdin io.Reader) (io.Reader, error)
	// Name of pipe.
	Name() string
}

Pipe reads from a reader and returns another reader.

type PipeFn 

type PipeFn func(io.Reader) (io.Reader, error)

PipeFn is a function that implements Pipe.

func (PipeFn) Name added in v1.0.3 

func (f PipeFn) Name() string

func (PipeFn) Pipe added in v1.0.3 

func (f PipeFn) Pipe(stdin io.Reader) (io.Reader, error)

type Stream 

type Stream struct {
	// contains filtered or unexported fields
}

Stream is a chain of operations on a stream of bytes. The stdout of each operation in the stream feeds the following operation stdin. The stream object have different methods that allow manipulating it, most of them resemble well known linux commands.

A custom modifier can be used with the `Through` or with the `Modify` functions.

The stream object is created by some in this library or from any `io.Reader` using the `From` function. It can be dumped using some functions in this library, to a custom reader using the `To` method.

func Cat 

func Cat(paths ...string) Stream

Cat outputs the contents of the given files.

Shell command: cat <path>.

func Echo 

func Echo(s string) Stream

Echo writes to stdout.

Shell command: `echo <s>`

func Exec 

func Exec(cmd string, args ...string) Stream

Exec executes a command and returns a stream of the stdout of the command.

func ExecHandleStderr 

func ExecHandleStderr(stderr io.Writer, cmd string, args ...string) Stream

ExecHandleStderr executes a command, returns a stream of the stdout of the command and enable collecting the stderr of the command.

If the stderr is nil, it will be ignored.

For example, collecting the stderr to memory can be done by providing a `&bytes.Buffer` as `stderr`. Writing it to stderr can be done by providing `os.Stderr` as `stderr`. Logging it to a file can be done by providing an `os.File` as the `stderr`.

func From added in v1.0.3 

func From(name string, r io.Reader) Stream

From creates a stream from a reader.

func Stdin 

func Stdin() Stream

Stdin starts a stream from stdin.

func Writer added in v1.1.1 

func Writer(name string, writer func(io.Writer) error) Stream

Writer creates a stream from a function that writes to a writer.

func (Stream) AppendFile added in v1.0.1 

func (s Stream) AppendFile(path string) error

AppendFile appends the output of the stream to a file.

func (Stream) Close 

func (s Stream) Close() error

Close closes all the stages in the stream and return the errors that occurred in all of the stages.

func (Stream) Cut 

func (s Stream) Cut(fields ...int) Stream

Cut takes selected fields from each line. The fields are 1 based (first field is 1).

Shell command: `cut -f<Fields>`.

func (Stream) Discard 

func (s Stream) Discard() error

Discard executes the stream pipeline but discards the output.

func (Stream) Exec 

func (s Stream) Exec(cmd string, args ...string) Stream

Exec executes a command and returns a stream of the stdout of the command.

func (Stream) ExecHandleStderr 

func (s Stream) ExecHandleStderr(stderr io.Writer, cmd string, args ...string) Stream

ExecHandleStderr executes a command, returns a stream of the stdout of the command and enable collecting the stderr of the command.

If the stderr is nil, it will be ignored.

func (Stream) Grep 

func (s Stream) Grep(re *regexp.Regexp) Stream

Grep filters only line that match the given regexp.

Shell command: `grep <re>`.

func (Stream) Head 

func (s Stream) Head(n int) Stream

Head reads only the n first lines of the given reader. If n is a negative number, all lines besides last n lines will be read.

Shell command: `head -n <n>`

func (Stream) Modify 

func (s Stream) Modify(modifier Modifier) Stream

Modify applies modifier on every line of the input.

func (Stream) Read 

func (s Stream) Read(b []byte) (int, error)

Read can be used to read from the stream.

func (Stream) Sort 

func (s Stream) Sort(reverse bool) Stream

Sort returns a stream with lines ordered alphabetically.

Shell command: `wc`.

func (Stream) Tail added in v1.0.3 

func (s Stream) Tail(n int) Stream

Tail reads only the n last lines of the given reader. If n is a negative number, all lines besides the first n lines will be read.

Shell command: `tail -n <n>`

func (Stream) Through added in v1.0.3 

func (s Stream) Through(pipe Pipe) Stream

Through passes the current stream through a pipe. This function can be used to add custom commands that are not available in this library.

func (Stream) To added in v1.0.3 

func (s Stream) To(w io.Writer) error

To writes the output of the stream to an io.Writer and closes it.

func (Stream) ToFile 

func (s Stream) ToFile(path string) error

ToFile dumps the output of the stream to a file.

func (Stream) ToStdout 

func (s Stream) ToStdout() error

ToStdout pipes the stdout of the stream to screen.

func (Stream) ToString 

func (s Stream) ToString() (string, error)

ToString reads stdout of the stream and returns it as a string.

func (Stream) ToTempFile added in v1.0.1 

func (s Stream) ToTempFile() (path string, err error)

ToTempFile dumps the output of the stream to a temporary file and returns the temporary files' path.

func (Stream) Uniq 

func (s Stream) Uniq() Stream

Uniq report or omit repeated lines.

Shell command: `uniq`.

func (Stream) Wc 

func (s Stream) Wc() Count

Wc counts the number of lines, words and characters.

Shell command: `wc`.

type Uniq 

type Uniq struct {
	WriteCount bool
	// contains filtered or unexported fields
}

Uniq report or omit repeated lines.

Usage: 

<Stream>.Modify(&Uniq{...})...

Shell command: `uniq`.

func (*Uniq) Modify 

func (u *Uniq) Modify(line []byte) ([]byte, error)

func (*Uniq) Name 

func (u *Uniq) Name() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.