shlex

README

shlex

Go implementation of Python's shlex for shell-like lexical analysis.

Install

Add the module to your Go project:

go get github.com/chhongzh/shlex@latest

Then import it:

import "github.com/chhongzh/shlex"

Usage

Split a command line into arguments

Split tokenizes a string using shell-like syntax (POSIX mode, whitespace-splitting, comments disabled by default):

package main

import (
	"fmt"

	"github.com/chhongzh/shlex"
)

func main() {
	input := `echo "hello world" 'and more'`

	tokens, err := shlex.Split(input)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%#v\n", tokens)
	// []string{"echo", "hello world", "and more"}
}

If the input has unmatched quotes or an unfinished escape sequence, Split returns an error.

Quote and join arguments

Use Quote to safely shell-escape a single string, and Join to build a command line from a slice of arguments:

args := []string{"echo", "hello world", "it's fine"}

line := shlex.Join(args)
fmt.Println(line)
// echo hello\ world 'it'"'"'s\ fine

Quote leaves "safe" shell characters unmodified and wraps other strings in single quotes, following Python shlex.quote behavior.

Advanced usage with Shlex

For more control, you can work with the Shlex type directly:

package main

import (
	"fmt"
	"io"

	"github.com/chhongzh/shlex"
)

func main() {
	lex := shlex.NewString("a && b || c")
	lex.SetPosix(true)
	lex.SetPunctuationChars("true") // split on shell punctuation like &&, ||, etc.

	for {
		tok, err := lex.GetToken()
		if err == io.EOF {
			break
		}
		if err != nil {
			panic(err)
		}
		fmt.Println(tok)
	}
	// Output:
	// a
	// &&
	// b
	// ||
	// c
}

Key configuration points on Shlex:

• SetPosix(true) enables POSIX-compatible behavior.
• SetPunctuationChars("true") or a custom string controls how punctuation is tokenized.
• Fields like whitespace, commenters, and whitespaceSplit let you fine-tune parsing.

License

MIT License. See LICENSE for details.

Documentation

Overview

Package shlex provides a lexical analyzer for simple shell-like syntaxes.

Index

• func Join(splitCommand []string) string
• func Quote(s string) string
• func Split(s string) ([]string, error)
• type Shlex
  • func New(instream io.Reader) *Shlex
  • func NewString(s string) *Shlex
  • func (s *Shlex) GetToken() (string, error)
  • func (s *Shlex) PushToken(tok string)
  • func (s *Shlex) SetPosix(posix bool)
  • func (s *Shlex) SetPunctuationChars(chars string)

Functions

func Join

func Join(splitCommand []string) string

Join returns a shell-escaped string from split_command.

func Quote

func Quote(s string) string

Quote returns a shell-escaped version of the string s.

func Split

func Split(s string) ([]string, error)

Split splits the string s using shell-like syntax.

Types

type Shlex

type Shlex struct {
	// contains filtered or unexported fields
}

Shlex is a lexical analyzer class for simple shell-like syntaxes.

func New

func New(instream io.Reader) *Shlex

New creates a new Shlex instance.

func NewString

func NewString(s string) *Shlex

NewString creates a new Shlex instance from a string.

func (*Shlex) GetToken

func (s *Shlex) GetToken() (string, error)

GetToken gets a token from the input stream.

func (*Shlex) PushToken

func (s *Shlex) PushToken(tok string)

PushToken pushes a token onto the stack.

func (*Shlex) SetPosix

func (s *Shlex) SetPosix(posix bool)

SetPosix sets the posix mode.

func (*Shlex) SetPunctuationChars

func (s *Shlex) SetPunctuationChars(chars string)

SetPunctuationChars sets the punctuation characters.