package module
v0.0.0-...-e7afc7f Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Dec 2, 2019 License: Apache-2.0 Imports: 4 Imported by: 1,010


go-shlex is a simple lexer for go that supports shell-style quoting,
commenting, and escaping.



Package shlex implements a simple lexer which splits input in to tokens using shell-style rules for quoting and commenting.

The basic use case uses the default ASCII lexer to split a string into sub-strings:

shlex.Split("one \"two three\" four") -> []string{"one", "two three", "four"}

To process a stream of strings:

l := NewLexer(os.Stdin)
for ; token, err := l.Next(); err != nil {
	// process token

To access the raw token stream (which includes tokens for comments):

  t := NewTokenizer(os.Stdin)
  for ; token, err := t.Next(); err != nil {
	// process token



This section is empty.


This section is empty.


func Split

func Split(s string) ([]string, error)

Split partitions a string into a slice of strings.


type Lexer

type Lexer Tokenizer

Lexer turns an input stream into a sequence of tokens. Whitespace and comments are skipped.

func NewLexer

func NewLexer(r io.Reader) *Lexer

NewLexer creates a new lexer from an input stream.

func (*Lexer) Next

func (l *Lexer) Next() (string, error)

Next returns the next word, or an error. If there are no more words, the error will be io.EOF.

type Token

type Token struct {
	// contains filtered or unexported fields

Token is a (type, value) pair representing a lexographical token.

func (*Token) Equal

func (a *Token) Equal(b *Token) bool

Equal reports whether tokens a, and b, are equal. Two tokens are equal if both their types and values are equal. A nil token can never be equal to another token.

type TokenType

type TokenType int

TokenType is a top-level token classification: A word, space, comment, unknown.

const (
	UnknownToken TokenType = iota

Classes of lexographic token

type Tokenizer

type Tokenizer struct {
	// contains filtered or unexported fields

Tokenizer turns an input stream into a sequence of typed tokens

func NewTokenizer

func NewTokenizer(r io.Reader) *Tokenizer

NewTokenizer creates a new tokenizer from an input stream.

func (*Tokenizer) Next

func (t *Tokenizer) Next() (*Token, error)

Next returns the next token in the stream.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL