iterator

Documentation

Overview

Package iterator provides an easy API to create an iterator to read objects from a file. Depends on the following packages in go-simple-serializer.

• github.com/spatialcurrent/go-simple-serializer/pkg/jsonl
• github.com/spatialcurrent/go-simple-serializer/pkg/sv
• github.com/spatialcurrent/go-simple-serializer/pkg/tags

Index

• Variables
• type ErrInvalidFormat
  • func (e ErrInvalidFormat) Error() string
• type Iterator
  • func NewIterator(input *NewIteratorInput) (Iterator, error)
• type NewIteratorInput

Examples

• Iterator (Jsonl)
• Iterator (Tags)

Variables

var (
	ErrMissingLineSeparator = errors.New("missing line separator")
	ErrMissingType          = errors.New("missing type")
)

Types

type ErrInvalidFormat

type ErrInvalidFormat struct {
	Format string // the name of the invalid format
}

ErrInvalidFormat is used when an invalid format is provided.

func (ErrInvalidFormat) Error

func (e ErrInvalidFormat) Error() string

Error returns the error as a string.

type Iterator

type Iterator interface {
	Next() (interface{}, error) // Returns the next object or error if any.  When input is exhausted, returns (nil, io.EOF).
}

Iterator is a simple interface that supports iterating over an input object source.

Example (Jsonl)

This example shows how to use an iterate through an input of JSON Lines.

text := `
  {"a": "b"}
  {"c": "d"}
  {"e": "f"}
  false
  true
  "foo"
  "bar"
  `

it, err := NewIterator(&NewIteratorInput{
	Reader:            strings.NewReader(text),
	Format:            "jsonl",
	SkipLines:         0,
	Comment:           "",
	Trim:              true,
	SkipBlanks:        false,
	SkipComments:      false,
	KeyValueSeparator: "=",
	LineSeparator:     []byte("\n")[0],
	DropCR:            true,
})
if err != nil {
	panic(err)
}

for {
	obj, err := it.Next()
	if err != nil {
		if err == io.EOF {
			break
		}
		panic(err)
	}
	fmt.Println(obj)
}

Output:

<nil>
map[a:b]
map[c:d]
map[e:f]
false
true
foo
bar
<nil>

Example (Tags)

This example shows how to use an iterate through an input of a lines of tags.

text := `
  a=b x=y
  c=d y=z
  e=f h=i
  `

it, err := NewIterator(&NewIteratorInput{
	Reader:            strings.NewReader(text),
	Format:            "tags",
	SkipLines:         0,
	Comment:           "",
	Trim:              true,
	SkipBlanks:        false,
	SkipComments:      false,
	KeyValueSeparator: "=",
	LineSeparator:     []byte("\n")[0],
	DropCR:            true,
})
if err != nil {
	panic(err)
}

for {
	obj, err := it.Next()
	if err != nil {
		if err == io.EOF {
			break
		}
		panic(err)
	}
	fmt.Println(obj)
}

Output:

<nil>
map[a:b x:y]
map[c:d y:z]
map[e:f h:i]
<nil>

func NewIterator

func NewIterator(input *NewIteratorInput) (Iterator, error)

NewIterator returns an Iterator for the given input source, format, and other options. Supports formats:

• csv - Comma-Separated Values
• jsonl - JSON Lines
• tags - Tags (key-value pairs)
• tsv - Tab-Separated Values

type NewIteratorInput

type NewIteratorInput struct {
	Reader            io.Reader     // the underlying reader
	Format            string        // the format
	Header            []interface{} // for csv and tsv, the header.  If not given, then reads first line of stream as header.
	ScannerBufferSize int           // the initial buffer size for the scanner
	SkipLines         int           // Skip a given number of lines at the beginning of the stream.
	SkipBlanks        bool          // Skip blank lines.  If false, Next() returns a blank line as (nil, nil).  If true, Next() simply skips forward until it finds a non-blank line.
	SkipComments      bool          // Skip commented lines.  If false, Next() returns a commented line as (nil, nil).  If true, Next() simply skips forward until it finds a non-commented line.
	Comment           string        // The comment line prefix.  CSV and TSV only support single characters.  JSON Lines support any string.
	Trim              bool          // Trim each input line before parsing into an object.
	LazyQuotes        bool          // for csv and tsv, parse with lazy quotes
	Limit             int           // Limit the number of objects to read and return from the underlying stream.
	KeyValueSeparator string        // For tags, the key-value separator.
	LineSeparator     string        // For JSON Lines, the new line byte.
	DropCR            bool          // For JSON Lines, drop carriage returns at the end of lines.
	Type              reflect.Type  //
}

Input for NewIterator function.