jsonl

Documentation

Overview

Package jsonl provides a simple API for reading and writing to JSON Lines (aka jsonl). jsonl also supports iterators for efficiently reading through a stream. jsonl uses the github.com/spatialcurrent/go-simple-serializer/pkg/json for marshaling/unmarshaling JSON. See the examples below for usage.

• https://godoc.org/pkg/github.com/spatialcurrent/go-simple-serializer/pkg/json

Index

• Variables
• func Marshal(object interface{}, lineSeparator string, keySerializer stringify.Stringer, ...) ([]byte, error)
• func Read(input *ReadInput) (interface{}, error)
• func Write(input *WriteInput) error
• type Flusher
• type Iterator
  • func NewIterator(input *NewIteratorInput) *Iterator
  • func (it *Iterator) Next() (interface{}, error)
• type NewIteratorInput
• type ReadInput
• type WriteFlusher
• type WriteInput
• type Writer
  • func NewWriter(w io.Writer, separator string, keySerializer stringify.Stringer, pretty bool) *Writer
  • func (w *Writer) Close() error
  • func (w *Writer) Flush() error
  • func (w *Writer) WriteObject(obj interface{}) error
  • func (w *Writer) WriteObjects(objects interface{}) error

Examples

• Write (Map)
• Write (Map_upper)
• Write (Pretty)
• Write (Slice)

Variables

var (
	ErrMissingType          = errors.New("missing type")
	ErrMissingLineSeparator = errors.New("missing line separator")
)

Functions

func Marshal

func Marshal(object interface{}, lineSeparator string, keySerializer stringify.Stringer, pretty bool, limit int) ([]byte, error)

func Read

func Read(input *ReadInput) (interface{}, error)

Read reads the json lines from the input reader of the type given.

func Write

func Write(input *WriteInput) error

Write writes the given object(s) as JSON Lines (aka jsonl). If the type of the input object is of kind Array or Slice, then writes each object on its own line. Otherwise, the object is simply written as JSON.

Example (Map)

This example shows you can marshal a single map into a JSON object

obj := map[string]interface{}{
	"a": 1,
	"b": 2,
	"c": 3,
}
buf := new(bytes.Buffer)
err := Write(&WriteInput{
	Writer:        buf,
	LineSeparator: "\n",
	KeySerializer: stringify.NewStringer("", false, false, false),
	Object:        obj,
	Pretty:        false,
	Limit:         -1,
})
if err != nil {
	panic(err)
}
fmt.Println(buf.String())

Output:

{"a":1,"b":2,"c":3}

Example (Map_upper)

This example shows you can marshal a single map into a JSON object

obj := map[string]interface{}{
	"a": 1,
	"b": 2,
	"c": 3,
}
buf := new(bytes.Buffer)
err := Write(&WriteInput{
	Writer:        buf,
	LineSeparator: "\n",
	KeySerializer: stringify.NewStringer("", false, false, true),
	Object:        obj,
	Pretty:        false,
	Limit:         -1,
})
if err != nil {
	panic(err)
}
fmt.Println(buf.String())

Output:

{"A":1,"B":2,"C":3}

Example (Pretty)

This example shows you can marshal a slice maps into lines of JSON objects with pretty formatting.

obj := []interface{}{
	map[string]interface{}{
		"a": 1,
		"b": 2,
		"c": 3,
	},
	map[string]interface{}{
		"a": 4,
		"b": 5,
		"c": 6,
	},
}
buf := new(bytes.Buffer)
err := Write(&WriteInput{
	Writer:        buf,
	LineSeparator: "\n",
	KeySerializer: stringify.NewStringer("", false, false, false),
	Object:        obj,
	Pretty:        true,
	Limit:         -1,
})
if err != nil {
	panic(err)
}
fmt.Println(buf.String())

Output:

{
  "a": 1,
  "b": 2,
  "c": 3
}
{
  "a": 4,
  "b": 5,
  "c": 6
}

Example (Slice)

This examples shows that you can marshal a slice of maps into lines of JSON.

obj := []interface{}{
	map[string]interface{}{
		"a": 1,
		"b": 2,
		"c": 3,
	},
	map[string]interface{}{
		"a": 4,
		"b": 5,
		"c": 6,
	},
}
buf := new(bytes.Buffer)
err := Write(&WriteInput{
	Writer:        buf,
	LineSeparator: "\n",
	KeySerializer: stringify.NewStringer("", false, false, false),
	Object:        obj,
	Pretty:        false,
	Limit:         -1,
})
if err != nil {
	panic(err)
}
fmt.Println(buf.String())

Output:

{"a":1,"b":2,"c":3}
{"a":4,"b":5,"c":6}

Types

type Flusher

type Flusher interface {
	Flush() error
}

Flusher interfaces is a simple interface that wraps the Flush() function.

type Iterator

type Iterator struct {
	Type         reflect.Type    // the type to unmarshal for each line
	Scanner      scanner.Scanner // the scanner that splits the underlying stream of bytes
	Comment      []byte          // The comment line prefix.  Can be any string.
	Trim         bool            // Trim each input line before parsing into an object.
	SkipBlanks   bool            // Skip blank lines.  If false, Next() returns a blank line as (nil, nil).  If true, Next() simply skips forward until it finds a non-blank line.
	SkipComments bool            // Skip commented lines.  If false, Next() returns a commented line as (nil, nil).  If true, Next() simply skips forward until it finds a non-commented line.
	Limit        int             // Limit the number of objects to read and return from the underlying stream.
	Count        int             // The current count of the number of objects read.
}

Iterator iterates trough a stream of bytes returning a new object on each call of Next() until it reaches the end and returns io.EOF.

func NewIterator

func NewIterator(input *NewIteratorInput) *Iterator

NewIterator returns a new JSON Lines (aka jsonl) Iterator base on the given input.

func (*Iterator) Next

func (it *Iterator) Next() (interface{}, error)

Next reads from the underlying reader and returns the next object and error, if any. If a blank line is found and SkipBlanks is false, then returns (nil, nil). If a commented line is found and SkipComments is false, then returns (nil, nil). When the input stream is exhausted, returns (nil, io.EOF).

type NewIteratorInput

type NewIteratorInput struct {
	Reader            io.Reader
	Type              reflect.Type // the type to unmarshal for each line
	ScannerBufferSize int          // the initial buffer size for the scanner
	SkipLines         int          // Skip a given number of lines at the beginning of the stream.
	SkipBlanks        bool         // Skip blank lines.  If false, Next() returns a blank line as (nil, nil).  If true, Next() simply skips forward until it finds a non-blank line.
	SkipComments      bool         // Skip commented lines.  If false, Next() returns a commented line as (nil, nil).  If true, Next() simply skips forward until it finds a non-commented line.
	Comment           string       // The comment line prefix. Can be any string.
	Trim              bool         // Trim each input line before parsing into an object.
	Limit             int          // Limit the number of objects to read and return from the underlying stream.
	LineSeparator     byte         // The new line byte.
	DropCR            bool         // Drop carriage returns at the end of lines.
}

Input for NewIterator function.

type ReadInput

type ReadInput struct {
	Type              reflect.Type // the output type
	Reader            io.Reader    // the underlying reader
	ScannerBufferSize int          // the initial buffer size of the scanner
	SkipLines         int
	SkipBlanks        bool
	SkipComments      bool
	Comment           string // the comment prefix
	Trim              bool   // trim lines
	LineSeparator     byte   // the newline byte
	DropCR            bool   // drop carriage return
	Limit             int
}

ReadInput provides the input for the Read function.

type WriteFlusher

type WriteFlusher interface {
	io.Writer
	Flusher
}

WriterFlusher is a simple interface that wraps io.Writer and Flusher.

type WriteInput

type WriteInput struct {
	Writer        io.Writer // the underlying writer
	LineSeparator string    // the line separator
	KeySerializer stringify.Stringer
	Object        interface{} // the object to write
	Pretty        bool        // pretty output
	Limit         int
}

WriteInput provides the input for the Write function.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer formats and writes objects to the underlying writer as JSON Lines (aka jsonl).

func NewWriter

func NewWriter(w io.Writer, separator string, keySerializer stringify.Stringer, pretty bool) *Writer

NewWriter returns a writer for formating and writing objets to the underlying writer as JSON Lines (aka jsonl).

func (*Writer) Close

func (w *Writer) Close() error

Close closes the underlying writer, if it has a Close method.

func (*Writer) Flush

func (w *Writer) Flush() error

Flush flushes the underlying writer, if it has a Flush method. This writer itself does no buffering.

func (*Writer) WriteObject

func (w *Writer) WriteObject(obj interface{}) error

WriteObject formats and writes a single object to the underlying writer as JSON and appends the writer's line separator.

func (*Writer) WriteObjects

func (w *Writer) WriteObjects(objects interface{}) error

WriteObjects formats and writes the given objects to the underlying writer as JSON lines and separates the objects using the writer's line separator.