Documentation
¶
Overview ¶
Package json is an faster alternative to stdlib
It allows you to parse bytes buffer or io.Reader stream with no allocs and without knowing document structure and field names. You also can do the same in opposite direction: building json in a streaming fashion.
It's could be used as a replacement to "encoding/json". Or you can build complex analyzer on top of it.
It's an one time reader, so read data are dropped if buffer gets full.
It's doesn't do things you don't need. It won't parse or read anything until you ask. It doesn't decode escape sequences at strings you don't need. It copy readed string only if it contain escape sequences or if it doesn't fit info the buffer. And it's copied to reusable buffer. There are two versions on skipString functions: fast (default), that looks only for closing '"' byte, and strict, that check utf8 encoding also (you can choose between them by strict build tag). Reader.CompareKey doesn't check utf8 encoding and doesn't decode escape sequences (did you see object keys like this "\t\nkey \n"? if so, use (*Reader).NextString() instead).
You should be very attentive since some checks were sacrificed in favor of speed. So you will not be notified if for example you forgot to read some value in the object and requested the next key.
See examples for more ideas of usage
Index ¶
- Variables
- func Cause(err error) error
- func Copy(w *Writer, r *Reader)
- func UnsafeStringToBytes(s string) []byte
- type Base64Reader
- type Base64Writer
- type Error
- type Num
- type Reader
- func (r *Reader) Base64Reader(enc *base64.Encoding) Base64Reader
- func (r *Reader) Bool() (bool, error)
- func (r *Reader) CheckString() (string, error)
- func (r *Reader) CompareKey(k []byte) bool
- func (r *Reader) Err() error
- func (r *Reader) ErrorHere(err error) error
- func (r *Reader) Float64() (float64, error)
- func (r *Reader) GoOut(d int)
- func (r *Reader) HasNext() (res bool)
- func (r *Reader) Inspect(cb func(r *Reader) bool, ks ...interface{}) bool
- func (r *Reader) Int() (int, error)
- func (r *Reader) Int64() (int64, error)
- func (r *Reader) IsNull() bool
- func (r *Reader) Lock()
- func (r *Reader) MustCheckString() string
- func (r *Reader) MustInt() int
- func (r *Reader) MustInt64() int64
- func (r *Reader) MustUint() uint
- func (r *Reader) MustUint64() uint64
- func (r *Reader) NextAsBytes() []byte
- func (r *Reader) NextNumber() []byte
- func (r *Reader) NextString() (res []byte)
- func (r *Reader) Reset(b []byte) *Reader
- func (r *Reader) ResetErr()
- func (r *Reader) ResetReader(rd io.Reader) *Reader
- func (r *Reader) ResetString(s string) *Reader
- func (r *Reader) Return()
- func (r *Reader) Search(keys ...interface{}) *Reader
- func (r *Reader) Skip()
- func (r *Reader) StringOrEmpty() string
- func (r *Reader) StringReader() StringReader
- func (r *Reader) Type() Type
- func (r *Reader) Uint() (uint, error)
- func (r *Reader) Uint64() (uint64, error)
- func (r *Reader) Unlock()
- type StringReader
- type StringWriter
- type Type
- type Writer
- func (w *Writer) ArrayEnd()
- func (w *Writer) ArrayStart()
- func (w *Writer) Base64Writer(enc *base64.Encoding) Base64Writer
- func (w *Writer) Bool(f bool)
- func (w *Writer) Bytes() []byte
- func (w *Writer) Err() error
- func (w *Writer) Flush() error
- func (w *Writer) NewLine()
- func (w *Writer) Null()
- func (w *Writer) Number(v []byte)
- func (w *Writer) ObjEnd()
- func (w *Writer) ObjKey(k []byte)
- func (w *Writer) ObjKeyString(k string)
- func (w *Writer) ObjStart()
- func (w *Writer) RawBytes(v []byte)
- func (w *Writer) RawString(v []byte)
- func (w *Writer) RawStringString(v string)
- func (w *Writer) SetIndent(pref, ind []byte)
- func (w *Writer) String(v []byte)
- func (w *Writer) StringString(v string)
- func (w *Writer) StringWriter() StringWriter
- func (w *Writer) Write(p []byte) (int, error)
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrIncompatibleTypes = errors.New("incompatible types") ErrEncoding = errors.New("string encoding") )
Functions ¶
func Cause ¶ added in v0.2.0
Cause returns cause of underlying error if any
copy of github.com/pkg/errors.Cause
func UnsafeStringToBytes ¶ added in v0.2.0
Types ¶
type Base64Reader ¶ added in v0.2.0
type Base64Reader struct {
StringReader
// contains filtered or unexported fields
}
Base64Reader allows to read raw bytes stream encoded in base64 encoding
type Base64Writer ¶ added in v0.2.0
type Base64Writer struct {
StringWriter
// contains filtered or unexported fields
}
Base64Writer allows to write raw bytes stream as an base64 encoded string
func (Base64Writer) Close ¶ added in v0.2.0
func (w Base64Writer) Close() error
Close must be called to finish the string
type Error ¶
type Error struct {
// contains filtered or unexported fields
}
Error keeps position and context of an error and can pretty print it It supports 3 forms of formatting:
%v - produces output:
parse error at pos 15: expected object key
%#v
parse error at pos 15: expected object key `{"some":"text",123}`
%+v
parse error at pos 15: expected object key
{"some":"text",123}
_______________^___
It also supports width setting which changes max size of context shown in extended forms
type Reader ¶ added in v0.2.0
type Reader struct {
// contains filtered or unexported fields
}
Reader parses byte stream and allows you to manipulate structured data
Example ¶
This example shows how you can collect some info out of json without fully parsing it
data := `[{"user":"Kyle","visits":134},{"user":"Rose","visits":45,"online":true},{"user":"Adam","visits":37}]`
r := WrapString(data)
var visits int
for r.HasNext() {
r.Search("visits")
visits += r.MustInt()
r.GoOut(1) // number of keys passed to Search()
}
fmt.Printf("Total visits: %d\n", visits)
Output: Total visits: 216
Example (Messenger) ¶
data := `
{"status":"online","term":3,"messages":[
{"text":"Tomorrow at 6pm Remember!!","sender":"Violet","term":3}]}
{"status":"online","term":3,"messages":[
{"text":"Tomorrow at 6pm Remember!!","sender":"Violet","term":3}]}
{"status":"online","term":4,"messages":[
{"text":"Tomorrow at 6pm Remember!!","sender":"Violet","term":3},
{"text":"Hi, Nik! You won't believe who have I met just now!","sender":"Dan","term":4},
{"text":"When did you see Dan last time?","sender":"Jane","term":4},
]}
`
// Let's imagine that `stream` is a socket and we are taking server updates once a 5 seconds
stream := strings.NewReader(data)
type Message struct {
Text string
Sender string
Term int
}
r := NewReader(stream)
messagesTotal := 0
lastTerm := 0
for r.Type() != None {
ok := r.Inspect(func(r *Reader) bool {
term, err := r.Int()
if err != nil {
return false
}
res := term != lastTerm
lastTerm = term
return res
}, "term")
if !ok {
continue
}
r.Inspect(func(r *Reader) bool {
for r.HasNext() {
var m Message
/*
err := r.Unmarshal(&m)
if err != nil {
return false
}
*/
for r.HasNext() {
k := string(r.NextString())
switch k {
case "text":
m.Text = string(r.NextString())
case "sender":
m.Sender = string(r.NextString())
case "term":
v, err := json.Number(r.NextNumber()).Int64()
if err != nil {
return false
}
m.Term = int(v)
default:
r.Skip()
}
}
if m.Term != lastTerm {
continue
}
messagesTotal++
fmt.Printf("Message from %v: \t%v\n", m.Sender, m.Text)
}
return false
}, "messages")
}
if err := r.Err(); err != io.EOF && err != nil {
// process eny errors at one place
}
fmt.Printf("Connection closed. Total messages got: %d", messagesTotal)
Output: Message from Violet: Tomorrow at 6pm Remember!! Message from Dan: Hi, Nik! You won't believe who have I met just now! Message from Jane: When did you see Dan last time? Connection closed. Total messages got: 3
func NewReaderBufferSize ¶ added in v0.2.0
NewReaderBufferSize creates reader from stream with specified buffer size
func (*Reader) Base64Reader ¶ added in v0.2.0
func (r *Reader) Base64Reader(enc *base64.Encoding) Base64Reader
Base64Reader allows to read raw bytes stream encoded in base64 encoding
Example ¶
var buf bytes.Buffer
token := make([]byte, 10)
_, _ = rand.New(rand.NewSource(1)).Read(token)
// Writer
fmt.Printf("Writer sends message with a raw bytes token: % 2x\n", token)
// create json writer on top of io.Writer
w := NewStreamWriter(&buf)
// generate object
w.ObjStart()
w.ObjKey([]byte("user"))
w.String([]byte("nikandfor"))
w.ObjKey([]byte("token"))
// this way we can handle json strings of any size with constant memory usage
sw := w.Base64Writer(base64.RawStdEncoding)
_, _ = sw.Write(token) // we don't care errors here, we'll care them later at once
_ = sw.Close()
w.ObjEnd()
// check all errors until now
err := w.Flush()
if err != nil {
// process
}
// Reader
// read io.Reader stream on the fly with constant memory usage
r := NewReader(&buf)
var user string
var tokenDec []byte = make([]byte, len(token))
// check if it's really object (but we can skip this)
if tp := r.Type(); tp != Object {
fmt.Printf("Expected object, got %v\n", tp)
}
// read all key-value pairs
for r.HasNext() {
// key
k := r.NextString()
// value
switch {
case bytes.Equal(k, []byte("user")):
user = string(r.NextString())
case bytes.Equal(k, []byte("token")):
sr := r.Base64Reader(base64.RawStdEncoding)
_, err := io.ReadFull(sr, tokenDec)
if err != nil {
// will be io.EOF if there is less that len(tokenDec) bytes at string
}
sr.Close()
default:
r.Skip()
}
}
// check error just before application logic
err = r.Err()
if err != nil {
// process
}
fmt.Printf("Reader got message from user: %v, token % 2x\n", user, tokenDec)
Output: Writer sends message with a raw bytes token: 52 fd fc 07 21 82 65 4f 16 3f Reader got message from user: nikandfor, token 52 fd fc 07 21 82 65 4f 16 3f
func (*Reader) CheckString ¶ added in v0.2.0
func (*Reader) CompareKey ¶ added in v0.2.0
CompareKey compares given key weth the next string value. It doesn't decodes escape sequences and doesn't check for utf8 correctness
func (*Reader) Err ¶ added in v0.2.0
Err returns first happened error. It returns Error type that could nicely format error message for you if you will see it at console or with monospace font
func (*Reader) GoOut ¶ added in v0.2.0
GoOut goes out of object or array (reads until end). d means number of objects you want to out of. It's the ideal pair for (*Reader).Search method (See (*Reader).Inspect source code as an example)
func (*Reader) HasNext ¶ added in v0.2.0
HasNext checks if you can read next element at the current object (or array). It also reads opening and closing brakets internally
It moves position inside object (or array), so you'll get different result if you call it twice in a row. For example you have the following json
{"a": [{"b": "c"}, {"d": "e"}]}
^ - cursor is here (you've just read "a" and want to iterate over it's value)
// first time you call HasNext it returns true and moves cursor
{"a": [{"b": "c"}, {"d": "e"}]}
^ - here
// second time you call HasNext it returns true and moves cursor
{"a": [{"b": "c"}, {"d": "e"}]}
^ - here (inside b->c object)
// all subsequent calls will not move cursor anywhere until you read "b" key
See example how to use it properly
Example ¶
data := `{"a": [{"b": "c"}, {"d": "e"}],"f": true}`
r := WrapString(data)
r.Search("a")
for r.HasNext() { // over array
for r.HasNext() { // over array elements key-value pairs
// we must use for loop here even if we know it's the only pair inside
// because we have to read closing braket
key := r.NextString()
val := r.NextString()
fmt.Printf("pair: %q -> %q\n", key, val)
}
}
fKey := r.NextString()
switch r.Type() {
case Bool:
fVal, _ := r.Bool() // error could be here if there is truu instead of true for example
fmt.Printf("%q -> %v\n", fKey, fVal)
}
r.GoOut(1) // to get out of the most outer object
// it's the pair call to the first Search and we always have to call it if we want to read following data correctly
fmt.Printf("end of buffer, next value type: %v\n", r.Type())
Output: pair: "b" -> "c" pair: "d" -> "e" "f" -> true end of buffer, next value type: None
func (*Reader) Inspect ¶ added in v0.2.0
Inspect inspects requested value and returns back or skips object according to callback returned value: true - return back, false - skip the object. Whole data from the beginning of the object until inspected value (including) is stored in memory and buffer doesn't shrunk back
func (*Reader) Lock ¶ added in v0.2.0
func (r *Reader) Lock()
Lock locks all data since the current position and prevents them from freeing. So if buffer has ended it would be grown to accommodate new data You should use it for inspecting some small part of data and decide will you parse them from the beginning or just unlock reader and will go further.
See (*Reader).Inspect() source code for better understanding.
func (*Reader) MustCheckString ¶ added in v0.2.0
func (*Reader) MustUint64 ¶ added in v0.2.0
func (*Reader) NextAsBytes ¶ added in v0.2.0
NextAsBytes reads next object of any type and returns it as a raw byte slice without decoding (including string quotes)
func (*Reader) NextNumber ¶ added in v0.2.0
NextNumber reads next number (possible incorrect) of any size and returns it as bytes It doesn't check if nubmer is correct. It could have two points or '+' and '-' signs in the middle of string
func (*Reader) NextString ¶ added in v0.2.0
NextString read next object key or string checking utf8 encoding correctness and decoding '\t', '\n', '\r' escape sequences
func (*Reader) ResetReader ¶ added in v0.2.0
ResetReader resets the reader it reuses buffer (if you created reader by Wrap for example) if you used Lock and read a big object into the memory it doesn't shrink buffer back
func (*Reader) ResetString ¶ added in v0.2.0
ResetString resets the reader
func (*Reader) Return ¶ added in v0.2.0
func (r *Reader) Return()
Return returns the reading position to the previously locked by (*Reader).Lock
func (*Reader) Search ¶ added in v0.2.0
Search searches for value at the specified path. It supports keys of type int for arrays and string or []byte for objects
Example ¶
data := `
{"day": "Mon", "stats": {"views": {"by_partofday": [1, 2, 3]}}}
{"day": "Tue", "stats": {"views": {"by_partofday": [3, 2, 3]}}}
{"day": "Wed", "stats": {"views": {"by_partofday": [4, 1, 5]}}}
{"day": "Thu", "stats": {"views": {"by_partofday": [1, 2, 2]}}}
`
s := strings.NewReader(data)
r := NewReader(s)
sum := 0
days := 0
for r.Type() != None {
days++
r.Search("stats", "views", "by_partofday") // goes inside to requested value
for r.HasNext() {
sum += r.MustInt()
}
r.GoOut(3) // goes back up at 3 levels (to the end of the current day object)
}
fmt.Printf("total views for %v days: %v\n", days, sum)
Output: total views for 4 days: 29
func (*Reader) Skip ¶ added in v0.2.0
func (r *Reader) Skip()
Skip reads and skips one next value. It doesn't metter if you call it before ':' or just at start of value (it actually doesn't matter if ':' or ',' were or not at all)
func (*Reader) StringOrEmpty ¶ added in v0.2.0
func (*Reader) StringReader ¶ added in v0.2.0
func (r *Reader) StringReader() StringReader
StringReader allows to read string value as an io.ReadCloser interface
func (*Reader) Type ¶ added in v0.2.0
Type returns type of the next value. It skips commas ',' and colons ':' silently It doesn't skip or read over other tokens even if called multiple times
Example ¶
data := `{"first": "string", "second": 123, "third": [1.1, 3.3, 7.7], "fourth": {"again": "string", "and": {"object": "here"}}}`
var parse func(*Reader, int)
parse = func(r *Reader, d int) {
switch tp := r.Type(); tp {
case String, Number, Bool, Null:
// read by one of accordingly
// r.CheckString()
// r.Int() or r.Float64()
// r.Bool()
// r.Skip() // you can skip any value like this
val := r.NextAsBytes() // reads any value including object and array as raw bytes
fmt.Printf("%*s is %v\n", d*4, val, tp)
case Array:
for r.HasNext() {
parse(r, d+1)
}
case Object:
for r.HasNext() {
key := r.NextString()
fmt.Printf("%*s ->\n", d*4, key)
parse(r, d+1)
}
default:
err := r.Err()
fmt.Printf("%*s got type %v err %#v\n", d*4, "", tp, err)
}
}
r := WrapString(data)
parse(r, 1)
if err := r.Err(); err != nil {
fmt.Printf("reader: %+40v", err)
}
Output: first -> "string" is String second -> 123 is Number third -> 1.1 is Number 3.3 is Number 7.7 is Number fourth -> again -> "string" is String and -> object -> "here" is String
type StringReader ¶ added in v0.2.0
type StringReader struct {
*Reader
}
StringReader allows to read string value as an io.ReadCloser interface
func (StringReader) Close ¶ added in v0.2.0
func (r StringReader) Close() error
Close must be called to finish reading current string value It will skip unread part of string if any
type StringWriter ¶ added in v0.2.0
type StringWriter struct {
*Writer
}
StringWriter allows to write string value as an io.WriteCloser interface
func (StringWriter) Close ¶ added in v0.2.0
func (w StringWriter) Close() error
Close must be called to finish the string
type Writer ¶ added in v0.2.0
type Writer struct {
// contains filtered or unexported fields
}
func NewIndentWriter ¶ added in v0.2.0
func NewStreamWriter ¶ added in v0.2.0
func NewStreamWriterBuffer ¶ added in v0.2.0
func (*Writer) ArrayStart ¶ added in v0.2.0
func (w *Writer) ArrayStart()
func (*Writer) Base64Writer ¶ added in v0.2.0
func (w *Writer) Base64Writer(enc *base64.Encoding) Base64Writer
Base64Writer allows to write raw bytes stream as an base64 encoded string
func (*Writer) ObjKeyString ¶ added in v0.2.0
func (*Writer) RawStringString ¶ added in v0.2.0
func (*Writer) StringString ¶ added in v0.2.0
func (*Writer) StringWriter ¶ added in v0.2.0
func (w *Writer) StringWriter() StringWriter
StringWriter allows to write string value as an io.WriteCloser interface