Documentation ¶
Overview ¶
Package jsonpack is a fast and space efficiency JSON serialization golang library. It is a schema oriented design which leverages schema definition to encode JSON document into compact binary encoded format, and decodes back into JSON document.
Introduction ¶
When we want to exchange data between services or over network, the JSON is most popular format to do it. In golang world, the most convenient way is using official `encoding/json` package to marshal and unmarshal JSON document from/to struct or map. For usual RESTful web service scenario, JSON format is quite convenience and representative, but for real-time message exchanging or other scenarios that has small footprint data and low-letency requirement, JSON is a bit too heavyweight, not only data footprint is not space saving, but also has heavy loading in encode/decode procedure.
So if we want to a compact, small footprint data for exchanging over network, and also leverages the convenience of JSON, we need an encoding format that removes "property name" and other notations likes ':', '[', '{'...etc from original JSON document, and leaves "value" only.
To achieve this goal, we need a schematic to define our JSON document and provide enough information for serialization engine to know sequence and data type of every properties in document. it's the reason why jsonpack is a schema oriented design library.
Key Features ¶
* Similar Marshal / Unmarshal API to standard `encoding/json` package.
* Space saving encoded format, the size of encoded data is similar to Protocol Buffers, can be 30-80% of original JSON document, depends on data.
* Blazing fast, provides about 3.x decoding speed compared to `protobuf` and many times than other JSON packages.
* Memory saving design, avoids any un-necessary memory allocations, suitable for embedded environment.
* Has production ready javascript implementation https://github.com/arloliu/buffer-plus, can be used in node.js and Web browser environment.
* No need to write schema definition by hand, `jsonpack` will generate schema definition from golang struct automatically.
Index ¶
- type ByteOrder
- type CompileError
- type DecodeError
- type EncodeError
- type JSONPack
- func (p *JSONPack) AddSchema(schemaName string, v ...interface{}) (*Schema, error)
- func (p *JSONPack) Decode(schemaName string, data []byte, v interface{}) error
- func (p *JSONPack) Encode(schemaName string, v interface{}) ([]byte, error)
- func (p *JSONPack) EncodeTo(schemaName string, v interface{}, dataPtr *[]byte) error
- func (p *JSONPack) GetAllSchemaDefTexts() map[string][]byte
- func (p *JSONPack) GetAllSchemaDefs() map[string]*SchemaDef
- func (p *JSONPack) GetAllSchemas() map[string]*Schema
- func (p *JSONPack) GetSchema(schemaName string) *Schema
- func (p *JSONPack) GetSchemaDef(schemaName string) (*SchemaDef, error)
- func (p *JSONPack) GetSchemaDefText(schemaName string) ([]byte, error)
- func (p *JSONPack) Marshal(schemaName string, v interface{}) ([]byte, error)
- func (p *JSONPack) RemoveSchema(schemaName string) error
- func (p *JSONPack) Reset()
- func (p *JSONPack) Unmarshal(schemaName string, data []byte, v interface{}) error
- type NotImplementedError
- type Schema
- func (s *Schema) Decode(data []byte, v interface{}) (err error)
- func (s *Schema) Encode(d interface{}) ([]byte, error)
- func (s *Schema) EncodeTo(d interface{}, dataPtr *[]byte) (err error)
- func (s *Schema) GetSchemaDef() (*SchemaDef, error)
- func (s *Schema) GetSchemaDefText() []byte
- func (s *Schema) Marshal(d interface{}) ([]byte, error)
- func (s *Schema) SetEncodeBufSize(size int64)
- func (s *Schema) Unmarshal(data []byte, v interface{}) (err error)
- type SchemaDef
- type SchemaNonExistError
- type StructFieldNonExistError
- type TypeAssertionError
- type UnknownTypeError
- type WrongTypeError
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ByteOrder ¶
type ByteOrder int
ByteOrder represents the byte order of numeric type that will be encoded to and decoded from. Possible values are LittleEndian and BigEndian
type CompileError ¶
CompileError represents an error from calling AddSchema method, it indicates there has an error occurs in compiling procedure of schema definition.
func (*CompileError) Error ¶
func (e *CompileError) Error() string
func (*CompileError) Unwrap ¶
func (e *CompileError) Unwrap() error
Unwrap returns the underlying error.
type DecodeError ¶
DecodeError represents an error from calling Decode or Unmarshal methods.
func (*DecodeError) Error ¶
func (e *DecodeError) Error() string
func (*DecodeError) Unwrap ¶
func (e *DecodeError) Unwrap() error
Unwrap returns the underlying error.
type EncodeError ¶
EncodeError represents an error from calling Encode or Marshal methods.
func (*EncodeError) Error ¶
func (e *EncodeError) Error() string
func (*EncodeError) Unwrap ¶
func (e *EncodeError) Unwrap() error
Unwrap returns the underlying error.
type JSONPack ¶
type JSONPack struct {
// contains filtered or unexported fields
}
JSONPack provides top-level operations for schema.
func (*JSONPack) AddSchema ¶
AddSchema compiles schema definition and stores compiled result in internal schema manager.
It's a variadic function which accepts two input argument forms in the following.
AddSchema(schemaName string, v interface{})
The v is schema definition which want to compile. The value of v can be a JSON document with []byte/string type, a map represents JSON format of schema definition, or a SchemaDef struct which represents a schema definition.
Example of add new schema from JSON document:
schDef := ` { "type": "object", "properties": { "name": {"type": "string"}, "area": {"type": "uint32le"} }, "order": ["name", "area"] } ` jsonPack := jsonpack.NewJSONPack() sch, err := jsonPack.AddSchema("info", schDef)
Example of adding new schema from a map of schema definition:
schDef := map[string]interface{}{ "type": "object", "properties": map[string]interface{}{ "name": map[string]interface{}{"type": "string"}, "area": map[string]interface{}{"type": "uint32le"}, }, "order": []string{"name", "area"}, } jsonPack := jsonpack.NewJSONPack() sch, err := jsonPack.AddSchema("info", schDef)
Example of adding new schema from SchemaDef struct:
schDef := jsonpack.SchemaDef{ Type: "object", Properties: map[string]*jsonpack.SchemaDef{ "name": {Type: "string"}, "area": {Type: "uint32le"}, }, Order: []string{"name", "area"}, } jsonPack := jsonpack.NewJSONPack() sch, err := jsonPack.AddSchema("info", schDef)
AddSchema(schemaName string, v interface{}, byteOrder jsonpack.ByteOrder)
For fast prototyping, AddSchema method supports generate schema definition from existing struct without writing schema definition by hand.
In this scenario, the value of v is the target struct which to be generated, and byteOrder parameter indicates the byte order, can be either jsonpack.LittleEndian or jsonpack.BigEndian, it defaults to little-endian if not specified.
This method supports struct tag, use the same format as standard encoding/json excepts "omitempty" option, the "omitempty" option will be ignored.
Example of adding new schema and build schema definition from struct:
type Info struct { Name string `json:"name"` // "omitempty" option will be ignore, so this field will be not be omitted Area uint32 `json:"area,omitempty"` ExcludeField string `-` // this field is ignored } jsonPack := jsonpack.NewJSONPack() sch, err := jsonPack.AddSchema("Info", Info{}, jsonpack.BigEndian)
func (*JSONPack) Decode ¶
Decode is a wrapper of Schema.Decode, it returns *SchemaNonExistError error if schema not found.
func (*JSONPack) Encode ¶
Encode is a wrapper of Schema.Encode, it returns *SchemaNonExistError error if schema not found.
func (*JSONPack) EncodeTo ¶
EncodeTo is a wrapper of Schema.EncodeTo, it returns *SchemaNonExistError error if schema not found.
func (*JSONPack) GetAllSchemaDefTexts ¶
GetAllSchemaDefTexts returns a map which contains all existed schema text definitions, key of map it schema name, and value of map is text format of schema definition which presented as []byte.
func (*JSONPack) GetAllSchemaDefs ¶
GetAllSchemaDefs returns a map which contains all existed schema definitions, key of map it schema name, and value of map is *SchemaDef.
func (*JSONPack) GetAllSchemas ¶
GetAllSchemas returns a map which contains all existed schema instances, the key of map it schema name, and value of map is *Schema.
func (*JSONPack) GetSchema ¶
GetSchema returns a schema instance by schemaName, returns nil if schema not found.
func (*JSONPack) GetSchemaDef ¶
GetSchemaDef is a wrapper of Schema.GetSchemaDef, it gets a Schema instance by schemaName, it returns *SchemaNonExistError error if schema not found.
func (*JSONPack) GetSchemaDefText ¶
GetSchemaDefText is a wrapper of Schema.GetSchemaDefText, it returns *SchemaNonExistError error if schema not found.
func (*JSONPack) Marshal ¶
Marshal is an alias to Encode function, provides familiar interface of standard json package.
func (*JSONPack) RemoveSchema ¶
RemoveSchema removes schema by schemaName, it returns *SchemaNonExistError error if schema not found.
type NotImplementedError ¶
type NotImplementedError struct {
Name string // operation name that not implemented by handler
}
NotImplementedError is returned when the operation in handler doesn't implemented.
func (*NotImplementedError) Error ¶
func (e *NotImplementedError) Error() string
type Schema ¶
type Schema struct { // schema name Name string // contains filtered or unexported fields }
Schema represents a compiled jsonpack schema instance which created by jsonpack.AddSchema function
func (*Schema) Decode ¶
Decode reads encoded data with compiled schema definition and stores the result in the value pointed to v.
If type of v is not a pointer type that pointed to a map or struct, Decode function will return DecodeError.
The valid type of v is either a *map[string]interface{} or a pointer to the struct which added by AddSchema function.
Example of decoding data into map with "Info" schema
decodeInfoMap = make(map[string]interface{}) err := jsonPack.Decode("Info", encodedData, &decodeInfoMap)
Example of decoding data into Info struct instance with "Info" schema
decodeInfoStruct = Info{} err := jsonPack.Decode("Info", encodedData, &decodeInfoStruct)
func (*Schema) Encode ¶
Encode returns packed binary data of v with compiled schema definition. The compiled schema definition is specified by schemaName and return data will be nil if error occurs.
The type of v can be a map[string]interface{} which represents valid JSON data, or a struct instance which added by AddSchema function.
The return data contains encoded binary data that can then be decoded by Decode function.
Before encode data into jsonpack encoding format, we need to create a jsonpack instance and create a schema.
// the Info struct we want to decode type Info struct { Name string `json:"name"` Area uint32 `json:"area"` // omit this field ExcludeField string `-` } // create a new jsonpack instance jsonPack := jsonpack.NewJSONPack() // create schema with Info struct sch, err := jsonPack.AddSchema("Info", Info{}, jsonpack.LittleEndian)
If we want to encode a map with Info schema.
data := map[string]interface{} { "name": "example name", "area": uint32(888), } // encodes data into encodedResult encodedResult, err := sch.Encode(data)
Or if we want to encode a Info struct instance with Info schema.
data := &Info{ Name: "example name", Area: 888, } // encodes data into encodedResult encodedResult, err := sch.Encode(infoStruct)
func (*Schema) EncodeTo ¶
EncodeTo is similar to Encode function, but passing a pointer to []byte to store encoded data instead of returning new allocated []byte encoded data.
This method is useful with buffer pool for saving memory allocation usage and improving performance.
Caution: the encoder might re-allocate and grow the slice if necessary, the length and capacity of slice might be changed.
func (*Schema) GetSchemaDef ¶
GetSchemaDef returns a schema definition instance, returns nil and error if error occurs.
func (*Schema) GetSchemaDefText ¶
GetSchemaDefText returns JSON document of schema definition that represented as []byte type.
func (*Schema) Marshal ¶
Marshal is an alias to Encode function, provides familiar interface of json package
func (*Schema) SetEncodeBufSize ¶
SetEncodeBufSize sets default allocation byte size of encode buffer.
The encode buffer will be allocated with this value when calling Encode/Marshall method, and will be re-allocate and growed automatically if necessary. The encoder will also adjust this value by latest encoded result.
Sets a proper default allocation size might help to reduce re-allocation frequency and saves memory usage.
type SchemaDef ¶
type SchemaDef struct { Type string `json:"type"` Properties map[string]*SchemaDef `json:"properties,omitempty"` Items *SchemaDef `json:"items,omitempty"` Order []string `json:"order,omitempty"` }
SchemaDef represents a schema definition that defines the structure of JSON document.
Example:
schDef := SchemaDef{ Type: "object", Properties: map[string]*jsonpack.SchemaDef{ "name": {Type: "string"}, "area": {Type: "uint32le"}, }, Order: []string{"name", "area"}, }
type SchemaNonExistError ¶
type SchemaNonExistError struct {
Name string // schema name
}
SchemaNonExistError indicates an error that occurs when pre-compiled schema definition does not exist.
func (*SchemaNonExistError) Error ¶
func (e *SchemaNonExistError) Error() string
type StructFieldNonExistError ¶
type StructFieldNonExistError struct { Name string // name of structure Field string // field of structure that expects to be existed }
StructFieldNonExistError indicates an error that occurs when structure doesn't have required field.
func (*StructFieldNonExistError) Error ¶
func (e *StructFieldNonExistError) Error() string
type TypeAssertionError ¶
type TypeAssertionError struct { Data interface{} // the data that failed on type assertion ExpectedType string // expected data type }
TypeAssertionError indicates an error that occurs when doing data type asserrion.
func (*TypeAssertionError) Error ¶
func (e *TypeAssertionError) Error() string
type UnknownTypeError ¶
type UnknownTypeError struct {
DataType string // data type that un-supported by schema definition
}
UnknownTypeError is returned when un-supported data type found, it happens in AddSchema method.
func (*UnknownTypeError) Error ¶
func (e *UnknownTypeError) Error() string
type WrongTypeError ¶
type WrongTypeError struct {
DataType string // wrong type of data
}
WrongTypeError is returned when wrong data type found in data, it happens in Encode/Decode methods.
func (*WrongTypeError) Error ¶
func (e *WrongTypeError) Error() string
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package buffer provides a easy way to manipulate byte slices.
|
Package buffer provides a easy way to manipulate byte slices. |
cmd
|
|
jsonpack-parser
jsonpack-parser is a tool to generate jsonpack schema definition from specified package and struct name.
|
jsonpack-parser is a tool to generate jsonpack schema definition from specified package and struct name. |
internal
|
|