jsonpath

package module
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 12, 2023 License: MIT Imports: 10 Imported by: 7

README

AsaiYusuke/JSONPath

Test Go Report Card Coverage Status Go Reference Awesome Go

AsaiYusuke/JSONPath

This Go library is for retrieving a part of JSON according to the JSONPath query syntax.

The core JSONPath syntax on which this library based:

Note:

For syntax compatibility among other libraries, please check 📝 my comparison results.

Table of Contents

Getting started

Install:
go get github.com/AsaiYusuke/jsonpath
Simple example:
package main

import (
  "encoding/json"
  "fmt"

  "github.com/AsaiYusuke/jsonpath"
)

func main() {
  jsonPath, srcJSON := `$.key`, `{"key":"value"}`
  var src interface{}
  json.Unmarshal([]byte(srcJSON), &src)
  output, _ := jsonpath.Retrieve(jsonPath, src)
  outputJSON, _ := json.Marshal(output)
  fmt.Println(string(outputJSON))
  // Output:
  // ["value"]
}

Basic design

Streamlined Development
  • The JSONPath syntax analysis functionality has been separated using PEG, resulting in a more simplified source code.
  • Robust unit testing has been implemented to prevent bugs and ensure consistent outcomes.
User-Friendly Interface
  • The library is equipped with a comprehensive error specification, allowing users to effectively handle any errors that may arise.
Unwavering Compatibility

How to use

* Retrieve one-time or repeated

The Retrieve function returns retrieved result using JSONPath and JSON object:

output, err := jsonpath.Retrieve(jsonPath, src)

📝 Example

The Parse function returns a parser-function that completed to check JSONPath syntax. By using parser-function, it can repeat to retrieve with the same JSONPath :

jsonPath, err := jsonpath.Parse(jsonPath)
output1, err1 := jsonPath(src1)
output2, err2 := jsonPath(src2)
:

📝 Example

* Error handling

If there is a problem with the execution of APIs, an error type returned. These error types define the corresponding symptom, as listed below:

Syntax check errors from Retrieve, Parse
Error type Message format Symptom Ex
ErrorInvalidSyntax invalid syntax (position=%d, reason=%s, near=%s) The invalid syntax found in the JSONPath.
The reason including in this message will tell you more about it.		 📝
ErrorInvalidArgument invalid argument (argument=%s, error=%s) The argument specified in the JSONPath treated as the invalid error in Go syntax. 📝
ErrorFunctionNotFound function not found (function=%s) The function specified in the JSONPath is not found. 📝
ErrorNotSupported not supported (feature=%s, path=%s) The unsupported syntaxes specified in the JSONPath. 📝
Runtime errors from Retrieve, parser-functions
Error type Message format Symptom Ex
ErrorMemberNotExist member did not exist (path=%s) The object/array member specified in the JSONPath did not exist in the JSON object. 📝
ErrorTypeUnmatched type unmatched (expected=%s, found=%s, path=%s) The node type specified in the JSONPath did not exist in the JSON object. 📝
ErrorFunctionFailed function failed (function=%s, error=%s) The function specified in the JSONPath failed. 📝

The type checking is convenient to recognize which error happened.

  :
  _,err := jsonpath.Retrieve(jsonPath, srcJSON)
  if err != nil {
    switch err.(type) {
    case jsonpath.ErrorMemberNotExist:
      fmt.printf(`retry with other srcJSON: %v`, err)
      continue
    case jsonpath.ErrorInvalidArgumentFormat:
      return nil, fmt.errorf(`specified invalid argument: %v`, err)
    }
    :
  }
* Function syntax

Function enables to format results by using user defined functions. The function syntax comes after the JSONPath.

There are two ways to use function:

Filter function

The filter function applies a user function to each values in the result to get converted.

📝 Example

Aggregate function

The aggregate function converts all values in the result into a single value.

📝 Example

* Accessing JSON

You can get the accessors ( Getters / Setters ) of the input JSON instead of the retrieved values. These accessors can use to update for the input JSON.

This feature can get enabled by giving Config.SetAccessorMode().

📝 Example

Note:

It is not possible to use Setter for some results, such as for JSONPath including function syntax.

Also, operations using accessors follow the map/slice manner of Go language. If you use accessors after changing the structure of JSON, you need to pay attention to the behavior. If you don't want to worry about it, get the accessor again every time you change the structure.

Differences

Some behaviors that differ from the consensus exists in this library. For the entire comparisons, please check 📝 this result.

These behaviors will change in the future if appropriate ones found.

Character types

The following character types can be available for identifiers in dot-child notation.

Character type Available Escape required
* Numbers and alphabets (0-9 A-Z a-z)
* Hyphen and underscore (- _)
* Non-ASCII Unicode characters (0x80 - 0x10FFFF)		 Yes No
* Other printable symbols (Space ! " # $ % & ' ( ) * + , . / : ; < = > ? @ [ \ ] ^ ` { | } ~) Yes Yes
* Control code characters (0x00 - 0x1F, 0x7F) No -

The printable symbols except hyphen and underscore can use by escaping them.

JSONPath : $.abc\.def
srcJSON  : {"abc.def":1}
Output   : [1]
Wildcard in qualifier

The wildcards in qualifier can specify as a union of subscripts.

JSONPath : $[0,1:3,*]
srcJSON  : [0,1,2,3,4,5]
Output   : [0,1,2,0,1,2,3,4,5]
Regular expression

The regular expression syntax works as a regular expression in Go lang. In particular, you can use "(?i)" to specify the regular expression as the ignore case option.

JSONPath : $[?(@=~/(?i)CASE/)]
srcJSON  : ["Case","Hello"]
Output   : ["Case"]
JSONPaths in the filter-qualifier

JSONPaths that return a value group cannot use with comparator or regular expression. However, existence check can use these syntaxes.

JSONPaths that return a value group example
Recursive descent @..a
Multiple identifier @['a','b']
Wildcard identifier @.*
Slice qualifier @[0:1]
Wildcard qualifier @[*]
Union in the qualifier @[0,1]
Filter qualifier @.a[?(@.b)]
  • comparator example (error)
JSONPath : $[?(@..x == "hello world")]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Error    : ErrorInvalidSyntax
  • regular expression example (error)
JSONPath : $[?(@..x=~/hello/)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Error    : ErrorInvalidSyntax
  • existence check example
JSONPath : $[?(@..x)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Output   : [{"b":{"x":"hello world"}}]

JSONPath filter that begins with Root is a whole-match operation when any one is detected.

JSONPath : $[?($..x)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Output   : [{"a":1},{"b":{"x":"hello world"}}]

Benchmarks

The benchmarks for various JSONPath libraries in Go language can be compared in the following repository.

Project progress

  • Syntax
    • Identifier
      • identifier in dot notations
      • identifier in bracket notations
      • wildcard
      • multiple-identifier in bracket
      • recursive retrieve
    • Qualifier
      • index
      • slice
      • wildcard
      • Filter
        • logical operation
        • comparator
        • JSONPath retrieve in filter
      • script
    • Function
      • filter
      • aggregate
    • Refer to the consensus behaviors
  • Architecture
    • PEG syntax analyzing
    • Error handling
    • Function
    • Accessing JSON
  • Go language manner
    • retrieve with the object in interface unmarshal
    • retrieve with the json.Number type
  • Source code
    • Release version
    • Unit tests
      • syntax tests
      • benchmark
      • coverage >80%
    • Examples
    • CI automation
    • Documentation
      • README
      • API doc
    • comparison result (local)
  • Development status
    • determine requirements / functional design
    • design-based coding
    • testing
    • documentation
  • Future ToDo
    • Refer to the something standard
    • Go language affinity
      • retrieve with the object in struct unmarshal
      • retrieve with the struct tags
      • retrieve with the user defined objects

Documentation

Overview

Package jsonpath is for retrieving a part of JSON according to the JSONPath query syntax.

Example 
package main

import (
	"encoding/json"
	"fmt"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$.key`, `{"key":"value"}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, _ := jsonpath.Retrieve(jsonPath, src)
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

["value"]

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Parse 

func Parse(jsonPath string, config ...Config) (f func(src interface{}) ([]interface{}, error), err error)

Parse returns the parser function using the given JSONPath.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath := `$.key`
	srcJSON1 := `{"key":"value1"}`
	srcJSON2 := `{"key":"value2"}`
	jsonPathParser, err := jsonpath.Parse(jsonPath)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	var src1, src2 interface{}
	json.Unmarshal([]byte(srcJSON1), &src1)
	json.Unmarshal([]byte(srcJSON2), &src2)
	output1, err := jsonPathParser(src1)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	output2, err := jsonPathParser(src2)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON1, _ := json.Marshal(output1)
	outputJSON2, _ := json.Marshal(output2)
	fmt.Println(string(outputJSON1))
	fmt.Println(string(outputJSON2))
}

Output:

["value1"]
["value2"]

func Pretty added in v1.2.0 

func Pretty(pretty bool) func(*pegJSONPathParser) error

func Retrieve 

func Retrieve(jsonPath string, src interface{}, config ...Config) ([]interface{}, error)

Retrieve returns the retrieved JSON using the given JSONPath.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$.key`, `{"key":"value"}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

["value"]

func Size added in v1.2.0 

func Size(size int) func(*pegJSONPathParser) error

Types

type Accessor added in v1.2.0 

type Accessor struct {
	Get func() interface{}
	Set func(interface{})
}

Accessor represents the accessor to the result nodes of JSONPath.

type Config added in v1.1.0 

type Config struct {
	// contains filtered or unexported fields
}

Config represents the configuration parameters.

func (*Config) SetAccessorMode added in v1.2.0 

func (c *Config) SetAccessorMode()

SetAccessorMode sets a collection of accessors to the result.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	config := jsonpath.Config{}
	config.SetAccessorMode()
	jsonPath, srcJSON := `$.a`, `{"a":1,"b":0}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src, config)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	accessor := output[0].(jsonpath.Accessor)
	srcMap := src.(map[string]interface{})

	fmt.Printf("Get : %v\n", accessor.Get())

	accessor.Set(2)
	fmt.Printf("Set -> Src : %v\n", srcMap[`a`])

	accessor.Set(3)
	fmt.Printf("Set -> Get : %v\n", accessor.Get())

	srcMap[`a`] = 4
	fmt.Printf("Src -> Get : %v\n", accessor.Get())

}

Output:

Get : 1
Set -> Src : 2
Set -> Get : 3
Src -> Get : 4

func (*Config) SetAggregateFunction added in v1.1.0 

func (c *Config) SetAggregateFunction(id string, function func([]interface{}) (interface{}, error))

SetAggregateFunction sets the custom function.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	config := jsonpath.Config{}
	config.SetAggregateFunction(`max`, func(params []interface{}) (interface{}, error) {
		var result float64
		for _, param := range params {
			if floatParam, ok := param.(float64); ok {
				if result < floatParam {
					result = floatParam
				}
				continue
			}
			return nil, fmt.Errorf(`type error`)
		}
		return result, nil
	})
	jsonPath, srcJSON := `$[*].max()`, `[1,3]`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src, config)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

[3]

func (*Config) SetFilterFunction added in v1.1.0 

func (c *Config) SetFilterFunction(id string, function func(interface{}) (interface{}, error))

SetFilterFunction sets the custom function.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	config := jsonpath.Config{}
	config.SetFilterFunction(`twice`, func(param interface{}) (interface{}, error) {
		if floatParam, ok := param.(float64); ok {
			return floatParam * 2, nil
		}
		return nil, fmt.Errorf(`type error`)
	})
	jsonPath, srcJSON := `$[*].twice()`, `[1,3]`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src, config)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

[2,6]

type ErrorFunctionFailed added in v1.1.0 

type ErrorFunctionFailed struct {
	// contains filtered or unexported fields
}

ErrorFunctionFailed represents the error that the function specified in the JSONPath failed.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	config := jsonpath.Config{}
	config.SetFilterFunction(`invalid`, func(param interface{}) (interface{}, error) {
		return nil, fmt.Errorf(`invalid function executed`)
	})
	jsonPath, srcJSON := `$.invalid()`, `{}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src, config)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

jsonpath.ErrorFunctionFailed, function failed (function=.invalid(), error=invalid function executed)

func (ErrorFunctionFailed) Error added in v1.1.0 

func (e ErrorFunctionFailed) Error() string

type ErrorFunctionNotFound added in v1.1.0 

type ErrorFunctionNotFound struct {
	// contains filtered or unexported fields
}

ErrorFunctionNotFound represents the error that the function specified in the JSONPath is not found.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$.unknown()`, `{}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

jsonpath.ErrorFunctionNotFound, function not found (function=.unknown())

func (ErrorFunctionNotFound) Error added in v1.1.0 

func (e ErrorFunctionNotFound) Error() string

type ErrorInvalidArgument 

type ErrorInvalidArgument struct {
	// contains filtered or unexported fields
}

ErrorInvalidArgument represents the error that argument specified in the JSONPath is treated as the invalid error in Go syntax.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$[?(1.0.0>0)]`, `{}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

jsonpath.ErrorInvalidArgument, invalid argument (argument=1.0.0, error=strconv.ParseFloat: parsing "1.0.0": invalid syntax)

func (ErrorInvalidArgument) Error 

func (e ErrorInvalidArgument) Error() string

type ErrorInvalidSyntax 

type ErrorInvalidSyntax struct {
	// contains filtered or unexported fields
}

ErrorInvalidSyntax represents the error that have syntax error in the JSONPath.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$.`, `{}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

jsonpath.ErrorInvalidSyntax, invalid syntax (position=1, reason=unrecognized input, near=.)

func (ErrorInvalidSyntax) Error 

func (e ErrorInvalidSyntax) Error() string

type ErrorMemberNotExist 

type ErrorMemberNotExist struct {
	// contains filtered or unexported fields
}

ErrorMemberNotExist represents the error that the member specified in the JSONPath did not exist in the JSON object.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$.none`, `{}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

jsonpath.ErrorMemberNotExist, member did not exist (path=.none)

func (ErrorMemberNotExist) Error 

func (e ErrorMemberNotExist) Error() string

type ErrorNotSupported 

type ErrorNotSupported struct {
	// contains filtered or unexported fields
}

ErrorNotSupported represents the error that the unsupported syntaxes specified in the JSONPath.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$[(command)]`, `{}`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

jsonpath.ErrorNotSupported, not supported (feature=script, path=[(command)])

func (ErrorNotSupported) Error 

func (e ErrorNotSupported) Error() string

type ErrorTypeUnmatched 

type ErrorTypeUnmatched struct {
	// contains filtered or unexported fields
}

ErrorTypeUnmatched represents the error that the node type specified in the JSONPath did not exist in the JSON object.

Example 
package main

import (
	"encoding/json"
	"fmt"
	"reflect"

	"github.com/AsaiYusuke/jsonpath"
)

func main() {
	jsonPath, srcJSON := `$.a`, `[]`
	var src interface{}
	json.Unmarshal([]byte(srcJSON), &src)
	output, err := jsonpath.Retrieve(jsonPath, src)
	if err != nil {
		fmt.Printf(`%v, %v`, reflect.TypeOf(err), err)
		return
	}
	outputJSON, _ := json.Marshal(output)
	fmt.Println(string(outputJSON))
}

Output:

jsonpath.ErrorTypeUnmatched, type unmatched (expected=object, found=[]interface {}, path=.a)

func (ErrorTypeUnmatched) Error 

func (e ErrorTypeUnmatched) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.