unsafejson

package module
v0.0.0-...-3b3ef25 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 30, 2016 License: BSD-2-Clause Imports: 16 Imported by: 0

README

UnsafeJSON

Use at your own risk.

A fork of Go's encoding/json package that doesn't escape the marshaled JSON. Go's stdlib JSON package does escaping for safety reasons and should be preferred over this package. Only use this if it is safe to not escape the JSON. Improper usage of this package may lead to security issues and other problems.

This package also contains some convenience funcs and a convenience struct, which are described below.

All code contained in this package, with the exception of the modifications necessary to enable the changes listed below, along with the updating of affected tests, were originally written by The Go Authors.

Usage: Same as the Go encode/json package. Use when you don't want the marshalled JSON to be HTMLEscaped. Please note that the JSON output from this package is not safe for use in browsers or use within the HTML <script> tags. For those scenarios, Go's enconding/json package should be used as it is designed for those use cases.

Changes:

  • Elided code responsible for the HTML escaping of <, >, & and associated tests.
  • Elided HTMLEscape() and related tests.
  • Added MarshalIndentToString() function, which wraps MarshalIndent and returns a string.*

Elided HTML Escape functionality

Main purpose, at time of creation, was to provide a marshaller that doesn't HTMLEscape the marshalled bytes. While this is useful for HTML and browser usage, this is neither helpful nor useful in some situations. One example is the creation of Packer templates, which may have inlined bash commands. HTMLEscaping the bash commands is not desirable, as all of the original encoding values must be preserved.

The strings which are no longer HTMLEscaped are <, >, &

Added MarshalIndentToString(f(v interface{}, prefix, indent string) string {}

MarshalIndentToString provides a convenient way of converting something to JSON and then returning a string version of it. Since MarshallIndentToString only has a single return value, which is a string, it is easy to use whenever you want to see a JSON version of a struct in a formatted string version. This is helpful in debugging because it can expose some information that can be elided when being printed out, e.g. nil being displayed as nulls, display of quotees, etc. Adding the Indent to the string output makes complex structures more parsable by humans.

fmt.Println(json.MarshalIndentToString(someInterface, "", "        "))

The above would print a JSON version of someInterface{} as a formatted string.

MarshalIndentToString wraps a call to MarshalIndent. If an error occurs, the error information is discarded an an empty string, "", is returned. If you need the error information, call MarshalIndent() instead. Otherwise, the []byte is converted to a string and returned.

Added MarshalToString(f(v interface{}) string {}

MarshallToString provides a convenient way of converting something to JSON and then returning a string version of it. Since MarshallToString only has a single return value, which is a string, it is easy to use whenever you want to see a JSON version of a struct. This is helpful in debugging because it can expose some information that can be elided when being printed out, e.g. nil being displayed as nulls, display of quotees, etc. This can be useful in testing.

fmt.Println(json.MarshalToString(someInterface))

The above would print a JSON version of someInterface{} as a string.

MarshalToString wraps a call to Marshal. If an error occurs, the error information is discarded an an empty string, "", is returned. If you need the error information, call MarshalIndent() instead. Otherwise, the []byte is converted to a string and returned.

Added StringMarshaller struct

The StringMarshaller struct provides an easy, compact way of using its customjson's MarshalToString and MarshalIndentToString functions. StringMarshaller offers two Get methods that wrap access to MarshalToString and MarshalIndentToString, making calls to these functions more compact.

To use, call customjson.NewStringMarshaller() and a StringMarshaller will be returned with its defaults set. To override the defaults, call StringMarshaller.Indent(string) and StringMarshaller.Prefix(string). Each method sets their respective unexported variable. Otherwise the default of "" and will be used.

Currently there are two methods on StringMarshaller:

Get(interface{}) string: Takes an interface and returns it as a JSON object converted to a string. GetIndented(interface{}) string: Takes an interface and returns it as a JSON object coverted to a formatted string, i.e. indented.

Example: package main

import (
	"fmt"

	json "github.com/mohae/customjson"
)

type team struct {
	City	string
	Name	string
	Mascot	string
}

func  main() {

	//Create a new StringMarshaller
	marshalString := json.NewStringMarshaller()

	bulls := &team{City: "Chicago", Name: "Bulls", Mascot: "Benny the Bull"}			
	fmt.Println(marshalString.Get(bulls))
	fmt.Println(marshalString.GetIndented(bulls))
}

Notes

I am reconsidering the elision of the HTMLEscape function as it doesn't affect the behavior that this fork of the json package was supposed to address. Since this is an exported function that others can use, when needed, it seems useful to leave it in.

Onl considering because I have higher priority code atm. It will be done at some point in the future. This will be signified by both a related commit and revision of this document.

Documentation

Overview

Package json implements encoding and decoding of JSON objects as defined in RFC 4627. The mapping between JSON objects and Go values is described in the documentation for the Marshal and Unmarshal functions.

See "JSON and Go" for an introduction to this package: http://golang.org/doc/articles/json_and_go.html

customjson customizes the Go JSON package. DO NOT USE if the output of the encoded json is meant for HTML.

Deltas:

  • elided the encoding of <, >, & to a html safe format, which is fine for HTML output, but not desirable for certain situations: e.g. outputting embedded shell commands within a JSON file.
  • Add MarshalToString functions and helper objects.

Copyright 2014, Joel Scoble (github.com/mohae): All rights reserved. Use of this source code is governed by a BSD-style license that can be found in the LICENSE file

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Compact 

func Compact(dst *bytes.Buffer, src []byte) error

Compact appends to dst the JSON-encoded src with insignificant space characters elided.

func HTMLEscape 

func HTMLEscape(dst *bytes.Buffer, src []byte)

HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029 characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029 so that the JSON will be safe to embed inside HTML <script> tags. For historical reasons, web browsers don't honor standard HTML escaping within <script> tags, so an alternative JSON encoding must be used.

func Indent 

func Indent(dst *bytes.Buffer, src []byte, prefix, indent string) error

Indent appends to dst an indented form of the JSON-encoded src. Each element in a JSON object or array begins on a new, indented line beginning with prefix followed by one or more copies of indent according to the indentation nesting. The data appended to dst does not begin with the prefix nor any indentation, and has no trailing newline, to make it easier to embed inside other formatted JSON data.

Example 
package main

import (
	"bytes"
	json "github.com/mohae/unsafejson"
	"log"
	"os"
)

func main() {
	type Road struct {
		Name   string
		Number int
	}
	roads := []Road{
		{"Diamond Fork", 29},
		{"Sheep Creek", 51},
	}

	b, err := json.Marshal(roads)
	if err != nil {
		log.Fatal(err)
	}

	var out bytes.Buffer
	json.Indent(&out, b, "=", "\t")
	out.WriteTo(os.Stdout)
}

Output:

[
=	{
=		"Name": "Diamond Fork",
=		"Number": 29
=	},
=	{
=		"Name": "Sheep Creek",
=		"Number": 51
=	}
=]

func Marshal 

func Marshal(v interface{}) ([]byte, error)

Marshal returns the JSON encoding of v.

Marshal traverses the value v recursively. If an encountered value implements the Marshaler interface and is not a nil pointer, Marshal calls its MarshalJSON method to produce JSON. The nil pointer exception is not strictly necessary but mimics a similar, necessary exception in the behavior of UnmarshalJSON.

Otherwise, Marshal uses the following type-dependent default encodings:

Boolean values encode as JSON booleans.

Floating point, integer, and Number values encode as JSON numbers.

String values encode as JSON strings. InvalidUTF8Error will be returned if an invalid UTF-8 sequence is encountered. The angle brackets "<" and ">" are escaped to "\u003c" and "\u003e" to keep some browsers from misinterpreting JSON output as HTML. Ampersand "&" is also escaped to "\u0026" for the same reason.

Array and slice values encode as JSON arrays, except that []byte encodes as a base64-encoded string, and a nil slice encodes as the null JSON object.

Struct values encode as JSON objects. Each exported struct field becomes a member of the object unless

  • the field's tag is "-", or
  • the field is empty and its tag specifies the "omitempty" option.

The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero. The object's default key string is the struct field name but can be specified in the struct field's tag value. The "json" key in the struct field's tag value is the key name, followed by an optional comma and options. Examples: 

// Field is ignored by this package.
Field int `json:"-"`

// Field appears in JSON as key "myName".
Field int `json:"myName"`

// Field appears in JSON as key "myName" and
// the field is omitted from the object if its value is empty,
// as defined above.
Field int `json:"myName,omitempty"`

// Field appears in JSON as key "Field" (the default), but
// the field is skipped if empty.
// Note the leading comma.
Field int `json:",omitempty"`

The "string" option signals that a field is stored as JSON inside a JSON-encoded string. It applies only to fields of string, floating point, or integer types. This extra level of encoding is sometimes used when communicating with JavaScript programs: 

Int64String int64 `json:",string"`

The key name will be used if it's a non-empty string consisting of only Unicode letters, digits, dollar signs, percent signs, hyphens, underscores and slashes.

Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph. An anonymous struct field with a name given in its JSON tag is treated as having that name, rather than being anonymous.

The Go visibility rules for struct fields are amended for JSON when deciding which field to marshal or unmarshal. If there are multiple fields at the same level, and that level is the least nested (and would therefore be the nesting level selected by the usual Go rules), the following extra rules apply:

1) Of those fields, if any are JSON-tagged, only tagged fields are considered, even if there are multiple untagged fields that would otherwise conflict. 2) If there is exactly one field (tagged or not according to the first rule), that is selected. 3) Otherwise there are multiple fields, and all are ignored; no error occurs.

Handling of anonymous struct fields is new in Go 1.1. Prior to Go 1.1, anonymous struct fields were ignored. To force ignoring of an anonymous struct field in both current and earlier versions, give the field a JSON tag of "-".

Map values encode as JSON objects. The map's key type must be string; the object keys are used directly as map keys.

Pointer values encode as the value pointed to. A nil pointer encodes as the null JSON object.

Interface values encode as the value contained in the interface. A nil interface value encodes as the null JSON object.

Channel, complex, and function values cannot be encoded in JSON. Attempting to encode such a value causes Marshal to return an UnsupportedTypeError.

JSON cannot represent cyclic data structures and Marshal does not handle them. Passing cyclic structures to Marshal will result in an infinite recursion.

Example 
package main

import (
	"fmt"
	json "github.com/mohae/unsafejson"
	"os"
)

func main() {
	type ColorGroup struct {
		ID     int
		Name   string
		Colors []string
	}
	group := ColorGroup{
		ID:     1,
		Name:   "Reds",
		Colors: []string{"Crimson", "Red", "Ruby", "Maroon"},
	}
	b, err := json.Marshal(group)
	if err != nil {
		fmt.Println("error:", err)
	}
	os.Stdout.Write(b)
}

Output:

{"ID":1,"Name":"Reds","Colors":["Crimson","Red","Ruby","Maroon"]}

func MarshalIndent 

func MarshalIndent(v interface{}, prefix, indent string) ([]byte, error)

MarshalIndent is like Marshal but applies Indent to format the output.

func MarshalIndentToString 

func MarshalIndentToString(v interface{}, prefix, indent string) string

MarshalIndentToString wraps MarshalIndent, converting the []byte to a string before returning the result, if it didn't error. Errors are thrown away and an empty string is returned.

Not ideal to ignore errors but since this function is designed to create a readable printout, i.e. MarshalIndent'd, version of an interface, in JSON. This makes it useful for debugging, logging, etc.

If error check is necessary, call MarshalIndent first.

func MarshalToString 

func MarshalToString(v interface{}) string

MarshalIndentToString wraps MarshalIndent, converting the []byte to a string before returning the result, if it didn't error. Errors are thrown away and an empty string is returned.

Not ideal to ignore errors but since this function is designed to create a readable printout, i.e. MarshalIndent'd, version of an interface, in JSON. This makes it useful for debugging, logging, etc.

If error check is necessary, call MarshalIndent first.

func Unmarshal 

func Unmarshal(data []byte, v interface{}) error

Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v.

Unmarshal uses the inverse of the encodings that Marshal uses, allocating maps, slices, and pointers as necessary, with the following additional rules:

To unmarshal JSON into a pointer, Unmarshal first handles the case of the JSON being the JSON literal null. In that case, Unmarshal sets the pointer to nil. Otherwise, Unmarshal unmarshals the JSON into the value pointed at by the pointer. If the pointer is nil, Unmarshal allocates a new value for it to point to.

To unmarshal JSON into a struct, Unmarshal matches incoming object keys to the keys used by Marshal (either the struct field name or its tag), preferring an exact match but also accepting a case-insensitive match.

To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value: 

bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null

If a JSON value is not appropriate for a given target type, or if a JSON number overflows the target type, Unmarshal skips that field and completes the unmarshalling as best it can. If no more serious errors are encountered, Unmarshal returns an UnmarshalTypeError describing the earliest such error.

The JSON null value unmarshals into an interface, map, pointer, or slice by setting that Go value to nil. Because null is often used in JSON to mean “not present,” unmarshaling a JSON null into any other Go type has no effect on the value and produces no error.

When unmarshaling quoted strings, invalid UTF-8 or invalid UTF-16 surrogate pairs are not treated as an error. Instead, they are replaced by the Unicode replacement character U+FFFD.

Example 
package main

import (
	"fmt"
	json "github.com/mohae/unsafejson"
)

func main() {
	var jsonBlob = []byte(`[
		{"Name": "Platypus", "Order": "Monotremata"},
		{"Name": "Quoll",    "Order": "Dasyuromorphia"}
	]`)
	type Animal struct {
		Name  string
		Order string
	}
	var animals []Animal
	err := json.Unmarshal(jsonBlob, &animals)
	if err != nil {
		fmt.Println("error:", err)
	}
	fmt.Printf("%+v", animals)
}

Output:

[{Name:Platypus Order:Monotremata} {Name:Quoll Order:Dasyuromorphia}]

Types

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

A Decoder reads and decodes JSON objects from an input stream.

Example

This example uses a Decoder to decode a stream of distinct JSON values. 

package main

import (
	"fmt"
	json "github.com/mohae/unsafejson"
	"io"
	"log"
	"strings"
)

func main() {
	const jsonStream = `
		{"Name": "Ed", "Text": "Knock knock."}
		{"Name": "Sam", "Text": "Who's there?"}
		{"Name": "Ed", "Text": "Go fmt."}
		{"Name": "Sam", "Text": "Go fmt who?"}
		{"Name": "Ed", "Text": "Go fmt yourself!"}
	`
	type Message struct {
		Name, Text string
	}
	dec := json.NewDecoder(strings.NewReader(jsonStream))
	for {
		var m Message
		if err := dec.Decode(&m); err == io.EOF {
			break
		} else if err != nil {
			log.Fatal(err)
		}
		fmt.Printf("%s: %s\n", m.Name, m.Text)
	}
}

Output:

Ed: Knock knock.
Sam: Who's there?
Ed: Go fmt.
Sam: Go fmt who?
Ed: Go fmt yourself!

func NewDecoder 

func NewDecoder(r io.Reader) *Decoder

NewDecoder returns a new decoder that reads from r.

The decoder introduces its own buffering and may read data from r beyond the JSON values requested.

func (*Decoder) Buffered 

func (dec *Decoder) Buffered() io.Reader

Buffered returns a reader of the data remaining in the Decoder's buffer. The reader is valid until the next call to Decode.

func (*Decoder) Decode 

func (dec *Decoder) Decode(v interface{}) error

Decode reads the next JSON-encoded value from its input and stores it in the value pointed to by v.

See the documentation for Unmarshal for details about the conversion of JSON into a Go value.

func (*Decoder) UseNumber 

func (dec *Decoder) UseNumber()

UseNumber causes the Decoder to unmarshal a number into an interface{} as a Number instead of as a float64.

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

An Encoder writes JSON objects to an output stream.

func NewEncoder 

func NewEncoder(w io.Writer) *Encoder

NewEncoder returns a new encoder that writes to w.

func (*Encoder) Encode 

func (enc *Encoder) Encode(v interface{}) error

Encode writes the JSON encoding of v to the stream, followed by a newline character.

See the documentation for Marshal for details about the conversion of Go values to JSON.

type InvalidUTF8Error 

type InvalidUTF8Error struct {
	S string // the whole string value that caused the error
}

Before Go 1.2, an InvalidUTF8Error was returned by Marshal when attempting to encode a string value with invalid UTF-8 sequences. As of Go 1.2, Marshal instead coerces the string to valid UTF-8 by replacing invalid bytes with the Unicode replacement rune U+FFFD. This error is no longer generated but is kept for backwards compatibility with programs that might mention it.

func (*InvalidUTF8Error) Error 

func (e *InvalidUTF8Error) Error() string

type InvalidUnmarshalError 

type InvalidUnmarshalError struct {
	Type reflect.Type
}

An InvalidUnmarshalError describes an invalid argument passed to Unmarshal. (The argument to Unmarshal must be a non-nil pointer.)

func (*InvalidUnmarshalError) Error 

func (e *InvalidUnmarshalError) Error() string

type MarshalString 

type MarshalString struct {
	// contains filtered or unexported fields
}

MarshalString is a struct to wrap the MarshalToString functions. This is mainly to simplify the use of MarshalIndentToString as all that needs to be passed is the interface to be marshalled to a string.

The defaults for MarshalToString are the most commonly used, imo: 

prefix: ""
indent: "        "

Each of these settings can be overridden individually by calling its respective public method.

func NewMarshalString 

func NewMarshalString() *MarshalString

func (*MarshalString) Get 

func (m *MarshalString) Get(v interface{}) string

func (*MarshalString) GetIndented 

func (m *MarshalString) GetIndented(v interface{}) string

func (*MarshalString) Indent 

func (m *MarshalString) Indent(s string)

func (*MarshalString) Prefix 

func (m *MarshalString) Prefix(s string)

type Marshaler 

type Marshaler interface {
	MarshalJSON() ([]byte, error)
}

Marshaler is the interface implemented by objects that can marshal themselves into valid JSON.

type MarshalerError 

type MarshalerError struct {
	Type reflect.Type
	Err  error
}

func (*MarshalerError) Error 

func (e *MarshalerError) Error() string

type Number 

type Number string

A Number represents a JSON number literal.

func (Number) Float64 

func (n Number) Float64() (float64, error)

Float64 returns the number as a float64.

func (Number) Int64 

func (n Number) Int64() (int64, error)

Int64 returns the number as an int64.

func (Number) String 

func (n Number) String() string

String returns the literal text of the number.

type RawMessage 

type RawMessage []byte

RawMessage is a raw encoded JSON object. It implements Marshaler and Unmarshaler and can be used to delay JSON decoding or precompute a JSON encoding.

Example

This example uses RawMessage to delay parsing part of a JSON message. 

package main

import (
	"fmt"
	json "github.com/mohae/unsafejson"
	"log"
)

func main() {
	type Color struct {
		Space string
		Point json.RawMessage // delay parsing until we know the color space
	}
	type RGB struct {
		R uint8
		G uint8
		B uint8
	}
	type YCbCr struct {
		Y  uint8
		Cb int8
		Cr int8
	}

	var j = []byte(`[
		{"Space": "YCbCr", "Point": {"Y": 255, "Cb": 0, "Cr": -10}},
		{"Space": "RGB",   "Point": {"R": 98, "G": 218, "B": 255}}
	]`)
	var colors []Color
	err := json.Unmarshal(j, &colors)
	if err != nil {
		log.Fatalln("error:", err)
	}

	for _, c := range colors {
		var dst interface{}
		switch c.Space {
		case "RGB":
			dst = new(RGB)
		case "YCbCr":
			dst = new(YCbCr)
		}
		err := json.Unmarshal(c.Point, dst)
		if err != nil {
			log.Fatalln("error:", err)
		}
		fmt.Println(c.Space, dst)
	}
}

Output:

YCbCr &{255 0 -10}
RGB &{98 218 255}

func (*RawMessage) MarshalJSON 

func (m *RawMessage) MarshalJSON() ([]byte, error)

MarshalJSON returns *m as the JSON encoding of m.

func (*RawMessage) UnmarshalJSON 

func (m *RawMessage) UnmarshalJSON(data []byte) error

UnmarshalJSON sets *m to a copy of data.

type SyntaxError 

type SyntaxError struct {
	Offset int64 // error occurred after reading Offset bytes
	// contains filtered or unexported fields
}

A SyntaxError is a description of a JSON syntax error.

func (*SyntaxError) Error 

func (e *SyntaxError) Error() string

type UnmarshalFieldError 

type UnmarshalFieldError struct {
	Key   string
	Type  reflect.Type
	Field reflect.StructField
}

An UnmarshalFieldError describes a JSON object key that led to an unexported (and therefore unwritable) struct field. (No longer used; kept for compatibility.)

func (*UnmarshalFieldError) Error 

func (e *UnmarshalFieldError) Error() string

type UnmarshalTypeError 

type UnmarshalTypeError struct {
	Value string       // description of JSON value - "bool", "array", "number -5"
	Type  reflect.Type // type of Go value it could not be assigned to
}

An UnmarshalTypeError describes a JSON value that was not appropriate for a value of a specific Go type.

func (*UnmarshalTypeError) Error 

func (e *UnmarshalTypeError) Error() string

type Unmarshaler 

type Unmarshaler interface {
	UnmarshalJSON([]byte) error
}

Unmarshaler is the interface implemented by objects that can unmarshal a JSON description of themselves. The input can be assumed to be a valid encoding of a JSON value. UnmarshalJSON must copy the JSON data if it wishes to retain the data after returning.

type UnsupportedTypeError 

type UnsupportedTypeError struct {
	Type reflect.Type
}

An UnsupportedTypeError is returned by Marshal when attempting to encode an unsupported value type.

func (*UnsupportedTypeError) Error 

func (e *UnsupportedTypeError) Error() string

type UnsupportedValueError 

type UnsupportedValueError struct {
	Value reflect.Value
	Str   string
}

func (*UnsupportedValueError) Error 

func (e *UnsupportedValueError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.