dataflatten

Documentation

Overview

Package dataflatten contains tools to flatten complex data structures.

On macOS, many plists use an array of maps, these can be tricky to filter. This package knows how to flatten that structure, as well as rewriting it as a nested array, or filtering it. It is akin to xpath, though simpler.

This tool works primarily through string interfaces, so type information may be lost.

Query Syntax

The query syntax handles both filtering and basic rewriting. It is not perfect. The idea behind it, is that we descend through an data structure, specifying what matches at each level.

Each level of query can do:

• specify a filter, this is a simple string match with wildcard support. (prefix and/or postfix, but not infix)
• If the data is an array, specify an index
• For array-of-maps, specify a key to rewrite as a nested map

Each query term has 3 parts: [#]string[=>kvmatch]

1. An optional `#` This denotes a key to rewrite an array-of-maps with

2. A search term. If this is an integer, it is interpreted as an array index.

3. a key/value match string. For a map, this is to match the value of a key.

   Some examples: * data/users Return everything under { data: { users: { ... } } } * data/users/0 Return the first item in the users array * data/users/name=>A* Return users whose name starts with "A" * data/users/#id Return the users, and rewrite the users array to be a map with the id as the key

See the test suite for extensive examples.

Index

• type FlattenOpts
  • func IncludeNulls() FlattenOpts
  • func WithLogger(logger log.Logger) FlattenOpts
  • func WithQuery(q []string) FlattenOpts
• type Flattener
• type Row
  • func Flatten(data interface{}, opts ...FlattenOpts) ([]Row, error)
  • func Json(rawdata []byte, opts ...FlattenOpts) ([]Row, error)
  • func JsonFile(file string, opts ...FlattenOpts) ([]Row, error)
  • func Plist(rawdata []byte, opts ...FlattenOpts) ([]Row, error)
  • func PlistFile(file string, opts ...FlattenOpts) ([]Row, error)
  • func (r Row) ParentKey(sep string) (string, string)
  • func (r Row) StringPath(sep string) string

Types

type FlattenOpts

type FlattenOpts func(*Flattener)

func IncludeNulls

func IncludeNulls() FlattenOpts

IncludeNulls indicates that Flatten should return null values, instead of skipping over them.

func WithLogger

func WithLogger(logger log.Logger) FlattenOpts

WithLogger sets the logger to use

func WithQuery

func WithQuery(q []string) FlattenOpts

WithQuery Specifies a query to flatten with. This is used both for re-writing arrays into maps, and for filtering. See "Query Specification" for docs.

type Flattener

type Flattener struct {
	// contains filtered or unexported fields
}

Flattener is an interface to flatten complex, nested, data structures. It recurses through them, and returns a simplified form. At the simplest level, this rewrites:

{ foo: { bar: { baz: 1 } } }

To:

[ { path: foo/bar/baz, value: 1 } ]

It can optionally filtering and rewriting.

type Row

type Row struct {
	Path  []string
	Value string
}

Row is the record type we return.

func Flatten

func Flatten(data interface{}, opts ...FlattenOpts) ([]Row, error)

Flatten is the entry point to the Flattener functionality.

func Json

func Json(rawdata []byte, opts ...FlattenOpts) ([]Row, error)

func JsonFile

func JsonFile(file string, opts ...FlattenOpts) ([]Row, error)

func Plist

func Plist(rawdata []byte, opts ...FlattenOpts) ([]Row, error)

func PlistFile

func PlistFile(file string, opts ...FlattenOpts) ([]Row, error)

func (Row) ParentKey

func (r Row) ParentKey(sep string) (string, string)

func (Row) StringPath

func (r Row) StringPath(sep string) string