flatr

README

The Flat Rabbit, Barour Oskarsson

FLATR

Golang util to easily flat a nested map into a 1-level deep one.

Features

• Prefix selection
• Separator selection
• Transformers to preprocess each node for advanced use cases

Get it

go get github.com/pricelessrabbit/flatr

Use it

Basic usage

toFlat := map[string]any{
		"foo": "bar",
		"nest": map[string]any{
			"bar": "baz",
		},
		"list": []any{
			map[string]any{
				"id":   1,
				"data": "data1",
			},
			map[string]any{
				"id":   2,
				"data": "data2",
			},
		},
	}
f := flatr.New()
flatted, err := f.Flat(toFlat)


/* result
map[
  foo: bar 
  list.0.data: data1 
  list.0.id: 1 
  list.1.data: data2 
  list.1.id: 2 
  nest.bar: baz
]
*/

Options

Add options to the constructor to use them: flatr.New(Option1(),Option2()...)

Prefix option

Adds a prefix to all the flatted keys

f := New(flatr.WithPrefix("namespace"))
flatted, _ := f.Flat(toFlat)


/* result
map[
  namespace.foo: bar 
  namespace.list.0.data: data1 
  namespace.list.0.id: 1 
  namespace.list.1.data: data2 
  namespace.list.1.id: 2 
  namespace.nest.bar: baz
]
*/

Separator option

Choose path separator (default .)

f := New(flatr.WithSeparator("_"))
flatted, _ := f.Flat(toFlat)

/* result
map[
  foo: bar
  list_0_data: data1
  list_0_id: 1
  list_1_data: data2
  list_1_id: 2
  nest_bar: baz
]
*/

Transformers

Transformers:

• Have access to the entry related to node being flatted
• Can read and update node key, value, and set a stop flag to block children processing

Default transformers

flatr.Transformers.MaxDeep

define a max deep for the flattening process. after reaching the max deep, the children of the current will be left untouched.

 toFlat := map[string]any{
    "foo": []any{
        map[string]any{"id": "a", "data": 10},
        map[string]any{"id": "b", "data": 20},
    },
    "bar": []any{
        map[string]any{"id": 1, "data": 30},
        map[string]any{"id": 2, "data": 40},
    },
}
f := New(
    AddTransformer(MaxDeep(2)),
)
flatted, _ := f.Flat(toFlat)
/*
map[
	foo.0: {"id": "a", "data": 10}
	foo.1: {"id": 1, "data": 30}
	bar.0: {"id": 1, "data": 30}
    bar.1: {"id": 2, "data": 40}
]
*/
	

flatr.Transformers.UseFieldAsIndex

if the current node is a slice, use the value of the given field (eg id) as index instead of the array index.

toFlat := map[string]any{
    "foo": []any{
    map[string]any{"id": "a", "data": 10},
    map[string]any{"id": "b", "data": 20},
    },
    "bar": []any{
    map[string]any{"id": 1, "data": 30},
    map[string]any{"id": 2, "data": 40},
    },
}

f := New(
AddScopedTransformer("foo", UseFieldAsIndex("id")),
AddScopedTransformer("bar", UseFieldAsIndex("id")),
)

flatted, _ := f.Flat(toFlat)

/*
map[
foo.a.id: a
foo.a.data: 10
foo.b.id: b 
foo.b.data: 20

bar.1.id: 1
bar.1: data: 30
bar.2.id: 2
bar.2.data: 40
]
*/

Custom transfomers

It is possible to implement custom transformers. A trasformer is valid if:

• takes an Entry
• returns the transformed Entry and an optional error

The entry field that can be transformed are:

• K the key of the node
• V the value related to the key
• H deep of the node
• Stop stops processing of the children of the current node

Example

the transformer in the example adds the suffix "_transformed" to all the string values that are found in the structure. other value types are left untouched.

customTransformer := func (e Entry) (Entry, error) {
    //check if the current entry value is a string
    // and in that case add the suffix "transformed"
    if s, ok := e.V.(string); ok {
    e.V = s + "_transformed"
    }
    return e, nil
}

f := New(AddTransformer(customTransformer))
flatted, _ := f.Flat(toFlat)

/*
map[
	foo: bar_transformed
	list_0_data: data1_transformed
	list_0_id: 1
	list_1_data: data2_transformed
	list_1_id: 2
	nest_bar: baz_transformed
]
*/

Documentation

Index

• type Entry
• type Flattener
  • func New(options ...Option) *Flattener
  • func (f *Flattener) Flat(toFlat any) (map[string]any, error)
• type Option
  • func AddScopedTransformer(key string, tr Transformer) Option
  • func AddTransformer(tr Transformer) Option
  • func WithPrefix(prefix string) Option
  • func WithSeparator(separator string) Option
• type Transformer
  • func MaxDeep(h int) Transformer
  • func UseFieldAsIndex(idKey string) Transformer

Types

type Entry

type Entry struct {
	K    string
	V    any
	H    int
	Stop bool
}

Entry keep intermediate structure when flatten is ongoing and is passed to transformers tracked data include key and value of the node, height in the tree and if process have to stop without processing children nodes

type Flattener

type Flattener struct {
	// contains filtered or unexported fields
}

Flattener keep memory of options and transformers to flatten maps

func New

func New(options ...Option) *Flattener

New instantiate new flatter with provided options WithPrefix option add a prefix to keys WithSeparator option change default separator AddTransformer and AddScopedTransformer add custom transfomers that apply a custom function to each entry before processing it

func (*Flattener) Flat

func (f *Flattener) Flat(toFlat any) (map[string]any, error)

Flat flatten a nested data structure (scalar, maps and slices) to a flat map. returns a map of 1 level when key is the full path of the field divided by a separator (default to .)

type Option

type Option func(f *Flattener)

Option change default Flattener behaviour

func AddScopedTransformer

func AddScopedTransformer(key string, tr Transformer) Option

AddScopedTransformer add a transfomer called only in nodes that match the specific key

func AddTransformer

func AddTransformer(tr Transformer) Option

AddTransformer add a transformer called to all entries in the flattening process

func WithPrefix

func WithPrefix(prefix string) Option

WithPrefix add a prefix to flatted keys

func WithSeparator

func WithSeparator(separator string) Option

WithSeparator change default separator of flatted keys

type Transformer

type Transformer func(Entry) (Entry, error)

Transformer custom function called by Flattener to preprocess nodes

func MaxDeep

func MaxDeep(h int) Transformer

MaxDeep add a limit to the maximum tree-height of an entry to be processed if the height exceeds the limit the flattening stops and node is saved unflatted

func UseFieldAsIndex

func UseFieldAsIndex(idKey string) Transformer

UseFieldAsIndex applied to a list of maps, uses list element field as the list index. the list is transformed in a map with the choosen field as key and the element as value