flatten

package
v0.0.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 14, 2026 License: Apache-2.0 Imports: 5 Imported by: 0

README

flatten

English | 中文

flatten is a Go package for working with hierarchical key-value data structures, primarily for handling nested data in formats like JSON, YAML, or TOML.

Features

1. Data Flattening
  • Converts nested maps, slices, and arrays into a flat map[string]string.
  • Uses dot notation for maps (e.g., db.hosts).
  • Uses index notation for arrays/slices (e.g., hosts[0]).
  • Example: {"db": {"hosts": ["a", "b"]}} becomes {"db.hosts[0]": "a", "db.hosts[1]": "b"}.
2. Path Handling
  • Defines a Path abstraction, representing hierarchical keys as a sequence of typed segments (map keys or array indices).
  • Supports parsing string paths (e.g., "foo.bar[0]") into Path objects.
  • Supports converting Path objects back into string paths.
3. Storage Management
  • The Storage type manages a collection of flattened key-value pairs.
  • Internally builds and maintains a hierarchical tree structure to prevent key conflicts.
  • Associates values with their source files, supporting multi-file merging and source tracking.
4. Querying
  • Provides helper methods for retrieving values.
  • Checks for the existence of keys.
  • Enumerates subkeys.
  • Iterates in a deterministic order.

Typical Use Cases

  1. Standardizing configuration files from multiple sources into a flat key-value map for comparison, merging, or diffing.
  2. Querying nested data with simple string paths, avoiding direct reflection or manual traversal of nested maps.
  3. Building tools that unify structured data from multiple files while preserving source information and preventing conflicts.

Example

package main

import (
	"fmt"
	"github.com/go-spring/stdlib/flatten"
)

func main() {
	// Create a nested data structure
	data := map[string]interface{}{
		"database": map[string]interface{}{
			"host": "localhost",
			"port": 5432,
			"credentials": map[string]interface{}{
				"username": "admin",
				"password": "secret",
			},
		},
		"features": []interface{}{
			"feature1",
			"feature2",
			map[string]interface{}{
				"name":    "feature3",
				"enabled": true,
			},
		},
	}

	// Flatten the data
	flat := flatten.Flatten(data)

	// Print flattened results
	for key, value := range flat {
		fmt.Printf("%s: %s\n", key, value)
	}

	// Use Storage to manage data
	storage := flatten.NewStorage()
	fileID := storage.AddFile("config.yaml")

	// Set values
	storage.Set("database.host", "localhost", fileID)
	storage.Set("database.port", "5432", fileID)

	// Retrieve values
	host := storage.Get("database.host")
	fmt.Printf("Database host: %s\n", host)

	// Check if a key exists
	if storage.Has("database.host") {
		fmt.Println("Database host exists")
	}
}

License

Apache License 2.0

Documentation

Index

Constants

View Source 
const (
	// StorageCommandLine represents configuration provided via command line.
	// This usually has the highest priority.
	StorageCommandLine = iota

	// StorageEnvironment represents configuration from environment variables.
	StorageEnvironment

	// StorageProfileFile represents configuration loaded from profile-specific files.
	// Example: application-dev.properties.
	StorageProfileFile

	// StorageAppFile represents configuration from the main application file.
	// Example: application.properties or application.yml.
	StorageAppFile

	// StorageDefault represents built-in default configuration values.
	// This layer typically has the lowest priority.
	StorageDefault

	// StorageMax is the number of supported layers.
	StorageMax
)

Variables

This section is empty.

Functions

func Flatten 

func Flatten(m map[string]any) map[string]string

Flatten flattens a nested map[string]any into a map[string]string.

This function is intended for data produced by encoding/json.Unmarshal, where values are limited to the following kinds:

  • map[string]any
  • []any
  • primitive JSON types (bool, number, string, nil)

Structs, custom types, and non-string map keys are explicitly out of scope.

Flattening rules:

  • Nested maps are expanded using dot notation: {"a": {"b": 1}} -> "a.b" = "1"

  • Slices (and arrays, although arrays do not originate from json.Unmarshal) are expanded using index notation: {"a": [1, 2]} -> "a[0]" = "1", "a[1]" = "2"

  • Nil values (both untyped nil and typed nil) are represented as "<nil>".

  • Empty (zero-length but non-nil) maps are represented as "{}".

  • Empty (zero-length but non-nil) slices are represented as "[]".

  • Primitive values are converted to strings using deterministic, Go-native formatting (strconv).

The resulting map is intended for display-oriented use cases such as logging, diffing, diagnostics, or inspection. The output is not reversible and must not be treated as a lossless serialization format.

func JoinPath 

func JoinPath(path []Path) string

JoinPath converts a slice of Path objects into a string representation. Keys are joined with dots, and array indices are wrapped in square brackets. Example: [key, index(0), key] => "key[0].key".

Types

type ConfigSource added in v0.0.10 

type ConfigSource struct {
	*PropertiesStorage
	Name string
}

type LayeredStorage added in v0.0.10 

type LayeredStorage struct {
	// contains filtered or unexported fields
}

LayeredStorage aggregates multiple configuration sources with deterministic precedence rules.

The design follows the layered configuration model used by Spring-style environments: configuration values may come from multiple sources (command line, environment variables, files, etc.), and a predictable priority order determines which value wins.

Precedence rules:

  1. Layers with a lower index have higher priority.
  2. Within the same layer, sources added later override earlier ones.

For example: 

CommandLine
Environment
ProfileFile
AppFile
Default

Lookup always scans layers from highest priority to lowest.

Different data structures behave differently across layers:

  • Leaf values follow **override semantics**. The first value found wins.

  • Map properties follow **merge semantics**. Keys from all layers are combined.

  • Slice properties follow **override semantics**. The first layer defining the slice replaces all lower layers.

func (*LayeredStorage) AddStorage added in v0.0.10 

func (s *LayeredStorage) AddStorage(index int, source *PropertiesStorage, name string)

AddStorage registers a configuration source into the specified layer.

Sources within the same layer follow an override rule: the most recently added source has higher priority.

This is implemented by inserting the new source at the beginning of the slice so that iteration always sees newer sources first.

func (*LayeredStorage) Exists added in v0.0.10 

func (s *LayeredStorage) Exists(key string) bool

Exists reports whether the given key exists in any layer.

Lookup follows layer priority: once a higher-priority source reports the key exists, lower layers are not checked.

Exists considers both leaf nodes and intermediate prefixes, depending on the underlying Storage implementation.

func (*LayeredStorage) MapKeys added in v0.0.10 

func (s *LayeredStorage) MapKeys(key string, result map[string]struct{}) bool

MapKeys collects the child keys of a map node across all layers.

Unlike leaf values, map structures are merged across sources. This means keys defined in different layers are combined into a single logical map.

Example: 

source1:
    server.port=8080

source2:
    server.host=localhost

Final map: 

server.port
server.host

If the same key appears in multiple layers, the actual value resolution still follows the normal override rule in Value().

func (*LayeredStorage) SliceEntries added in v0.0.10 

func (s *LayeredStorage) SliceEntries(key string, result map[string]string) bool

SliceEntries collects slice entries for the specified key.

Lists follow an override rule across configuration layers. Once a higher-priority source defines a slice, lower layers are ignored entirely.

Example: 

source1:
    my.list[0]=a
    my.list[1]=b

source2:
    my.list[0]=c

Result: 

[c]

Not: 

[c,b]

Therefore the search stops as soon as a source containing slice entries is found.

func (*LayeredStorage) Value added in v0.0.10 

func (s *LayeredStorage) Value(key string) (string, bool)

Value retrieves the leaf value for a key.

The lookup follows override semantics across layers: the first matching value found in the highest-priority source is returned.

type Path 

type Path struct {
	// Whether the element is a key or an index.
	Type PathType

	// Actual key or index value as a string.
	// For PathTypeKey, it's the key string;
	// for PathTypeIndex, it's the index number as a string.
	Elem string
}

Path represents a single segment in a parsed key path. A path is composed of multiple Path elements that can be joined or split. For example, "foo.bar[0]" parses into: 

[{Type: PathTypeKey, Elem: "foo"},
 {Type: PathTypeKey, Elem: "bar"},
 {Type: PathTypeIndex, Elem: "0"}].

func SplitPath 

func SplitPath(key string) (_ []Path, err error)

SplitPath parses a hierarchical key string into a slice of Path objects. It supports dot-notation for maps and bracket-notation for arrays. Examples: 

"foo.bar[0]" -> [{Key:"foo"}, {Key:"bar"}, {Index:"0"}]
"a[1][2]"    -> [{Key:"a"}, {Index:"1"}, {Index:"2"}]

Rules:

  • Keys must be non-empty strings without spaces.
  • Indices must be unsigned integers (no sign, no decimal).
  • Empty maps/slices are not special-cased here.
  • Returns an error if the key is malformed (e.g. unbalanced brackets, unexpected characters, or empty keys if disallowed).

type PathType 

type PathType int8

PathType represents the type of a path element in a hierarchical key. A path element can either be a key (map field) or an index (array/slice element). 

const (
	PathTypeKey   PathType = iota // A named key in a map.
	PathTypeIndex                 // A numeric index in a list.
)

func (PathType) String added in v0.0.8 

func (t PathType) String() string

String returns the string representation of PathType.

type PrefixedStorage added in v0.0.10 

type PrefixedStorage struct {
	Storage
	Prefix string
}

PrefixedStorage wraps another Storage and automatically prepends a fixed prefix to all keys.

func NewPrefixedStorage added in v0.0.10 

func NewPrefixedStorage(s Storage, prefix string) *PrefixedStorage

NewPrefixedStorage creates a new PrefixedStorage instance.

func (*PrefixedStorage) Exists added in v0.0.10 

func (s *PrefixedStorage) Exists(key string) bool

Exists checks existence with the configured prefix.

func (*PrefixedStorage) MapKeys added in v0.0.10 

func (s *PrefixedStorage) MapKeys(key string, result map[string]struct{}) bool

MapKeys retrieves map keys with the configured prefix.

func (*PrefixedStorage) SliceEntries added in v0.0.10 

func (s *PrefixedStorage) SliceEntries(key string, result map[string]string) bool

SliceEntries retrieves slice entries with the configured prefix.

func (*PrefixedStorage) Value added in v0.0.10 

func (s *PrefixedStorage) Value(key string) (string, bool)

Value retrieves a value with the configured prefix.

type Properties added in v0.0.10 

type Properties struct {
	// contains filtered or unexported fields
}

Properties represents a flattened key-value storage.

func MapProperties added in v0.0.10 

func MapProperties(data map[string]any) *Properties

MapProperties creates a new Properties instance from a hierarchical map by flattening it into key-value pairs.

func NewProperties added in v0.0.10 

func NewProperties(data map[string]string) *Properties

NewProperties creates a new Properties instance.

func (*Properties) Data added in v0.0.10 

func (s *Properties) Data() map[string]string

Data returns the underlying flattened data.

func (*Properties) Get added in v0.0.10 

func (s *Properties) Get(key string) (string, bool)

Get retrieves the value of a leaf node.

func (*Properties) Set added in v0.0.10 

func (s *Properties) Set(key, val string)

Set sets the value of a leaf node.

type PropertiesStorage added in v0.0.10 

type PropertiesStorage struct {
	*Properties
}

PropertiesStorage adapts Properties to the Storage interface.

func NewPropertiesStorage added in v0.0.10 

func NewPropertiesStorage(s *Properties) *PropertiesStorage

NewPropertiesStorage creates a new PropertiesStorage instance.

func (*PropertiesStorage) Exists added in v0.0.10 

func (s *PropertiesStorage) Exists(key string) bool

Exists reports whether the key exists.

A key is considered existing if:

  • it exists as an exact leaf key
  • it is a prefix of other keys (intermediate node)

func (*PropertiesStorage) MapKeys added in v0.0.10 

func (s *PropertiesStorage) MapKeys(key string, result map[string]struct{}) bool

MapKeys collects child keys of a map node.

func (*PropertiesStorage) SliceEntries added in v0.0.10 

func (s *PropertiesStorage) SliceEntries(key string, result map[string]string) bool

SliceEntries collects all entries belonging to a slice node.

The implementation only checks for the presence of key[index]. It does not enforce index continuity.

func (*PropertiesStorage) Value added in v0.0.10 

func (s *PropertiesStorage) Value(key string) (string, bool)

Value retrieves the value of a leaf node.

type Storage 

type Storage interface {

	// Exists reports whether a key exists.
	//
	// A key is considered existing if:
	//   - it exists as an exact leaf key
	//   - it is a prefix of other keys
	//
	// Example:
	//
	//	server.port=8080
	//
	//	Exists("server")      -> true
	//	Exists("server.port") -> true
	//	Exists("server.host") -> false
	//
	// This method is typically used by property condition logic.
	Exists(key string) bool

	// Value returns the value of a leaf node.
	//
	// Only exact key matches are returned.
	Value(key string) (string, bool)

	// MapKeys collects the direct child keys of a map node.
	//
	// Example:
	//
	//	key = "server"
	//	data:
	//	    server.host
	//	    server.port
	//
	// result:
	//
	//	    {"host", "port"}
	//
	// Only the first map level is returned.
	MapKeys(key string, result map[string]struct{}) bool

	// SliceEntries collects all flattened entries belonging to a slice node.
	//
	// Example:
	//
	//	key = "users"
	//	data:
	//	    users[0].name
	//	    users[1].name
	//
	// result will contain all matching entries.
	SliceEntries(key string, result map[string]string) bool
}

Storage defines the minimal abstraction required by the bind system.

Data is stored in flattened form, for example: 

server.port=8080
server.host=localhost
users[0].name=tom

The implementation assumes the input data is already valid. Storage itself does not validate structural correctness.

The interface provides three capabilities used during binding:

  • leaf value lookup
  • map key discovery
  • slice entry discovery

Exists is mainly intended for property condition checks rather than binding.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.