store

package module
v1.3.55 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 30, 2025 License: Apache-2.0 Imports: 16 Imported by: 26

README

Store

nightly-build release-build GitHub go.mod Go version GitHub Release Go Report Card Coverage Status Go Dev deps.dev Mentioned in Awesome Go

hedzr/store provides an extensible, high-performance configuration management library. It is optimized for accessing hierarchical data.

The special is put any data and extract typed it. Which means, the store will try to convert the source data within underlying.

Another feature is the store traces config items' modification states. So you can extract the changed subset. See also Modified State.

The store is designed to provide the basic skeleton for hedzr/cmdr v2 (RC1 released). It also could be used as a standalone config manager. We split the original config (called as option-store at cmdr v1) and rewrote it. In the refactoring phrase, some excellent config-mgmt/centers get many tips to us, such as koanf and viper. Many respects.

The store accesses tree data with a dotted key path, that means you may point to a specified tree node and access it, modify it, monitor it or remove it. You can use a different delimiter char like / or \.

conf := store.New()
conf.Set("app.debug", false)
conf.Set("app.verbose", true)
conf.Set("app.dump", 3)
conf.Set("app.logging.file", "/tmp/1.log")
conf.Set("app.server.start", 5)

ss := conf.WithPrefix("app.logging")
ss.Set("rotate", 6)
ss.Set("words", []any{"a", 1, false})
ss.Set("keys", map[any]any{"a": 3.13, 1.73: "zz", false: true})

// Tag, Comment & Description
conf.Set("app.bool", "[on,off,   true]")
conf.SetComment("app.bool", "desc: a bool slice", "cmmt: remarks here")
conf.SetTag("app.bool", []any{"on", "off", true})
// println(conf.MustGetTag("app.bool"))
// println(conf.MustGetComment("app.bool"))

radix.StatesEnvSetColorMode(true) // to disable ansi escape sequences in dump output
_, _ = fmt.Println(conf.Dump())   // inspect it

conf.GetEx("app.bool", func(node radix.Node[any], data any, branch bool, kvpair radix.KVPair) {
  println(node.Comment())
})
conf.Update("app.bool", func(node radix.Node[any], old any) {
  node.SetComment("a bool slice", "remarks here")
})

// TTL to clear the node data to zero
conf.SetTTL("app.bool", 3*time.Second, func(_ *radix.TTL[any], nd radix.Node[any]) {
  log.Printf("%q (%v) cleared", "app.bool", nd.Data())
})  // since v1.2.5
defer conf.Close()  // when you used SetTTL, the Close() is must be had.

// Set/create a node at once by SetEx()
conf.SetEx("app.logging.auto-stop", true,
  func(path string, oldData any, node radix.Node[any], trie radix.Trie[any]) {
    trie.SetTTL(path, 30*time.Minute,
      func(s *radix.TTL[any], node radix.Node[any]) {
          trie.Remove(node.Key()) // erase the key with the node
      })
    // Or:
    trie.SetTTLFast(node, 39*time.Second, nil)
    // Or:
    node.SetTTL(39*time.Second, trie, nil)
  })

The dumping result looks like (internal data structure):

  app.                          <B>
    d                           <B>
      ebug                      <L> app.debug => false
      ump                       <L> app.dump => 3
    verbose                     <L> app.verbose => true
    logging.                    <B>
      file                      <L> app.logging.file => /tmp/1.log
      rotate                    <L> app.logging.rotate => 6
      words                     <L> app.logging.words => [a 1 false]
      keys                      <L> app.logging.keys => map[a:3.13 1.73:zz false:true]
    server.start                <L> app.server.start => 5
    bool                        <L> app.bool => [on,off,   true] // remarks here | tag = [on off true] ~ a bool slice

As you seen, the internal structure will be printed out for the deep researching.

image-20240221115843477

<B> is branch, <L> is leaf.

Leaf node contains data, comment, description and tag (any value).

To speed up the tree, any delimiter char is a part of the path.

The store provides advanced APIs to extract the typed data from some a node,

iData := conf.MustInt("app.logging.rotate")
stringData := conf.MustString("app.logging.rotate")
debugMode := conf.MustBool("app.debug")
...

The searching tools are also used to locate whether a key exists or not:

found := conf.Has("app.logging.rotate")
node, isBranch, isPartialMatched, found := conf.Locate("app.logging.rotate")
t.Logf("%v | %s | %v |     | %v, %v, found", node.Data(), node.Comment(), node.Tag(), isBranch, isPartialMatched, found)

Locate is a more friendly Has test for the developers when they want to inspect more extra information after searching.

For more information, browse these public sites:

To see the recently changes at CHANGELOG.

Since v1.1.0, unexported struct ptr (*storeS) removed from Store API.

These apis changed to:

  • Clone() (newStore Store)
  • Dup() (newStore Store)
  • WithPrefix(prefix ...string) (newStore Store)
  • WithPrefixReplaced(newPrefix ...string) (newStore Store)

Since v1.2.0, the prototypes of Locate/Query are changed.

  • an extra kvpair will be returned if there is :ident in trie path and matched ok.
  • support these url var matching: "/:id/", "/*filepath"

For example, matching /hello/bob on a router path pattern /hello/:name will get kvpair = {"name":"bob"}, and /search/any/thing/here on pattern /search/*keywords will get kvpair = {"keywords":"any/thing/here"}.

Since v1.2.5, SetTTL(path, ttl, callback) added.
Since v1.2.8, SetTTLFast(node, ttl, callback) added.
Since v1.2.8, SetEx(path, val, callback) added.
Since v1.3.0, moved the minimal toolchain to go1.23.7.
Since v1.3.55, moved the minimal toolchain to go1.24.5.

Using store as the core of a http router is possible. Since v1.2 we added builtin http route param support (like :user).

Using hedzr/store as a in-memory Cache provider is possible. Since v1.2.5 we added SetTTL to reset the data of a node (by key) to zero value. You can inject your codes to drop the key or whatever else.

More Features

The store gives many advanced features from out of the box, but the relative documents are not enough. We will try our best to fill more documentation at a certain point in the future.

In short, the store can load the data from a provider which will load from its external source, which data will be decoded by a codec decoder.

Once your configuration data is loaded or inserted into the store manually, you can read them at any time, in any way. That is, the original items can be extracted with a different data type if they can convert smoothly. For example, a string item 3d3h3s can be got as a time.Duration value (via MustDuration(path)).

In totally:

  • access fastly to the hierarchical tree data
  • app configurations management
  • load app configs from external sources
    • various codec translators and source providers
  • merge config fragment
  • save the modifications to alternated external source
  • standalone in-memory key-value pair
  • as a in memory cache provider
  • as a kernel of a http-router
  • highly customizable
Retrieve Node Data

A config entry, so-called as a node (in our Trie-tree), can be retrieved as a typed value:

func ExampleStoreS_Get() {
    trie := newBasicStore()
    fmt.Println(trie.MustInt("app.dump"))
    fmt.Println(trie.MustString("app.dump"))
    fmt.Println(trie.MustBool("app.dump")) // convert 3 to bool will get false, only 1 -> true.
    // Output:
    // 3
    // 3
    // false
}

func newBasicStore(opts ...Opt) *storeS {
    conf := New(opts...)
    conf.Set("app.debug", false)
    conf.Set("app.verbose", true)
    conf.Set("app.dump", 3)
    conf.Set("app.logging.file", "/tmp/1.log")
    conf.Set("app.server.start", 5)

    ss := conf.WithPrefix("app.logging")
    ss.Set("rotate", 6)
    ss.Set("words", []any{"a", 1, false})
    return conf
}
Extract A Subset

GetM(path, opts...) map[string]any is a power tool to extract the nodes as a map, which has the flattened keys. The extracted result looks like:

store_test.go:150: whole tree: map[app.debug:false app.dump:3 app.logging.file:/tmp/1.log app.logging.rotate:6 app.logging.words:[a 1 false] app.server.start:5 app.verbose:true]
store_test.go:160: app.logging sub-tree: map[app.logging.file:/tmp/1.log app.logging.rotate:6 app.logging.words:[a 1 false]]

The test code is:

func TestStore_GetM(t *testing.T) {
    conf := newBasicStore()

    m, err := conf.GetM("")
    if err != nil {
        t.Fatalf("wrong in calling GetM(\"\"): %v", err)
    }
    t.Logf("whole tree: %v", m)

    // filter by a functor

    m, err = conf.GetM("", WithFilter[any](func(node radix.Node[any]) bool {
        return strings.HasPrefix(node.Key(), "app.logging.")
    }))
    if err != nil {
        t.Fatalf("wrong in calling GetM(\"\"): %v", err)
    }
    t.Logf("app.logging sub-tree: %v", m)
}

GetM("") can extract the whole tree, and GetM("app.logging") extract that subtree.

With filter functor, you can extract app.logging subtree by GetM("", WithFilter[any](func(node radix.Node[any]) bool { return strings.HasPrefix(node.Key(), "app.logging.") }))).

Extract Subtree Into Struct

GetSectionFrom makes extracting to struct easier. For example,

func TestStore_GetSectionFrom(t *testing.T) {
    conf := newBasicStore()
    conf.Set("app.logging.words", []any{"a", 1, false})
    conf.Set("app.server.sites", -1)
    t.Logf("\nPath\n%v\n", conf.Dump())

    type loggingS struct {
        File   uint
        Rotate uint64
        Words  []any
    }

    type serverS struct {
        Start int
        Sites int
    }

    type appS struct {
        Debug   int
        Dump    int
        Verbose int64
        Logging loggingS
        Server  serverS
    }

    type cfgS struct {
        App appS
    }

    var ss cfgS
    err := conf.GetSectionFrom("", &ss) // extract the whole tree
    t.Logf("cfgS: %v | err: %v", ss, err)

    assertEqual(t, []any{"a", 1, false}, ss.App.Logging.Words)
    assertEqual(t, -1, ss.App.Server.Sites)

    if !reflect.DeepEqual(ss.App.Logging.Words, []any{"a", 1, false}) {
        t.Fail()
    }
}

TODO: Transferring a struct into the Store isn't in our plan yet.

Light-weight Sub-tree

The store has a dead lightweight subtree accessor. By using WithPrefix or WithPrefixReplaced, you can construct a subtree accessor and read/write a node:

func TestStore_WithPrefix(t *testing.T) {
    trie := newBasicStore()
    t.Logf("\nPath\n%v\n", trie.Dump())

    assertEqual(t, 6, trie.MustGet("app.logging.rotate"))
    conf := trie.WithPrefix("app")
    assertEqual(t, 6, conf.MustGet("logging.rotate"))
  
    conf = conf.WithPrefix("logging")
    assertEqual(t, 6, conf.MustGet("rotate"))
    
  conf = trie.WithPrefixReplaced("app.logging")
    assertEqual(t, 6, conf.MustGet("rotate"))
}
Easily Cutting

By using SetPrefix(prefix), a store and its whole subtree can be moved or associated with a new hierarchical tree structure.

By using Dup or Clone, and Merge, the store can be cut and layout.

Split key with delimiter

If a key contains delimiter, it will be split and insert into the Store. Technically, the inserter doesn't do special stuff for this key, but the getter will access the tree path separated by the delimiter char.

So when you're loading a YAML file (or others), the dotted key can make the file consicer:

app.demo.working: "~/demo"

It equals

app:
  demo:
    working: "~/demo"

This feature is builtin and cannot disable, due to we have a Trie-tree store and the node is always recognized at extracting.

A side effect is, when you're using a float-point number as a key, that will have matters. Our tip is, don't do that.

Decompound Map

The data can be split and orchestrated into tree structure when you're inserting a map.

This feature works when a provider is loading its external source. Set(k, v) doesn't decompound anything in v. But Merge(k, m) does:

func TestDecompoundMap(t *testing.T) {
    conf := newBasicStore()

    conf.Set("app.map", false) // ensure key path 'app.map' has already existed
    // and now merge a map into the point/node
    err := conf.Merge("app.map", map[string]any{
        "k1": 1,
        "k2": false,
        "m3": map[string]any{
            "bobo": "joe",
        },
    })

    if err != nil {
        t.Fatalf("Merge failed: %v", err)
    }

    assert.Equal(t, int(1), conf.MustGet("app.map.k1"))
    assert.Equal(t, false, conf.MustGet("app.map.k2"))
    assert.Equal(t, "joe", conf.MustGet("app.map.m3.bobo"))
}

Of course, it shall be a valid deep map[string]any.

Decompound Slice

A slice can be decompounded once you enabled WithStoreFlattenSlice(true).

It works for loading an external source, similar like Decompounding Map.

See the sample code for collapsed sections

For example:

func TestHjson(t *testing.T) {
    s := store.New()
    parser := hjson.New()
    if err := s.Load(context.TODO(),
        store.WithStorePrefix("app.hjson"),
        store.WithCodec(parser),
        store.WithProvider(file.New("../testdata/6.hjson")),

        store.WithStoreFlattenSlice(true),
    ); err != nil {
        t.Fatalf("Load failed: %v", err)
    }
    t.Logf("\n%-32sData\n%v\n", "Path", s.Dump())

    assert.Equal(t, `r.Header.Get("From")`, s.MustGet("app.hjson.messages.0.placeholders.0.expr"))
    assert.Equal(t, `r.Header.Get("User-Agent")`, s.MustGet("app.hjson.messages.1.placeholders.0.expr"))
}

The supplied hjson file has the following contents:

{
  "language": "zh",
  "messages": [
    {
      "id": "Hello {From}!",
      "message": "Hello {From}!",
      "translation": "",
      "placeholders": [
        {
          "id": "From",
          "string": "%[1]s",
          "type": "string",
          "underlyingType": "string",
          "argNum": 1,
          "expr": "r.Header.Get(\"From\")"
        }
      ]
    },
    {
      "id": "Do you like your browser ({User_Agent})?",
      "message": "Do you like your browser ({User_Agent})?",
      "translation": "",
      "placeholders": [
        {
          "id": "User_Agent",
          "string": "%[1]s",
          "type": "string",
          "underlyingType": "string",
          "argNum": 1,
          "expr": "r.Header.Get(\"User-Agent\")"
        }
      ]
    }
  ]
}
Notable Nodes

Different from other configuration managers, the store is not only a memory key-value store. The nodes in store are both notable and taggable.

conf.Set("debug", false)
conf.SetComment("debug", "a flag to identify app debug mode", "remarks here")
conf.SetTag("debug", map[string]any{
    "handler": func(){},
})

node, _, _, found := conf.Locate("debug")
if found {
    t.Log(node.Tag(), node.Description(), node.Comment())
}
t.Log(conf.Dump())

Dump() will produce the detailed output.

Walk The Whole Tree

Walk(path) gives a way to iterator the Store.

func TestStore_Walk(t *testing.T) {
    var conf Store = newBasicStore()
    conf.Walk("", func(path, fragment string, node radix.Node[any]) {
        t.Logf("%v / %v => %v", path, fragment, node)
    })
}

// Output:
//  /  => &{[]  [0xc0000e23f0] <nil>   <nil> 0}
// app. / app. => &{[97 112 112 46] app. [0xc0000e2480 0xc0000e2510 0xc0000e26c0 0xc0000e2750] false   <nil> 0}
// app.d / d => &{[100] app.d [0xc0000e25a0 0xc0000e2630] false   <nil> 0}
// app.debug / ebug => &{[101 98 117 103] app.debug [] false   <nil> 13}
// app.dump / ump => &{[117 109 112] app.dump [] 3   <nil> 13}
// app.verbose / verbose => &{[118 101 114 98 111 115 101] app.verbose [] true   <nil> 13}
// app.logging. / logging. => &{[108 111 103 103 105 110 103 46] app.logging. [0xc0000e2870 0xc0000e2900 0xc0000e2990] /tmp/1.log   <nil> 0}
// app.logging.file / file => &{[102 105 108 101] app.logging.file [] /tmp/1.log   <nil> 13}
// app.logging.rotate / rotate => &{[114 111 116 97 116 101] app.logging.rotate [] 6   <nil> 13}
// app.logging.words / words => &{[119 111 114 100 115] app.logging.words [] [a 1 false]   <nil> 13}
// app.server.start / server.start => &{[115 101 114 118 101 114 46 115 116 97 114 116] app.server.start [] 5   <nil> 13}

As a feature based on Trie-tree, Walk("app") will walk from the parent of app. node. And Walk("app.") will walk from the app. node.

Like GetM, passing "" will walk from the top-level root node.

Modified State

Each node has a modified state, so we can extract them from the Store:

func (s *loadS) Save(ctx context.Context) (err error) { return s.trySave(ctx) }
func (s *loadS) trySave(ctx context.Context) (err error) {
    if s.codec != nil {
        var m map[string]any
        if m, err = s.GetM("", WithFilter[any](func(node radix.Node[any]) bool {
            return node.Modified()
        })); err == nil {
            var data []byte
            if data, err = s.codec.Marshal(m); err == nil {
                switch fp := s.provider.(type) {
                case OnceProvider:
                    err = fp.Write(data)
                default:
                    err = ErrNotImplemented
                }

                if errors.Is(err, ErrNotImplemented) {
                    if wr, ok := s.provider.(io.Writer); ok {
                        _, err = wr.Write(data)
                    }
                }
            }
        }
    }
    return
}

We assume the user calling Set(k, v) will cause modified state was set to true. And app loading and merging to the Store at startup will be treated as initial state, so the modified state keeps unset (i.e., false).

Provider for External Source

Providers could be used to describe an external source, such as file, env, or consul and vice versa.

Codecs are used to describe how to decode a streaming input loaded by Provider, such as yaml, toml, json, hcl, etc.

A loading logic typically is:

func TestTOML(t *testing.T) {
    s := store.New()
    parser := toml.New()
    if err := s.Load(context.TODO(),
        store.WithStorePrefix("app.toml"),
        store.WithCodec(parser),
        store.WithProvider(file.New("../testdata/5.toml")),

        store.WithStoreFlattenSlice(true),
    ); err != nil {
        t.Fatalf("Load failed: %v", err)
    }
    t.Logf("\n%-32sData\n%v\n", "Path", s.Dump())

    assert.Equal(t, `127.0.0.1`, s.MustGet("app.toml.host"))
    assert.Equal(t, `TLS 1.3`, s.MustGet("app.toml.TLS.version"))
    assert.Equal(t, `AEAD-AES128-GCM-SHA256`, s.MustGet("app.toml.TLS.cipher"))
    assert.Equal(t, `go`, s.MustGet("app.toml.tags.0"))
}

More tests at tests/*_test.go .

Implement A Provider

A Provider should support Read():

type Provider interface {
    Read() (m map[string]any, err error) // return ErrNotImplemented as an identifier

    ProviderSupports
}

type ProviderSupports interface {
    GetCodec() (codec Codec)   // return the bound codec decoder
    GetPosition() (pos string) // return a position pointed to a trie node path
    WithCodec(codec Codec)
    WithPosition(pos string)
}

Your provider can support OnceProvider or StreamProvider while its Read return radix.ErrNotImplemented. OnceProvider assumes the loader read binary content at once. StreamProvider allows reading the large content progressively.

Set TTL for A Key

Since v1.2.5, we added TTL support for a trie node. Once the TTL arrives, the data of the target key will be cleared. Which means, the data field would be reset to zero value (or a nil value for an any type T).

The code could be:

func TestStore_SetTTL(t *testing.T) {
	conf := newBasicStore()
	defer conf.Close()

	path := "app.logging.rotate"
	conf.SetTTL(path, 200*time.Millisecond, func(s *radix.TTL[any], nd radix.Node[any]) {
		t.Logf("%q cleared", path)
	})
	path2 := "app.logging.file"
	conf.SetTTL(path2, 200*time.Millisecond, func(s *radix.TTL[any], nd radix.Node[any]) {
		t.Logf("%q (%q) cleared", path2, nd.Data())
	})

	time.Sleep(450 * time.Millisecond)
	assertEqual(t, true, conf.Has(path2))
	assertEqual(t, nil, conf.MustGet(path2))
	assertEqual(t, 0, conf.MustInt(path))
}

Benchmarks

The store is designed to reduce the allocations much better, and up the performance much better. We have a zero-allocation implementation in reading a key-value pair, currently.

Our benchmark testing (test/bench_test.go) shows:

goos: darwin
goarch: amd64
pkg: github.com/hedzr/store/tests
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkTrieSearch
BenchmarkTrieSearch/hedzr/storeT[any]
BenchmarkTrieSearch/hedzr/storeT[any]-16             59983291            18.99 ns/op           0 B/op           0 allocs/op
BenchmarkTrieSearch/hedzr/store
BenchmarkTrieSearch/hedzr/store-16                   60454639            19.43 ns/op           0 B/op           0 allocs/op
PASS

Some control groups with the same executive environment produced:

...
BenchmarkTrieSearch/kzzzz
BenchmarkTrieSearch/kzzzz-16                         46264582            28.88 ns/op          16 B/op           1 allocs/op
BenchmarkTrieSearch/vzzzz
BenchmarkTrieSearch/vzzzz-16                         22824562            51.21 ns/op          32 B/op           2 allocs/op
...

  1. To avoid controversy, pkg-name masked.

  2. Both of these testing data sets have the same scale basically (lower than 20 keys). Also, the querying words are same.

    Screenshot 2024-02-22 at 10.55.13

  3. No Warrenties.

The performance benefits mainly from the refresh implemented about our internal Trie-tree (radix-tree).

As an addition, here are huger/larger benches:

Test updated:
goos: darwin
goarch: arm64
pkg: github.com/hedzr/go-zapall/triebench
cpu: Apple M3 Pro
    ./triebench/bench_test.go:35: store/normal keys: 104 keys
    ./triebench/bench_test.go:37: kxxxx/normal keys: 102 keys
    ./triebench/bench_test.go:39: bxxxxxxx/normal keys: 106 keys
    ./triebench/bench_test.go:55: store/Huge keys: 422 keys
    ./triebench/bench_test.go:57: kxxxx/Huge keys: 318 keys
BenchmarkTrieSearch/hedzr/storeT[any]
BenchmarkTrieSearch/hedzr/storeT[any]-11         	96411034	        12.25 ns/op	       9 B/op	       0 allocs/op
BenchmarkTrieSearch/hedzr/store
BenchmarkTrieSearch/hedzr/store-11               	40387551	        29.89 ns/op	       9 B/op	       0 allocs/op
BenchmarkTrieSearch/hedzr/store/Huge
BenchmarkTrieSearch/hedzr/store/Huge-11          	24667755	        50.84 ns/op	     222 B/op	       0 allocs/op
BenchmarkTrieSearch/kxxxx
BenchmarkTrieSearch/kxxxx-11                     	32698178	        36.40 ns/op	      16 B/op	       1 allocs/op
BenchmarkTrieSearch/kxxxx/Huge
BenchmarkTrieSearch/kxxxx/Huge-11                	11557538	       106.6 ns/op	     459 B/op	      12 allocs/op
BenchmarkTrieSearch/bxxxe
BenchmarkTrieSearch/bxxxe-11                     	17893599	        66.24 ns/op	      88 B/op	       2 allocs/op
BenchmarkTrieSearch/vxxxx
BenchmarkTrieSearch/vxxxx-11                     	22784323	        53.80 ns/op	      32 B/op	       2 allocs/op

Test again:
    ./triebench/bench_test.go:35: store/normal keys: 105 keys
    ./triebench/bench_test.go:37: kxxxx/normal keys: 103 keys
    ./triebench/bench_test.go:39: bxxxxxxx/normal keys: 106 keys
    ./triebench/bench_test.go:55: store/Huge keys: 422 keys
    ./triebench/bench_test.go:57: kxxxx/Huge keys: 318 keys
BenchmarkTrieSearch/hedzr/storeT[any]
BenchmarkTrieSearch/hedzr/storeT[any]-11         	86327053	        11.59 ns/op	       8 B/op	       0 allocs/op
BenchmarkTrieSearch/hedzr/store
BenchmarkTrieSearch/hedzr/store-11               	38669240	        29.97 ns/op	       8 B/op	       0 allocs/op
BenchmarkTrieSearch/hedzr/store/Huge
BenchmarkTrieSearch/hedzr/store/Huge-11          	24919380	        49.81 ns/op	     221 B/op	       0 allocs/op
BenchmarkTrieSearch/kxxxx
BenchmarkTrieSearch/kxxxx-11                     	32515147	        36.97 ns/op	      16 B/op	       1 allocs/op
BenchmarkTrieSearch/kxxxx/Huge
BenchmarkTrieSearch/kxxxx/Huge-11                	12414290	        96.13 ns/op	     411 B/op	      11 allocs/op
BenchmarkTrieSearch/bxxxx
BenchmarkTrieSearch/bxxxx-11                     	17892199	        67.23 ns/op	      89 B/op	       2 allocs/op
BenchmarkTrieSearch/vxxxx
BenchmarkTrieSearch/vxxxx-11                     	22975539	        53.40 ns/op	      32 B/op	       2 allocs/op

This benchmark tests ruled out the effects from consutructing the config set, and do a large numbers of GET a key as bench result. UPDATED at 2025-07-03 with both of their recent versions.

You can find out that our store has a better score while working on a large configuration set, although it might take more latency than on a tiny set.

The datasource of huge test is a pet-store openapi swagger doc, coming from https://editor.swagger.io/. With a same input like a YAML file, the store could get more key-value pairs because store.WithStoreFlattenSlice(true) applied, which will expand slices and maps in a value as nested key-value pairs.

Dependencies

The store[^1] imports some modules of mine:

  1. [hedzr/evendeep[^2]]
  2. [hedzr/logg/slog[^3]]
  3. [hedzr/errors.v3[^4]]
  4. [hedzr/is[^5]]

The dependency graph is:

graph BT
  hzis(hedzr/is)-->hzlogg(hedzr/logg/slog)
  hzis-->hzdiff(hedzr/evendeep)
  hzlogg-->hzdiff
  hzerrors(gopkg.in/hedzr/errors.v3)-->hzdiff
  hzerrors-->hzstore(hedzr/store)
  hzis-->hzstore(hedzr/store)
  hzlogg-->hzstore(hedzr/store)
  hzdiff-->hzstore(hedzr/store)

[^1]: hedzr/store is a high-performance configure management library [^2]: hedzr/evendeep offers a customizable deepcopy tool to you. There are also deepequal, deepdiff tools in it. [^3]: hedzr/logg provides a slog like and colorful logging library [^4]: hedzr/errors.v3 provides some extensions and compatible layer over go1.11 ~ nowadays. [^5]: hedzr/is is a basic environ detectors library

LICENSE

Apache 2.0

Documentation

Overview

Package store provides an extensible, high-performance configuration management library, specially optimized for hierarchical data.

The Store interface gives these APIs.

The `hedzr/store` (https://github.com/hedzr/store) accesses tree data with a dotted key path, which means you may point to a specified tree node and access it, monitor it or remove it. 

conf := store.New()
conf.Set("app.debug", false)
conf.Set("app.verbose", true)
conf.Set("app.dump", 3)
conf.Set("app.logging.file", "/tmp/1.log")
conf.Set("app.server.start", 5)

ss := conf.WithPrefix("app.logging")
ss.Set("rotate", 6)
ss.Set("words", []any{"a", 1, false})
ss.Set("keys", map[any]any{"a": 3.13, 1.73: "zz", false: true})

conf.Set("app.bool", "[on,off,   true]")
conf.SetComment("app.bool", "a bool slice", "remarks here")
conf.SetTag("app.bool", []any{"on", "off", true})

states.Env().SetNoColorMode(true) // to disable ansi escape sequences in dump output
fmt.Println(conf.Dump())

data, found := conf.Get("app.logging.rotate")
println(data, found)
data := conf.MustGet("app.logging.rotate")
println(data)

The `store` provides advanced APIs to extract typed data from node. 

iData := conf.MustInt("app.logging.rotate")
debugMode := conf.MustBool("app.debug")
...

The searching tool is also used to locate whether a key exists or not: 

found := conf.Has("app.logging.rotate")
node, isBranch, isPartialMatched, found := conf.Locate("app.logging.rotate")
t.Logf("%v | %s | %v |     | %v, %v, found: %v", node.Data(), node.Comment(), node.Tag(), isBranch, isPartialMatched, found)

The `store` provides many providers and codecs. A provider represents an external data source, such as file, environment, consul, etc. And a codec represents the data format, just like yaml, json, toml, etc.

So an app can [Store.Load] the external yaml files like the following way: 

func TestStoreS_Load(t *testing.T) {
    conf := newBasicStore(WithWatchEnable(true))
    defer conf.Close()
    ctx := context.Background()

    parser := yaml.New()
    _, err := conf.Load(ctx,
        store.WithStorePrefix("app.yaml"),
        store.WithCodec(parser),
        store.WithProvider(file.New("../../../testdata/2.yaml")),

        store.WithStoreFlattenSlice(true), // decode and flatten slice into tree structure instead treat it as a simple value
    )

    assert.Equal(t, `-s`, s.MustGet("app.yaml.app.bgo.build.projects.000-default-group.items.001-bgo.ldflags.0"))
    assert.Equal(t, `-w`, s.MustGet("app.yaml.app.bgo.build.projects.000-default-group.items.001-bgo.ldflags.1"))

    m := map[string]any{
        "m1.s1": "cool",
        "m1.s2": 9,
        "key2": map[any]any{
            9: 1,
            8: false,
        },
        "slice": []map[any]any{
            {7.981: true, "cool": "maps"},
            {"hello": "world"},
        },
    }
    _, err := conf.Load(ctx,
        WithProvider(maps.New(m, ".")),
        WithStoreFlattenSlice(true),
        WithStorePrefix("app.maps"),
        WithPosition(""),
    )
    if ErrorIsNotFound(err) {
        t.Fail()
    }
    if err != nil {
        t.Fatalf("err: %v", err)
    }

    t.Logf("\nPath of 'conf' (delimeter=%v, prefix=%v)\n%v\n",
        conf.Delimiter(),
        conf.Prefix(),
        conf.Dump())

    assertEqual(t, false, conf.MustBool("app.maps.key2.8"))
    assertEqual(t, 1, conf.MustInt("app.maps.key2.9", -1))
    assertEqual(t, "cool", conf.MustString("app.maps.m1.s1"))
    assertEqual(t, 9, conf.MustInt("app.maps.m1.s2", -1))
}

For more information, browse these public sites:

- https://pkg.go.dev/github.com/hedzr/store

- https://github.com/hedzr/store

Index

Constants

View Source 
const Version = "v1.3.55" // Version of libs.store

Variables

View Source 
var ErrNotImplemented = stderr.New("not implemented")

ErrNotImplemented is used to identify unimplemented API.

View Source 
var ErrWritableDisabled = errors.New("writeable flag disabled")

Functions

func ErrorIsNotFound 

func ErrorIsNotFound(err error) bool

ErrorIsNotFound checks if TypedGetters returning a NotFound error. 

_, err := trie.GetFloat64("app.dump.")
println(store.ErrorIsNotFound(err))       # this should be 'true'

If you don't care about these errors, use MustXXX such as radix.Trie.MustFloat64.

func NewDummyStore 

func NewDummyStore() *dummyS

NewDummyStore returns an empty store with dummy abilities implemented.

func To added in v1.3.2 

func To[T any](s Store, path string, holder *T, opts ...radix.MOpt[any]) (err error)

To finds a given path and loads the subtree into 'holder', typically 'holder' could be a struct.

For yaml input 

app:
  server:
    sites:
      - name: default
        addr: ":7999"
        location: ~/Downloads/w/docs

The following codes can load it into sitesS struct: 

var sites sitesS
err = store.To(store.WithPrefix("app"), "server.sites", &sites)

type sitesS struct{ Sites []siteS }

type siteS struct {
  Name        string
  Addr        string
  Location    string
}

In this above case, 'store' loaded yaml and built it into memory, and extract 'server.sites' into 'sitesS'. Since 'server.sites' is a yaml array, it was loaded as a store entry and holds a slice value, so GetSectionFrom extract it to sitesS.Sites field.

The optional MOpt operators could be:

  • WithKeepPrefix
  • WithFilter

func WithFilter 

func WithFilter[T any](filter radix.FilterFn[T]) radix.MOpt[T]

WithFilter can be used in calling GetM(path, ...)

func WithKeepPrefix 

func WithKeepPrefix[T any](b bool) radix.MOpt[T]

WithKeepPrefix can construct tree nodes hierarchy with the key prefix.

By default, the prefix will be stripped from a given key path.

For example, if a store has a prefix 'app.server', `store.Put("app.server.tls", map[string]any{ "certs": "some/where.pem" }` will produce the tree structure like: 

app.
  Server.
    tls.
      certs   => "some/where.pem"

But if you enable keep-prefix setting, the code can only be written as: 

store.Put("tls", map[string]any{ "certs": "some/where.pem" }

We recommend using our default setting except that you knew what you want. By using the default setting, i.e., keepPrefix == false, we will strip the may-be-there prefix if necessary. So both "app.server.tls" and "tls" will work properly as you really want.

func WithoutFlattenKeys 

func WithoutFlattenKeys[T any](b bool) radix.MOpt[T]

WithoutFlattenKeys allows returns a nested map. If the keys contain delimiter char, they will be split as nested sub-map.

Types

type Change 

type Change interface {
	Next() (key string, val any, ok bool)

	Path() string // specially for 'file' provider

	Op() Op //
	Has(op Op) bool
	Timestamp() time.Time

	Provider() Provider
}

Change is an abstract interface for Watchable object.

type Codec 

type Codec interface {
	Marshal(m map[string]any) (data []byte, err error)
	Unmarshal(b []byte) (data map[string]any, err error)
}

Codec is decoder and/or encoder for text format.

For example, a file can be encoded with JSON format. So you need a JSON codec parser here.

Well-known codec parsers can be JSON, YAML, TOML, ....

type CodecEx 

type CodecEx interface {
	MarshalEx(m map[string]ValPkg) (data []byte, err error)
	UnmarshalEx(b []byte) (data map[string]ValPkg, err error)
}

CodecEx reserved.

type Dumpable 

type Dumpable interface {
	Dump() string
}

Dumpable interface identify an object can be represented as a string for debugging.

type FallbackProvider 

type FallbackProvider interface {
	Reader() (r Reader, err error) // return ErrNotImplemented as an identifier if it wants to be skipped

	ProviderSupports
}

FallbackProvider reserved for future.

type LoadOpt 

type LoadOpt func(*Loader) // options for loadS

func WithCodec 

func WithCodec(codec Codec) LoadOpt

WithCodec specify the decoder to decode the loaded data.

func WithLoaderCopy added in v1.3.45 

func WithLoaderCopy(copy **Loader) LoadOpt

func WithPosition 

func WithPosition(position string) LoadOpt

WithPosition sets the

func WithProvider 

func WithProvider(provider Provider) LoadOpt

WithProvider is commonly required. It specify what Provider will be [storeS.Load].

func WithStoreFlattenSlice 

func WithStoreFlattenSlice(b bool) LoadOpt

WithStoreFlattenSlice can destruct slice/map as tree hierarchy instead of treating it as a node value.

func WithStorePrefix 

func WithStorePrefix(prefix string) LoadOpt

WithStorePrefix gives a prefix position, which is the store location that the external settings will be merged at.

func WithoutWatch added in v1.3.45 

func WithoutWatch(dontWatch bool) LoadOpt

type Loader added in v1.3.45 

type Loader struct {
	// contains filtered or unexported fields
}

func (Loader) BR added in v1.3.45 

func (s Loader) BR() (newStore Store)

func (Loader) Clone added in v1.3.45 

func (s Loader) Clone() (newStore Store)

func (Loader) Close added in v1.3.45 

func (s Loader) Close()

Close cleanup the internal resources. See [basics.Peripheral] for more information.

func (Loader) Dump added in v1.3.45 

func (s Loader) Dump() (text string)

Dump prints internal data tree for debugging

func (Loader) Dup added in v1.3.45 

func (s Loader) Dup() (newStore Store)

Dup is a native Clone tool.

After Dup, a copy of the original store will be created, but closers not. Most of the closers are cleanup code fragments coming from Load(WithProvider()), some of them needs to shut down the remote connection such as what want to do by consul provider.

At this scene, the parent store still holds the cleanup closers.

func (Loader) Get added in v1.3.45 

func (s Loader) Get(path string) (data any, found bool)

Get the value at path point 'path'.

func (Loader) Has added in v1.3.45 

func (s Loader) Has(path string) (found bool)

Has tests if the given path exists.

func (Loader) Load added in v1.3.45 

func (s Loader) Load(ctx context.Context, opts ...LoadOpt) (wr Writeable, err error)

Load loads an external data source by the specified Provider, a Codec parser is optional.

WithProvider and WithCodec are useful. The sample code is: 

s := newBasicStore()
if _, err := s.Load(
   context.TODO(),
   store.WithStorePrefix("app.json"),
   store.WithCodec(json.New()),
   store.WithProvider(file.New("../testdata/4.json")),

   store.WithStoreFlattenSlice(true),
); err != nil {
   t.Fatalf("failed: %v", err)
}

func (Loader) Locate added in v1.3.45 

func (s Loader) Locate(path string, kvpair radix.KVPair) (node radix.Node[any], branch, partialMatched, found bool)

Locate provides an advanced interface for locating a path.

func (Loader) MarshalJSON added in v1.3.45 

func (s Loader) MarshalJSON() ([]byte, error)

func (Loader) Merge added in v1.3.45 

func (s Loader) Merge(pathAt string, data map[string]any) (err error)

Merge a map at path point 'pathAt'.

func (Loader) MustGet added in v1.3.45 

func (s Loader) MustGet(path string) (data any)

MustGet is a shortcut to Get without error returning.

func (Loader) N added in v1.3.45 

func (s Loader) N() (newStore Store)

func (Loader) R added in v1.3.45 

func (s Loader) R() (newStore Store)

func (Loader) RecursiveMode added in v1.3.45 

func (s Loader) RecursiveMode() radix.RecusiveMode

func (Loader) Remove added in v1.3.45 

func (s Loader) Remove(path string) (removed bool)

func (Loader) RemoveEx added in v1.3.45 

func (s Loader) RemoveEx(path string) (nodeRemoved, nodeParent radix.Node[any], removed bool)

func (*Loader) Save added in v1.3.45 

func (s *Loader) Save(ctx context.Context) (err error)

func (*Loader) SaveAs added in v1.3.45 

func (s *Loader) SaveAs(ctx context.Context, outfile string, opts ...SaveAsOpt) (err error)

func (Loader) Set added in v1.3.45 

func (s Loader) Set(path string, data any) (node radix.Node[any], oldData any)

Set sets key('path') and value pair into storeS.

func (Loader) SetPrefix added in v1.3.45 

func (s Loader) SetPrefix(newPrefix ...string)

SetPrefix updates the prefix in current storeS.

func (Loader) SetTTL added in v1.3.45 

func (s Loader) SetTTL(path string, ttl time.Duration, cb radix.OnTTLRinging[any]) (state int)

SetTTL sets a ttl timeout for a branch or a leaf node.

At ttl arrived, the leaf node value will be cleared. For a branch node, it will be dropped.

Once you're using SetTTL, don't forget call Close(). For example: 

conf := newBasicStore()
defer conf.Close()

path := "app.verbose"
conf.SetTTL(path, 200*time.Millisecond, func(ctx context.Context, func(s *radix.TTL[any], nd radix.Node[any]) {
	t.Logf("%q cleared", path)
})

**[Pre-API]**

SetTTL is a prerelease API since v1.2.5, it's mutable in the several future releases recently.

The returned `state`: 0 assumed no error.

func (Loader) String added in v1.3.45 

func (s Loader) String() string

func (Loader) Update added in v1.3.45 

func (s Loader) Update(path string, cb func(node radix.Node[any], old any))

Update a node whether it existed or not.

func (Loader) WithPrefix added in v1.3.45 

func (s Loader) WithPrefix(prefix ...string) (newStore Store)

WithPrefix makes a lightweight copy from current storeS.

The new copy is enough light so that you can always use it with quite a low price.

WithPrefix appends an extra prefix at the end of the current prefix. For example, on a store with old prefix "app", WithPrefix("store") will return a new store 'NS' with prefix "app.server". And NS.MustGet("type") retrieve value at key path "app.server.type" now. 

conf := store.New()
s1 := conf.WithPrefix("app")
ns := s1.WithPrefix("server")
println(ns.MustGet("type"))     # print conf["app.server.type"]

It simplify biz-logic codes sometimes.

A [Delimiter] will be inserted at jointing prefix and key. Also at jointing old and new prefix.

func (Loader) WithPrefixReplaced added in v1.3.45 

func (s Loader) WithPrefixReplaced(newPrefix ...string) (newStore Store)

WithPrefixReplaced is similar with WithPrefix, but it replaces old prefix with new one instead of appending it. 

conf := store.New()
s1 := conf.WithPrefix("app")
ns := s1.WithPrefixReplaced("app.server")
println(ns.MustGet("type"))     # print conf["app.server.type"]

A [Delimiter] will be inserted at jointing prefix and key.

func (Loader) WithinLoading added in v1.3.45 

func (s Loader) WithinLoading(fn func())

WithinLoading is a helper to 'load' a 'fn'. The 'fn' will be run as is, and the internal flag 's.loading' will be set at beginning of fn executing, and reset at ending of fn.

type MinimalStoreT added in v1.0.0 

type MinimalStoreT[T any] interface {
	MustGet(path string) (data T)
	Get(path string) (data T, found bool)
	Set(path string, data T) (node radix.Node[T], oldData any)
	Has(path string) (found bool)
}

MinimalStoreT holds a minimal typed Store interface.

func NewStoreT 

func NewStoreT[T any]() MinimalStoreT[T]

NewStoreT allows reimplementing your own Store.

Any suggestions are welcome, please issue me.

type OnChangeHandler 

type OnChangeHandler func(path string, value, oldValue any, mergingMapOrLoading bool)

OnChangeHandler is called back when user setting key & value.

mergingMapOrLoading is true means that user is setting key recursively with a map (via [Store.Merge]), or a loader (re-)loading its source.

func (*OnChangeHandler) GobDecode added in v1.0.1 

func (*OnChangeHandler) GobDecode([]byte) error

func (OnChangeHandler) GobEncode added in v1.0.1 

func (OnChangeHandler) GobEncode() ([]byte, error)

type OnDeleteHandler 

type OnDeleteHandler func(path string, value any, mergingMapOrLoading bool) // when user deleting a key

func (*OnDeleteHandler) GobDecode added in v1.0.1 

func (*OnDeleteHandler) GobDecode([]byte) error

func (OnDeleteHandler) GobEncode added in v1.0.1 

func (OnDeleteHandler) GobEncode() ([]byte, error)

type OnNewHandler 

type OnNewHandler func(path string, value any, mergingMapOrLoading bool) // when user setting a new key

func (*OnNewHandler) GobDecode added in v1.0.1 

func (*OnNewHandler) GobDecode([]byte) error

func (OnNewHandler) GobEncode added in v1.0.1 

func (OnNewHandler) GobEncode() ([]byte, error)

type OnceProvider 

type OnceProvider interface {
	ReadBytes() (data []byte, err error) // return ErrNotImplemented as an identifier if it wants to be skipped
	Write(data []byte) (err error)       // return ErrNotImplemented as an identifier if it wants to be skipped

	ProviderSupports
}

OnceProvider is fit for a small-scale provider.

The kv data will be all loaded into memory.

type Op 

type Op uint32 // Op describes a set of file operations.

const (
	// OpCreate is a new pathname was created.
	OpCreate Op = 1 << iota

	// OpWrite the pathname was written to; this does *not* mean the write has finished,
	// and a write can be followed by more writes.
	OpWrite

	// OpRemove the path was removed; any watches on it will be removed. Some "remove"
	// operations may trigger a Rename if the file is actually moved (for
	// example "remove to trash" is often a rename).
	OpRemove

	// OpRename the path was renamed to something else; any watched on it will be
	// removed.
	OpRename

	// OpChmod file attributes were changed.
	//
	// It's generally not recommended to take action on this event, as it may
	// get triggered very frequently by some software. For example, Spotlight
	// indexing on macOS, anti-virus software, backup software, etc.
	OpChmod

	OpNone = 0
)

The operations fsnotify can trigger; see the documentation on [Watcher] for a full description, and check them with [Event.Has].

func (*Op) Marshal 

func (s *Op) Marshal() []byte

func (*Op) MarshalText 

func (s *Op) MarshalText() (text []byte, err error)

func (*Op) UnmarshalText 

func (s *Op) UnmarshalText(text []byte) error

type Opt 

type Opt func(s *storeS) // Opt(ions) for New Store

func WithDelimiter 

func WithDelimiter(delimiter rune) Opt

WithDelimiter sets the delimiter char.

A delimiter char is generally used for extracting the key-value pair via GetXXX, MustXXX, e.g., MustInt, MustStringSlice, ....

func WithFlattenSlice 

func WithFlattenSlice(b bool) Opt

WithFlattenSlice sets a bool flag to tell Store the slice value should be treated as node leaf. The index of the slice would be part of node path. For example, you're loading a slice []string{"A","B"} into node path "app.slice", the WithFlattenSlice(true) causes the following structure: 

app.slice.0 => "A"
app.slice.1 => "B"

Also, WithFlattenSlice makes the map values to be flattened into a tree.

func WithOnChangeHandlers 

func WithOnChangeHandlers(handlers ...OnChangeHandler) Opt

WithOnChangeHandlers allows user's handlers can be callback once a node changed.

func WithOnDeleteHandlers 

func WithOnDeleteHandlers(handlers ...OnDeleteHandler) Opt

WithOnDeleteHandlers allows user's handlers can be callback once a node removed.

func WithOnNewHandlers 

func WithOnNewHandlers(handlers ...OnNewHandler) Opt

WithOnNewHandlers allows user's handlers can be callback if a new node has been creating.

func WithPrefix 

func WithPrefix(prefix string) Opt

WithPrefix sets the associated prefix for the tree path.

func WithWatchEnable 

func WithWatchEnable(b bool) Opt

WithWatchEnable allows watching the external source if its provider supports Watchable ability.

type Peripheral 

type Peripheral interface {
	Close()
}

Peripheral is closeable.

type Provider 

type Provider interface {
	Read() (m map[string]ValPkg, err error) // return ErrNotImplemented as an identifier if it wants to be skipped

	ProviderSupports
}

The Provider gives a minimal set of interface to identify a data source.

The typical data sources are: consul, etcd, file, OS environ, ....

The interfaces are split to several groups: Streamable, Reader, Read, ReadBytes and Write.

A provider can implement just one of the above groups. At this time, the other interfaces should return ErrNotImplemented.

The Streamable API includes these: Keys, Count, Has, Next, Value and "MustValue". If you are implementing it, Keys, Value and Next are Must-Have. Because our kernel uses Keys to confirm the provider is Streamable, and invokes Next to iterate the key one by one. Once a key got, Value to get its associated value.

If the dataset is not very large scale, implementing Read is recommended to you. Read returns hierarchical data set as a nested `map[string]any` at once. Our kernel (loader) likes its simple logics.

Some providers may support Watchable API.

All providers should always accept Codec and Position and store them. When a provider monitored changes, storeS will request a reload action and these two Properties shall be usable.

Implementing OnceProvider.Write allows the provider to support Write-back mechanism.

type ProviderSupports 

type ProviderSupports interface {
	GetCodec() (codec Codec)   // return the bound codec decoder
	GetPosition() (pos string) // return a position pointed to a Trie-node path
	WithCodec(codec Codec)
	WithPosition(pos string)
}

ProviderSupports means which ability is supported by a Provider.

type Reader 

type Reader interface {
	Len() int // Len returns the number of bytes of the unread portion of the slice.
	// Size returns the original length of the underlying byte slice.
	// Size is the number of bytes available for reading via ReadAt.
	// The result is unaffected by any method calls except Reset.
	Size() int64
	// Read implements the io.Reader interface.
	Read(b []byte) (n int, err error)
	// ReadAt implements the io.ReaderAt interface.
	ReadAt(b []byte, off int64) (n int, err error)
	// ReadByte implements the io.ByteReader interface.
	ReadByte() (byte, error)
	// UnreadByte complements ReadByte in implementing the io.ByteScanner interface.
	UnreadByte() error
	// ReadRune implements the io.RuneReader interface.
	ReadRune() (ch rune, size int, err error)
	// UnreadRune complements ReadRune in implementing the io.RuneScanner interface.
	UnreadRune() error
	// Seek implements the io.Seeker interface.
	Seek(offset int64, whence int) (int64, error)
	// WriteTo implements the io.WriterTo interface.
	WriteTo(w io.Writer) (n int64, err error)
	// Reset resets the Reader to be reading from b.
	Reset(b []byte)
}

Reader reserved for future purpose.

type SaveAsOpt added in v1.3.45 

type SaveAsOpt func(*SaveAsOption)

func WithSaveAsCodec added in v1.3.45 

func WithSaveAsCodec(c Codec) SaveAsOpt

func WithSaveAsComment added in v1.3.45 

func WithSaveAsComment(includeComment bool) SaveAsOpt

func WithSaveAsProvider added in v1.3.45 

func WithSaveAsProvider(p Provider) SaveAsOpt

type SaveAsOption added in v1.3.45 

type SaveAsOption struct {
	// contains filtered or unexported fields
}

type Store 

type Store interface {
	// Close cleanup the internal resources.
	// See [basics.Peripheral] for more information.
	Close()

	// MustGet is the shortcut version of Get without
	// returning any error.
	MustGet(path string) (data any)

	// Get the value at path point 'path'.
	Get(path string) (data any, found bool)

	// Set sets key('path') and value pair into storeS.
	Set(path string, data any) (node radix.Node[any], oldData any)

	// Remove a key and its children
	Remove(path string) (removed bool)

	// Merge a map at path point 'pathAt'.
	Merge(pathAt string, data map[string]any) (err error)

	// Has tests if the given path exists
	Has(path string) (found bool)

	// Locate provides an advanced interface for locating a path.
	//
	// RETURNs:
	//   node: the matched node for retrieving node data later
	//   branch: true means a branch node matched (generally partialMatched is true)
	//   patialMatched: true means only a part of the node key was matched.
	//   found: any (fully or partially) found.
	//
	// When querying "app.logging.f" on a tree holding "app.logging.file",
	// Locate will return
	//   found = true, partialMatched = true,
	//   branch = false, and
	//   node is pointed to "app.logging.file".
	//
	// These high order apis (Has, Get(Xxx), Set(Xxx), Must(Xxx)) covers the
	// Locate's results and provides a dotted-key-path-based behaviors.
	// Which means, Has("app.logging.f") gets false and
	// Has("app.logging.file") is true.
	Locate(path string, kvpair radix.KVPair) (node radix.Node[any], branch, partialMatched, found bool)

	radix.TypedGetters[any] // getters

	// SetTTL sets a ttl timeout for a branch or a leaf node.
	//
	// At ttl arrived, the leaf node value will be cleared.
	// For a branch node, it will be dropped.
	//
	// Once you're using SetTTL, don't forget call Close().
	// For example:
	//
	//	conf := newTrieTree()
	//	defer conf.Close()
	//
	//	path := "app.verbose"
	//	conf.SetTTL(path, 200*time.Millisecond, func(ctx context.Context, s *TTL[any], nd *Node[any]) {
	//		t.Logf("%q cleared", path)
	//	})
	//
	// **[Pre-API]**
	//
	// SetTTL is a prerelease API since v1.2.5, it's mutable in the
	// several future releases recently.
	//
	// The returned `state`: 0 assumed no error.
	SetTTL(path string, ttl time.Duration, cb radix.OnTTLRinging[any]) (state int)
	SetTTLFast(node radix.Node[any], ttl time.Duration, cb radix.OnTTLRinging[any]) (state int)

	// GetEx gives a way to access node fields easily.
	GetEx(path string, cb func(node radix.Node[any], data any, branch bool, kvpair radix.KVPair))

	// SetEx is advanced version of Set.
	//
	// Using it to setup a new node at once. For example:
	//
	//	conf.SetEx("app.logging.auto-stop", true,
	//	  func(path string, oldData any, node radix.Node[any], trie radix.Trie[any]) {
	//	    conf.SetTTL(path, 30*time.Minute,
	//	      func(s *radix.TTL[any], node radix.Node[any]) {
	//	        conf.Remove(node.Key()) // erase the key with the node
	//	      })
	//	    // Or:
	//	    trie.SetTTLFast(node, 3*time.Second, nil)
	//	    // Or:
	//	    node.SetTTL(3*time.Second, trie, nil)
	//	  })
	SetEx(path string, data any, cb radix.OnSetEx[any]) (oldData any)

	// Update a node whether it existed or not.
	Update(path string, cb func(node radix.Node[any], old any))

	GetDesc(path string) (desc string, err error)       // get description field directly
	MustGetDesc(path string) (desc string)              // mustget description field directly
	GetTag(path string) (tag any, err error)            // get tag field directly
	MustGetTag(path string) (tag any)                   // mustget tag field directly
	GetComment(path string) (comment string, err error) // get comment field directly
	MustGetComment(path string) (comment string)        // mustget comment field directly

	SetComment(path, description, comment string) (ok bool) // set extra meta-info bound to a key
	SetTag(path string, tags any) (ok bool)                 // set extra notable data bound to a key

	// Dump prints internal data tree for debugging
	Dump() (text string)

	// Clone makes a clone copy for this store
	Clone() (newStore Store)

	// Dup is a native Clone tool.
	//
	// After Dup, a copy of the original store will be created,
	// but closers not.
	// Most of the closers are cleanup code fragments coming
	// from Load(WithProvider()), some of them needs to shut down the
	// remote connection such as what want to do by consul provider.
	//
	// At this scene, the parent store still holds the cleanup closers.
	Dup() (newStore Store)

	// Walk does iterate the whole Store.
	//
	// Walk("") walks from top-level root node.
	// Walk("app") walks from the parent of "app" node.
	// Walk("app.") walks from the "app." node.
	Walk(path string, cb func(path, fragment string, node radix.Node[any]))

	// WithPrefix makes a lightweight copy from current storeS.
	//
	// The new copy is enough light so that you can always use
	// it with quite a low price.
	//
	// WithPrefix appends an extra prefix at the end of the current
	// prefix.
	//
	// For example, on a store with old prefix "app",
	// WithPrefix("store") will return a new store 'NS' with prefix
	// "app.server". And NS.MustGet("type") retrieve value at key path
	// "app.server.type" now.
	//
	//	conf := store.New()
	//	s1 := conf.WithPrefix("app")
	//	ns := s1.WithPrefix("server")
	//	println(ns.MustGet("type"))     # print conf["app.server.type"]
	//
	// It simplify biz-logic codes sometimes.
	//
	// The arg 'prefix' can be an array, which will be joint
	// with the [Delimiter].
	WithPrefix(prefix ...string) (newStore Store)

	// WithPrefixReplaced is similar with WithPrefix, but it replaces old
	// prefix with new one instead of appending it.
	//
	//	conf := store.New()
	//	s1 := conf.WithPrefix("app")
	//	ns := s1.WithPrefixReplaced("app.server")
	//	println(ns.MustGet("type"))     # print conf["app.server.type"]
	//
	// The arg 'prefix' can be an array, which will be joint
	// with the [Delimiter].
	//
	// todo need a balance on returning *storeS or Store, for WithPrefixReplaced.
	WithPrefixReplaced(newPrefix ...string) (newStore Store)

	// SetPrefix updates the prefix in current storeS.
	//
	// The arg 'prefix' can be an array, which will be joint
	// with the [Delimiter].
	SetPrefix(newPrefix ...string)

	Prefix() string              // return current prefix string
	Delimiter() rune             // return current delimiter, generally it's dot ('.')
	SetDelimiter(delimiter rune) // setter. Change it at runtime doesn't update old delimiter inside tree nodes.

	// N makes a clone with trie.RecursiveNone mode.
	//
	// Get/Query and Locate are effected by this feature (by RecursiveUp).
	// Also, all MustXXX/GetXXX is also effected.
	N() (newStore Store)
	// R makes a clone with trie.RecursiveDown mode.
	//
	// This feature is no effect at this version.
	R() (newStore Store)
	// BR makes a clone with trie.RecursiveUp mode.
	//
	// Suppose you're at "path.to.someone.son" node and querying
	// "mid-name" key by MustString("mid-name"), and we knew
	// there're no matched key "mid-name" for node "son",
	// MustString will lookup all its parents along with the
	// owner chain till any node matched the key "mid-name".
	//
	// Get/Query and Locate are effected by this feature (by RecursiveUp).
	// Also, all MustXXX/GetXXX is also effected.
	BR() (newStore Store)

	RecursiveMode() radix.RecusiveMode

	// Load loads k-v pairs from external provider(s) with specified codec decoder(s).
	//
	// For those provider which run some service at background, such
	// as watching service, ctx gives a change to shut them down
	// gracefully. So you need pass a cancellable context into it.
	//
	// Or you know nothing or you don't care the terminating security,
	// simply passing context.TODO() is okay.
	Load(ctx context.Context, opts ...LoadOpt) (wr Writeable, err error)

	// WithinLoading executes a functor with loading state.
	//
	// About the Store's loading state:
	// If it's in loading, the k-v pairs will be put into store with a clean
	// modified flag.
	WithinLoading(fn func())
}

Store holds a standard Store interface.

func New 

func New(opts ...Opt) Store

New makes a new instance of storeS and returns it.

A storeS is a key-value container in memory with hierarchical tree data. A leaf or branch node can hold data. The dotted path

type StreamProvider 

type StreamProvider interface {
	Keys() (keys []string, err error)      // return ErrNotImplemented as an identifier if it wants to be skipped
	Count() int                            // count of keys and/or key-value pairs
	Has(key string) bool                   // test if the key exists
	Next() (key string, eol bool)          // return next usable key
	Value(key string) (value any, ok bool) // return the associated value
	MustValue(key string) (value any)      // return the value, or nil for a non-existence key

	ProviderSupports
}

StreamProvider is fit for a large-scale provider and load data on-demand.

type ValPkg 

type ValPkg struct {
	Value   any    // node's value
	Desc    string // description of a node
	Comment string // comment of a node
	Tag     any    // any extra data of a node
}

ValPkg is a value pack, It will be inserted into trie-tree as a data field. A node is commentable by Desc and Comment field.

type Watchable 

type Watchable interface {
	// Watch accepts user's func and callback it when the external
	// data source is changing, creating or deleting.
	//
	// The supported oprations are specified in Op.
	//
	// Tne user's func checks 'event' for which operation was occurring.
	// For more info, see also storeS.Load, storeS.applyExternalChanges,
	// and loader.startWatch.
	Watch(ctx context.Context, cb func(event any, err error)) error

	// Close provides a closer to cleanup the peripheral gracefully
	Close()
}

Watchable tips that a Provider can watch its external data source

type Writeable 

type Writeable interface {
	Save(ctx context.Context) (err error)
}

Writeable interface

Source Files

View all Source files

Directories

Path Synopsis
codecs
all module
gob module
hcl module
hjson module
json module
nestext module
toml module
yaml module
examples
simple module
internal
ctx
cvt
times
providers
consul module
env module
etcd module
file module
flags module
fs module
maps module
vault module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.