README
¶
Simpleblob
Simpleblob is a Go module that simplifies the storage of arbitrary data by key from Go code. It ships with the following backends:
s3: S3 bucket storagefs: File storage (one file per blob)memory: Memory storage (for tests)
The interface implemented by the backends is:
type Interface interface {
List(ctx context.Context, prefix string) (BlobList, error)
Load(ctx context.Context, name string) ([]byte, error)
Store(ctx context.Context, name string, data []byte) error
Delete(ctx context.Context, name string) error
}
To instantiate a backend, _-import all the backends that you want to register, and call:
func GetBackend(ctx context.Context, typeName string, options map[string]any, params ...Param) (Interface, error)
An example can be found in example_test.go.
Every backend accepts a map[string]any with options and performs its own validation on the options. If you use a YAML, TOML and JSON, you could structure it like this:
type Storage struct {
Type string `yaml:"type"`
Options map[string]any `yaml:"options"` // backend specific
}
Limitations
The interface currently does not support streaming of large blobs. In the future we may provide this by implementing fs.FS in the backend for reading, and a similar interface for writing new blobs.
API Stability
We support the last two stable Go versions, currently 1.17 and 1.18.
From a API consumer point of view, we do not plan any backward incompatible changes before a v1.0.
For storage backends, any future extensions most likely be added with optional interface, similar to the fs.FS design. Utility functions that return a compatible implementation will be used for backends that do not implement the interface, when possible.
Documentation
¶
Overview ¶
Example ¶
package main
import (
"context"
"fmt"
"github.com/PowerDNS/simpleblob"
"github.com/go-logr/logr"
// Register the memory backend plugin
_ "github.com/PowerDNS/simpleblob/backends/memory"
)
func main() {
// Do not forget the:
// import _ "github.com/PowerDNS/simpleblob/backends/memory"
ctx := context.Background()
storage, err := simpleblob.GetBackend(
ctx,
"memory",
map[string]interface{}{
// add key-value options here
"foo": "example",
},
simpleblob.WithLogger(logr.Discard()), // replace with a real logger
)
check(err)
err = storage.Store(ctx, "example.txt", []byte("hello"))
check(err)
data, err := storage.Load(ctx, "example.txt")
check(err)
fmt.Println("data:", string(data))
list, err := storage.List(ctx, "")
check(err)
for _, entry := range list {
fmt.Println("list:", entry.Name, entry.Size)
}
}
func check(err error) {
if err != nil {
panic(err)
}
}
Output: data: hello list: example.txt 5
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RegisterBackend ¶
RegisterBackend registers a new backend.
Types ¶
type BlobList ¶
type BlobList []Blob
BlobList is a slice of Blob structs
func (BlobList) WithPrefix ¶
WithPrefix filters the BlobList to returns only Blob structs where the name starts with the given prefix.
type InitFunc ¶
type InitFunc func(ctx context.Context, p InitParams) (Interface, error)
InitFunc is the type for the backend constructor function used to register backend.
type InitParams ¶
type InitParams struct {
OptionMap OptionMap // map of key-value options for this backend
Logger logr.Logger
}
InitParams contains the parameters for the InitFunc. This allows us to pass extra values in the future, without breaking existing backends that do not expect these.
func (InitParams) OptionsThroughYAML ¶
func (ip InitParams) OptionsThroughYAML(dest interface{}) error
OptionsThroughYAML performs a YAML roundtrip for the OptionMap to load them into a struct with yaml tags. dest: pointer to destination struct
type Interface ¶
type Interface interface {
// List retrieves a BlobList with given prefix.
List(ctx context.Context, prefix string) (BlobList, error)
// Load brings a whole value, chosen by name, into memory.
Load(ctx context.Context, name string) ([]byte, error)
// Store sends value to storage for a given name.
Store(ctx context.Context, name string, data []byte) error
// Delete entry, identified by name, from storage. No error is returned if it does not exist.
Delete(ctx context.Context, name string) error
}
Interface defines the interface storage plugins need to implement. This Interface MUST NOT be extended or changed without bumping the major version number, because it would break any backends in external repos. If you want to provide additional features, you can define additional optional interfaces that a backend can implement.
func GetBackend ¶
func GetBackend(ctx context.Context, typeName string, options OptionMap, params ...Param) (Interface, error)
GetBackend creates a new backend instance of given typeName. This type must have been previously registered with RegisterBackend.
The options map contains backend dependant key-value options. Some backends take no options, others require some specific options.
Additional parameters can be passed with extra arguments, like WithLogger.
The lifetime of the context passed in must span the lifetime of the whole backend instance, not just the init time, so do not set any timeout on it!
type OptionMap ¶
type OptionMap map[string]interface{}
OptionMap is the type for options that we pass internally to backends
type Param ¶ added in v0.2.0
type Param func(ip *InitParams)
Param is the type of extra init parameters. It is returned by calling functional params like WithLogger.
func WithLogger ¶ added in v0.2.0
WithLogger is a GetBackend parameter that sets the logr.Logger to use in the backends.
Source Files
Directories