Documentation
¶
Overview ¶
Package shelve provides a persistent, map-like object called Shelf. It lets you store and retrieve Go objects directly, with the serialization and storage handled automatically by the Shelf. Additionally, you can customize the underlying key-value storage and serialization codec to better suit your application's needs.
This package is inspired by the `shelve` module from the Python standard library and aims to provide a similar set of functionalities.
By default, a Shelf serializes data using the Gob format and stores it using `sdb` (for "shelve-db"), a simple key-value storage created for this project. This database should be good enough for a broad range of applications, but the modules in go-shelve/driver provide additional options for configuring the `Shelf` with other databases and Codecs.
Index ¶
- Constants
- type Codec
- type DB
- type Option
- type Shelf
- func (s *Shelf[K, V]) Close() error
- func (s *Shelf[K, V]) Delete(key K) error
- func (s *Shelf[K, V]) Get(key K) (value V, ok bool, err error)
- func (s *Shelf[K, V]) Has(key K) (bool, error)
- func (s *Shelf[K, V]) Items(start *K, n, step int, fn Yield[K, V]) error
- func (s *Shelf[K, V]) Keys(start *K, n, step int, fn Yield[K, V]) error
- func (s *Shelf[K, V]) Len() int64
- func (s *Shelf[K, V]) Put(key K, value V) error
- func (s *Shelf[K, V]) Sync() error
- func (s *Shelf[K, V]) Values(start *K, n, step int, fn Yield[K, V]) error
- type Yield
Examples ¶
Constants ¶
const ( // Asc and Desc can be used with the Shelf.Items method to make the // iteration order ascending or descending respectively. // // They are just syntactic sugar to make the iteration order more // explicit. Asc = 1 // Desc is the opposite of Asc. Desc = -1 // All can be used with the Shelf.Items method to iterate over all // items in the database. It is the same as the -1 value. All = -1 )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Codec ¶
type Codec interface {
// Encode returns the Codec encoding of v as a byte slice.
Encode(v any) ([]byte, error)
// Decode parses the encoded data and stores the result in the value
// pointed to by v. It is the inverse of Encode.
Decode(data []byte, v any) error
}
Codec is the interface for encoding and decoding data stored by Shelf.
The go-shelve module natively supports the following codecs:
- GobCodec: Returns a Codec for the gob format.
- JSONCodec: Returns a Codec for the JSON format.
- StringCodec: Returns a Codec for values that can be represented as strings.
Additional codecs are provided by the packages in driver/encoding.
func GobCodec ¶
func GobCodec() Codec
GobCodec Returns a Codec for the gob format, a self-describing serialization format native to Go.
Gob is a binary format and is more compact than text-based formats like JSON.
func StringCodec ¶
func StringCodec() Codec
StringCodec Returns a Codec for values that can be represented as strings.
type DB ¶
type DB interface {
// Close synchronizes and closes the database.
Close() error
// Len returns the number of items in the database. It returns the number
// of items as an int64. If an error occurs, it returns -1.
Len() int64
// Sync synchronizes the database to persistent storage.
Sync() error
// Has reports whether a key exists in the database.
Has(key []byte) (bool, error)
// Get retrieves the value associated with a key from the database. If the
// key is not found, it returns nil.
Get(key []byte) ([]byte, error)
// Put adds a key-value pair to the database. If the key already exists, it
// overwrites the existing value.
Put(key, value []byte) error
// Delete removes a key-value pair from the database.
Delete(key []byte) error
// Items iterates over key-value pairs in the database, calling fn(k, v)
// for each pair in the sequence. The iteration stops early if the function
// fn returns false.
//
// The start parameter specifies the key from which the iteration should
// start. If the start parameter is nil, the iteration will begin from the
// first key in the database.
//
// The order parameter specifies the order in which the items should be
// yielded. A negative value for order will cause the iteration to occur in
// reverse order.
//
// When iterating over key-value pairs in a database, the order of
// iteration may not be sorted. Some database implementations may ignore
// the start parameter or not support iteration in reverse order.
Items(
start []byte,
order int,
fn func(key, value []byte) (bool, error),
) error
}
DB is an interface that defines the methods for a database that can be used with Shelf. It takes the encoded binary representation of keys and values.
type Option ¶
type Option func(any)
Option is passed to the Open function to create a customized Shelf.
func WithCodec ¶
WithCodec specifies the Codec to use. By default, a codec for the Gob format from the standard library (encoding/gob) is used.
Additional Codecs can be found in the packages in driver/encoding.
func WithDatabase ¶
WithDatabase specifies the underlying database to use. By default, the sdb.DB ("shelve-db") key-value storage is used.
The packages in driver/db packages provide support for others databases in the Go ecosystem, like Bolt and Badger.
func WithKeyCodec ¶
WithKeyCodec specifies the Codec to use with keys. By default, if the key is a string or an integer type (both signed and unsigned), the StringCodec is used. Otherwise, keys are encoded as Gob (encoding/gob).
Additional Codecs can be found in the packages in driver/encoding.
type Shelf ¶
type Shelf[K comparable, V any] struct { // contains filtered or unexported fields }
A Shelf is a persistent, map-like object. It is used together with an underlying key-value storage to store Go objects directly.
Stored values can be of arbitrary types, but the keys must be comparable.
The values are encoded as Gob, and keys, if they are strings or integers, are encoded as strings. Otherwise, they will also be encoded as Gob.
For storage, the underlying database is an instance of the sdb.DB ("shelve-db") key-value store.
The underlying storage and codec Shelf uses can be configured with the Option functions.
func Open ¶
Open creates a new Shelf.
The path parameter specifies the filesystem path to the database files. It can be a directory or a regular file, depending on the underlying database implementation. With the default database sdb.DB, it will point to a directory.
Example ¶
package main
import (
"fmt"
"log"
"os"
"path/filepath"
"github.com/lucmq/go-shelve/shelve"
)
func main() {
// Note: In a real application be sure to replace the
// os.TempDir() with a non temporary directory and
// provide better error treatment.
path := filepath.Join(os.TempDir(), "go-shelve")
// Open the shelf with default options
shelf, err := shelve.Open[string, string](path)
if err != nil {
log.Printf("open: %s", err)
return
}
defer shelf.Close()
// Use the database
shelf.Put("language", "Go")
shelf.Put("module", "go-shelve")
// Note: Saved values will be available between restarts
value, ok, _ := shelf.Get("language")
fmt.Println(value, ok)
}
Output: Go true
Example (WithDatabase) ¶
package main
import (
"fmt"
"log"
"os"
"path/filepath"
"github.com/lucmq/go-shelve/sdb"
"github.com/lucmq/go-shelve/shelve"
)
func main() {
path := filepath.Join(os.TempDir(), "go-shelve")
// Configure some options for the default go-shelf database. This
// can also be used to open go-shelf with another database (like
// boltdb, badger, etc). Check the driver packages for more
// options.
db, _ := sdb.Open(
path,
sdb.WithCacheSize(1024*1024),
sdb.WithSynchronousWrites(true),
)
shelf, err := shelve.Open[string, string](
path,
shelve.WithDatabase(db),
shelve.WithCodec(shelve.JSONCodec()),
shelve.WithKeyCodec(shelve.StringCodec()),
)
if err != nil {
log.Printf("open: %s", err)
return
}
defer shelf.Close()
// Use the database
shelf.Put("example", "go-shelve")
value, ok, _ := shelf.Get("example")
fmt.Println(value, ok)
}
Output: go-shelve true
func (*Shelf[K, V]) Get ¶
Get retrieves the value associated with a key from the Shelf. If the key is not found, it returns nil.
func (*Shelf[K, V]) Items ¶
Items iterates over key-value pairs in the Shelf, calling fn(k, v) for each pair in the sequence. The iteration stops early if the function fn returns false.
The start parameter specifies the key from which the iteration should start. If the start parameter is nil, the iteration will begin from the first key in the Shelf.
The n parameter specifies the maximum number of items to iterate over. If n is -1 or less, all items will be iterated.
The step parameter specifies the number of items to skip between each iteration. A negative value for step will cause the iteration to occur in reverse order.
When iterating over key-value pairs in a Shelf, the order of iteration may not be sorted. Some database implementations may ignore the start parameter or not support iteration in reverse order.
The default database used with Shelf (sdb.DB) does not yield items in any particular order and ignores the start parameter.
Example ¶
package main
import (
"fmt"
"log"
"os"
"path/filepath"
"github.com/lucmq/go-shelve/shelve"
)
func main() {
path := filepath.Join(os.TempDir(), "go-shelve")
Clean(path) // Only for the example
shelf, err := shelve.Open[string, string](path)
if err != nil {
log.Printf("open: %s", err)
return
}
defer shelf.Close()
// Put some items
shelf.Put("a", "1")
shelf.Put("b", "2")
shelf.Put("c", "3")
fn := func(key, value string) (bool, error) {
fmt.Println(key, value)
return true, nil
}
start := "a"
err = shelf.Items(&start, shelve.All, shelve.Asc, fn)
if err != nil {
log.Printf("items: %s", err)
return
}
// Note: The call above is the same as shelve.Items(&start, -1, 1, fn)
// Note: The output will be sorted by key if the underlying
// database driver supports ordered iteration.
}
// Clean is a function used with example code to start with a clean storage.
func Clean(path string) error {
return os.RemoveAll(path)
}
Output: a 1 b 2 c 3
func (*Shelf[K, V]) Keys ¶
Keys iterates over all keys in the Shelf and calls the user-provided function fn for each key. The details of the iteration are the same as for Shelf.Items.
The value parameter for fn will always be the zero value for the type V.
func (*Shelf[K, V]) Len ¶
Len returns the number of items in the Shelf. It returns the number of items as an int64. If an error occurs, it returns -1.
func (*Shelf[K, V]) Put ¶
Put adds a key-value pair to the Shelf. If the key already exists, it overwrites the existing value.
Example ¶
package main
import (
"fmt"
"log"
"os"
"path/filepath"
"github.com/lucmq/go-shelve/shelve"
)
func main() {
path := filepath.Join(os.TempDir(), "go-shelve")
Clean(path) // Only for the example
type Example struct {
I int
M map[string]string
}
shelf, err := shelve.Open[string, Example](path)
if err != nil {
log.Printf("open: %s", err)
return
}
defer shelf.Close()
// Put an item
shelf.Put("a", Example{I: 42, M: map[string]string{"a": "1"}})
// Get the item
value, ok, _ := shelf.Get("a")
fmt.Println(value, ok)
// Mutate the item. Remember to Save the mutated item to
// make the change persistent.
value.M["b"] = "2"
shelf.Put("a", value)
newValue, ok, _ := shelf.Get("a")
fmt.Println(newValue, ok)
}
// Clean is a function used with example code to start with a clean storage.
func Clean(path string) error {
return os.RemoveAll(path)
}
Output: {42 map[a:1]} true {42 map[a:1 b:2]} true
func (*Shelf[K, V]) Values ¶
Values iterates over all values in the Shelf and calls the user-provided function fn for each value. The details of the iteration are the same as for Shelf.Items.
The key parameter for fn will always be the zero value for the type K.
Source Files