Documentation
¶
Overview ¶
Package rdb implements a Redis RDB File parser.
Basic Example ¶
The basic use case is very simple, just implements Filter interface and pass it to Parse:
package main
import (
"github.com/matthewjhe/rdb"
)
func (f filter) Key(k rdb.Key) bool { return false }
func (f filter) Type(t rdb.Type) bool { return false }
func (f filter) Database(db rdb.DB) bool { return false }
func (f filter) Set(v *rdb.Set) { }
func (f filter) List(v *rdb.List) { }
func (f filter) Hash(v *rdb.Hash) { }
func (f filter) String(v *rdb.String) { }
func (f filter) SortedSet(v *rdb.SortedSet) { }
func main() {
const file = "/tmp/dump.rdb"
reader, err := rdb.NewBufferReader(file, 0)
if err != nil {
panic(err)
}
if err := rdb.Parse(reader, rdb.WithFilter(filter{})); err != nil {
panic(err)
}
}
Skipping ¶
NOTE: RDB file is read sequentially, when we say skips a database, we also need to parse this database's every single key, we just simply skips actions like decompress, unzip ziplist, etc. It is impossible to skips a key without reading its metadata.
Global skip strategy applies to the whole parsing lifetime, it can be overwritten by Filter's Database method, it will be restored when parsing a database completely.
Global skip strategy is set by a ParseOption when rdb.Parse is called:
// reads memory report only
strategy := rdb.WithStrategy(rdb.SkipExpiry | rdb.SkipMeta | rdb.SkipValue)
err := rdb.Parse(reader, strategy, rdb.WithFilter(filter{}))
Database skip strategy applies to a database lifetime, it is set by Filter's Database method, and it can be overwritten by Filter's Key or Type method. Database skip strategy is set to global skip strategy by default.
func (f filter) Database(db rdb.DB) bool {
// skips this database
db.Skip(rdb.SkipAll)
// skips database's metadata and key's expiry
// db.Skip(rdb.SkipMeta | rdb.SkipExpiry)
return false
}
Type and key skip strategy overwrite database skip strategy. It only applies the key and value which will read next.
func (f filter) Type(t rdb.Type) bool {
// skips this type
t.Skip(rdb.SkipAll)
return false
}
func (f filter) Key(key rdb.Key) bool {
// skips this key
// key.Skip(rdb.SkipAll)
// skips value
key.Skip(rdb.SkipValue)
return false
}
Syncing ¶
By default, rdb uses multiple goroutines to parse keys which would cause keys disorder. If the order of keys is important, use EnableSync ParseOption.
rdb.Parse(reader, rdb.WithFilter(filter{}), rdb.EnableSync())
Aborting ¶
Sometimes we want to abort the parsing process, this can be done by return true when one of the following methods is called: Note that the abort action is taking effect immediately when the function returns.
func (f filter) Key(k rdb.Key) bool {
return true
}
func (f filter) Type(t rdb.Type) bool {
return true
}
func (f filter) Database(db rdb.DB) bool {
return true
}
Index ¶
- Constants
- Variables
- func Encoding2String(encoding byte) string
- func Encoding2Type(encoding byte) string
- func Parse(r Reader, opts ...ParseOption) error
- type BufferReader
- type DB
- type Filter
- type Hash
- type Key
- type List
- type MemReader
- type ParseOption
- type Parser
- type Reader
- type Set
- type SortedSet
- type String
- type Type
Constants ¶
const ( SkipMeta = 1 << iota SkipExpiry SkipValue SkipAll )
Parse strategies
const ( EncodingString = 0 EncodingList = 1 EncodingSet = 2 EncodingSortedSet = 3 EncodingHash = 4 EncodingSortedSet2 = 5 EncodingZipmap = 9 EncodingZiplist = 10 EncodingIntset = 11 EncodingSortedSetZip = 12 EncodingHashZip = 13 EncodingQuicklist = 14 )
Redis value encodings.
const ( TypeSet = "set" TypeList = "list" TypeHash = "hash" TypeString = "string" TypeSortedSet = "sortedset" )
Redis types.
Variables ¶
var ( ErrInvalidRDB = stderr.New("Invalid RDB file") ErrUnsupportedRDB = stderr.New("Unsupported RDB version") ErrInvalidZipmapEntry = stderr.New("Invalid zipmap entry") ErrInvalidLengthEncoding = stderr.New("Invalid length encoding") ErrInvalidCompressedData = stderr.New("Invalid compressed data") )
Parse errors
Functions ¶
func Encoding2String ¶
Encoding2String returns the string format of encoding.
func Encoding2Type ¶
Encoding2Type returns the actual redis type of encoding.
Types ¶
type BufferReader ¶
BufferReader is a Reader that reads from a *bufio.Reader.
func (*BufferReader) Discard ¶
func (r *BufferReader) Discard(n int)
Discard skips the next n bytes.
type DB ¶
type DB struct {
Num int
// contains filtered or unexported fields
}
DB represents a redis database.
type Filter ¶
type Filter interface {
Key(key Key) bool
Type(typ Type) bool
Database(db DB) bool
Set(s *Set)
List(l *List)
Hash(h *Hash)
String(s *String)
SortedSet(s *SortedSet)
}
A Filter controls the parser's behaviors.
NOTE: Filter is not safe for concurrent use.
type Key ¶
type Key struct {
Encoding byte
DB int
Expiry int
Key string
// contains filtered or unexported fields
}
Key represents a redis key.
type MemReader ¶
type MemReader struct {
// contains filtered or unexported fields
}
MemReader is a Reader that reads rdb data from memory.
type ParseOption ¶
type ParseOption func(*Parser)
ParseOption configures the behaviors when parsing a rdb file.
func EnableSync ¶
func EnableSync() ParseOption
EnableSync returns a ParseOption which disable async filtering.
func WithFilter ¶
func WithFilter(filter Filter) ParseOption
WithFilter returns a ParseOption which sets a parse filter.
func WithStrategy ¶
func WithStrategy(strategy int) ParseOption
WithStrategy returns a ParseOption which sets default parse strategy.
type Reader ¶
type Reader interface {
Discard(n int)
ReadByte() (byte, error)
ReadBytes(n int) ([]byte, error)
// contains filtered or unexported methods
}
Reader is the interface that wraps the operations against rdb data.
Discard skips the next n bytes. ReadByte reads and returns a single byte. ReadBytes reads and returns exactly n bytes.
func NewBufferReader ¶
NewBufferReader returns a new BufferReader reading from file. It's buffer has at least the specified size. If size == 0, use default size.
func NewMemReader ¶
NewMemReader memory-maps the named file and returns a MemReader that reads from it.
type Set ¶
type Set struct {
Key Key
Values map[interface{}]struct{}
// contains filtered or unexported fields
}
Set represents redis set.
type SortedSet ¶
type SortedSet struct {
Key Key
Values map[string]float64
// contains filtered or unexported fields
}
SortedSet represents redis sortedset.