rdr

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 16, 2018 License: Apache-2.0 Imports: 17 Imported by: 0

README

RDR: redis data reveal

RDR(redis data reveal) is a tool to parse redis rdbfile. Comparing to redis-rdb-tools, RDR is implemented by golang, much faster (5GB rdbfile takes about 2mins on my PC).

Usage

NAME:
   rdr - a tool to parse redis rdbfile

USAGE:
   rdr [global options] command [command options] [arguments...]

VERSION:
   v0.0.1

COMMANDS:
     show     show statistical information of rdbfile by webpage
     keys     get all keys from rdbfile
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version

NAME:
   rdr show - show statistical information of rdbfile by webpage

USAGE:
   rdr show [command options] FILE1 [FILE2] [FILE3]...

OPTIONS:
   --port value, -p value  Port for rdr to listen (default: 8080)

NAME:
   rdr keys - get all keys from rdbfile

USAGE:
   rdr keys FILE1 [FILE2] [FILE3]...

Linux amd64 Download

OSX Download

After downloading meybe need add permisson to execute.

$ chmod a+x ./rdr

Exapmle

$ ./rdr show -p 8080 *.rdb

Note that the memory usage is approximate. show example

$ ./rdr keys example.rdb
portfolio:stock_follower_count:ZH314136
portfolio:stock_follower_count:ZH654106
portfolio:stock_follower:ZH617824
portfolio:stock_follower_count:ZH001019
portfolio:stock_follower_count:ZH346349
portfolio:stock_follower_count:ZH951803
portfolio:stock_follower:ZH924804
portfolio:stock_follower_count:INS104806

License

This project is under Apache v2 License. See the LICENSE file for the full license text.

Documentation

Overview

This code is mostly copied from Beego BeeMap

Index

Constants

This section is empty.

Variables

View Source 
var Counters = NewSafeMap()
View Source 
var (
	TplCommonData = map[string]interface{}{}
)

Functions

func Index 

func Index(w http.ResponseWriter, r *http.Request, p httprouter.Params)

func InitHTMLTmpl 

func InitHTMLTmpl()

func RdbReveal 

func RdbReveal(w http.ResponseWriter, r *http.Request, p httprouter.Params)

func ServeHTML 

func ServeHTML(w http.ResponseWriter, layout string, content string, data map[string]interface{})

ServeHTML generate and write html to client

Types

type Counter 

type Counter struct {
	// contains filtered or unexported fields
}

Counter for redis memory useage

func NewCounter 

func NewCounter() *Counter

NewCounter return a pointer of Counter

func (*Counter) Count 

func (c *Counter) Count(in <-chan *Entry)

Count by various dimensions

func (*Counter) GetLargestEntries 

func (c *Counter) GetLargestEntries(num int) []*Entry

GetLargestEntries from heap, num max is 500

func (*Counter) GetLargestKeyPrefixes 

func (c *Counter) GetLargestKeyPrefixes() []*PrefixEntry

GetLargestKeyPrefixes from heap

func (*Counter) GetLenLevelCount 

func (c *Counter) GetLenLevelCount() []*PrefixEntry

GetLenLevelCount from map

type Decoder 

type Decoder struct {
	Entries chan *Entry

	nopdecoder.NopDecoder
	// contains filtered or unexported fields
}

Decoder decode rdb file

func NewDecoder 

func NewDecoder() *Decoder

NewDecoder new a rdb decoder

func (*Decoder) EndHash 

func (d *Decoder) EndHash(key []byte)

EndHash is called when there are no more fields in a hash.

func (*Decoder) EndList 

func (d *Decoder) EndList(key []byte)

EndList is called when there are no more values in a list.

func (*Decoder) EndRDB 

func (d *Decoder) EndRDB()

EndRDB is called when parsing of the RDB file is complete.

func (*Decoder) EndSet 

func (d *Decoder) EndSet(key []byte)

EndSet is called when there are no more fields in a set. Same as EndHash

func (*Decoder) EndZSet 

func (d *Decoder) EndZSet(key []byte)

EndZSet is called when there are no more members in a sorted set.

func (*Decoder) Hset 

func (d *Decoder) Hset(key, field, value []byte)

Hset is called once for each field=value pair in a hash.

func (*Decoder) Rpush 

func (d *Decoder) Rpush(key, value []byte)

Rpush is called once for each value in a list.

func (*Decoder) Sadd 

func (d *Decoder) Sadd(key, member []byte)

Sadd is called once for each member of a set.

func (*Decoder) Set 

func (d *Decoder) Set(key, value []byte, expiry int64)

Set is called once for each string key.

func (*Decoder) StartHash 

func (d *Decoder) StartHash(key []byte, length, expiry int64)

StartHash is called at the beginning of a hash. Hset will be called exactly length times before EndHash.

func (*Decoder) StartList 

func (d *Decoder) StartList(key []byte, length, expiry int64)

StartList is called at the beginning of a list. Rpush will be called exactly length times before EndList. If length of the list is not known, then length is -1

func (*Decoder) StartSet 

func (d *Decoder) StartSet(key []byte, cardinality, expiry int64)

StartSet is called at the beginning of a set. Sadd will be called exactly cardinality times before EndSet.

func (*Decoder) StartZSet 

func (d *Decoder) StartZSet(key []byte, cardinality, expiry int64)

StartZSet is called at the beginning of a sorted set. Zadd will be called exactly cardinality times before EndZSet.

func (*Decoder) Zadd 

func (d *Decoder) Zadd(key []byte, score float64, member []byte)

Zadd is called once for each member of a sorted set.

type Entry 

type Entry struct {
	Key                string
	Bytes              uint64
	Type               string
	NumOfElem          uint64
	LenOfLargestElem   uint64
	FieldOfLargestElem string
}

Entry is info of a redis recored

type MemProfiler 

type MemProfiler struct{}

MemProfiler get memory use for all kinds of data stuct

func (*MemProfiler) ElemLen 

func (m *MemProfiler) ElemLen(element []byte) uint64

ElemLen get length of a element

func (*MemProfiler) HashtableEntryOverhead 

func (m *MemProfiler) HashtableEntryOverhead() uint64

HashtableEntryOverhead get memory use of hashtable entry See https://github.com/antirez/redis/blob/unstable/src/dict.h Each dictEntry has 2 pointers + int64

func (*MemProfiler) HashtableOverhead 

func (m *MemProfiler) HashtableOverhead(size uint64) uint64

HashtableOverhead get memory use of a hashtable See https://github.com/antirez/redis/blob/unstable/src/dict.h See the structures dict and dictht 2 * (3 unsigned longs + 1 pointer) + int + long + 2 pointers

Additionally, see **table in dictht The length of the table is the next power of 2 When the hashtable is rehashing, another instance of **table is created Due to the possibility of rehashing during loading, we calculate the worse case in which both tables are allocated, and so multiply the size of **table by 1.5

func (*MemProfiler) KeyExpiryOverhead 

func (m *MemProfiler) KeyExpiryOverhead(expiry int64) uint64

KeyExpiryOverhead get memory useage of a key expiry Key expiry is stored in a hashtable, so we have to pay for the cost of a hashtable entry The timestamp itself is stored as an int64, which is a 8 bytes

func (*MemProfiler) LinkedListEntryOverhead 

func (m *MemProfiler) LinkedListEntryOverhead() uint64

LinkedListEntryOverhead get memory use of a linked list entry See https://github.com/antirez/redis/blob/unstable/src/adlist.h A node has 3 pointers

func (*MemProfiler) LinkedlistOverhead 

func (m *MemProfiler) LinkedlistOverhead() uint64

LinkedlistOverhead get memory use of a linked list See https://github.com/antirez/redis/blob/unstable/src/adlist.h A list has 5 pointers + an unsigned long

func (*MemProfiler) RobjOverhead 

func (m *MemProfiler) RobjOverhead() uint64

RobjOverhead get memory useage of a robj

func (*MemProfiler) SizeofString 

func (m *MemProfiler) SizeofString(bytes []byte) uint64

SizeofString get memory use of a string https://github.com/antirez/redis/blob/unstable/src/sds.h

func (*MemProfiler) SkiplistEntryOverhead 

func (m *MemProfiler) SkiplistEntryOverhead() uint64

SkiplistEntryOverhead get memory use of a skiplist entry

func (*MemProfiler) SkiplistOverhead 

func (m *MemProfiler) SkiplistOverhead(size uint64) uint64

SkiplistOverhead get memory use of a skiplist

func (*MemProfiler) TopLevelObjOverhead 

func (m *MemProfiler) TopLevelObjOverhead() uint64

TopLevelObjOverhead get memory use of a top level object Each top level object is an entry in a dictionary, and so we have to include the overhead of a dictionary entry

type PrefixEntry 

type PrefixEntry struct {
	Bytes uint64
	Num   uint64
	// contains filtered or unexported fields
}

PrefixEntry record value by prefix

type SafeMap 

type SafeMap struct {
	// contains filtered or unexported fields
}

SafeMap is a map which concurrency safe

func NewSafeMap 

func NewSafeMap() *SafeMap

NewSafeMap return new safemap

func (*SafeMap) Check 

func (m *SafeMap) Check(k interface{}) bool

Check seturns true if k is exist in the map.

func (*SafeMap) Delete 

func (m *SafeMap) Delete(k interface{})

Delete the given key and value.

func (*SafeMap) Get 

func (m *SafeMap) Get(k interface{}) interface{}

Get from maps return the k's value

func (*SafeMap) Items 

func (m *SafeMap) Items() map[interface{}]interface{}

Items returns all items in safemap.

func (*SafeMap) Set 

func (m *SafeMap) Set(k interface{}, v interface{}) bool

Set the given key and value. Returns false if the key is already in the map and changes nothing.

func (*SafeMap) String 

func (m *SafeMap) String() (s string)

String transfer map into json

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.