ratel

package
v0.0.0-...-d6038a4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 21, 2025 License: CC0-1.0 Imports: 50 Imported by: 0

Documentation

Overview

Package ratel is a badger DB based event store.

Index

Constants

View Source 
const DefaultMaxLimit = 2048

DefaultMaxLimit is set to a size that means the usual biggest batch of events sent to a client usually is at most about 256kb or so.

View Source 
const Version = 1

Variables

This section is empty.

Functions

func Create_a_Tag 

func Create_a_Tag(tagKey, tagValue string, CA *createdat.T,
	ser *serial.T) (prf index.P, elems []keys.Element, err error)

Create_a_Tag generates tag indexes from a tag key, tag value, created_at timestamp and the event serial.

func GetCounterKey 

func GetCounterKey(ser *serial.T) (key []byte)

GetCounterKey returns the proper counter key for a given event Id. This needs a separate function because of what it does, but is generated in the general GetIndexKeysForEvent function.

func GetIndexKeysForEvent 

func GetIndexKeysForEvent(ev *event.T, ser *serial.T) (keyz [][]byte)

GetIndexKeysForEvent generates all the index keys required to filter for events. evtSerial should be the output of Serial() which gets a unique, monotonic counter value for each new event.

func GetTagKeyPrefix 

func GetTagKeyPrefix(tagValue string) (key []byte, err error)

GetTagKeyPrefix returns tag index prefixes based on the initial field of a tag.

There is 3 types of index tag keys:

- TagAddr: [ 8 ][ 2b Kind ][ 8b Pubkey ][ address/URL ][ 8b Serial ]

- Tag32: [ 7 ][ 8b Pubkey ][ 8b Serial ]

- Tag: [ 6 ][ address/URL ][ 8b Serial ]

This function produces the initial bytes without the index.

func NewLogger 

func NewLogger(logLevel int32, label string) (l *logger)

NewLogger creates a new badger logger.

func PrepareQueries 

func PrepareQueries(f *filter.T) (
	qs []query,
	ext *filter.T,
	since uint64,
	err error,
)

PrepareQueries analyses a filter and generates a set of query specs that produce key prefixes to search for in the badger key indexes.

Types

type BackendParams 

type BackendParams struct {
	Ctx                                context.T
	WG                                 *sync.WaitGroup
	HasL2, UseCompact                  bool
	BlockCacheSize, LogLevel, MaxLimit int
	Compression                        string // none,snappy,zstd
	Extra                              []int
}

BackendParams is the configurations used in creating a new ratel.T.

type Results 

type Results struct {
	Ev  *event.T
	TS  *timestamp.T
	Ser *serial.T
}

type T 

type T struct {
	Ctx context.T
	WG  *sync.WaitGroup

	HasL2          bool
	BlockCacheSize int
	InitLogLevel   int32
	Logger         *logger
	// DB is the badger db
	*badger.DB

	// Threads is how many CPU threads we dedicate to concurrent actions, flatten and GC mark
	Threads int
	// MaxLimit is a default limit that applies to a query without a limit, to avoid sending out
	// too many events to a client from a malformed or excessively broad filter.
	MaxLimit int
	// Flatten should be set to true to trigger a flatten at close... this is mainly triggered
	// by running an import
	Flatten bool
	// UseCompact uses a compact encoding based on the canonical format (generate hash of it to
	// get Id field with the signature in raw binary after.
	UseCompact bool
	// Compression sets the compression to use, none/snappy/zstd. If zstd compression is enabled
	// there is less benefit to UseCompact, and instead of having to re-marshal the event it can
	// be directly delivered from the form returned from the database.
	Compression string
	// contains filtered or unexported fields
}

T is a badger event store database with layer2 and garbage collection.

func GetBackend deprecated 

func GetBackend(Ctx context.T, WG *sync.WaitGroup, hasL2, useCompact bool,
	blockCacheSize, logLevel, maxLimit int, compression string) (b *T)

GetBackend returns a reasonably configured badger.Backend.

The variadic params correspond to DBSizeLimit, DBLowWater, DBHighWater and GCFrequency as an integer multiplier of number of seconds.

Note that the cancel function for the context needs to be managed by the caller.

Deprecated: use New instead.

func New 

func New(p BackendParams) *T

New configures a a new ratel.T event store.

func (*T) Close 

func (r *T) Close() (err error)

Close the database. If the Flatten flag was set, then trigger the flattening of tables before shutting down.

func (*T) DeleteEvent 

func (r *T) DeleteEvent(c context.T, eid *eventid.T, noTombstone ...bool) (err error)

DeleteEvent deletes an event if it exists, and writes a tombstone for the event unless requested not to, so that the event can't be saved again.

func (*T) Export 

func (r *T) Export(c context.T, w io.Writer, pubkeys ...[]byte)

Export the complete database of stored events to an io.Writer in line structured minified JSON.

func (*T) FetchIds 

func (r *T) FetchIds(c context.T, evIds *tag.T, out io.Writer) (err error)

FetchIds retrieves events based on a list of event Ids that have been provided.

func (*T) GetConfiguration 

func (r *T) GetConfiguration() (c *config.C, err error)

GetConfiguration returns the current store.C stored in the database.

func (*T) Import 

func (r *T) Import(rr io.Reader)

Import a collection of events in line structured minified JSON format (JSONL).

func (*T) Init 

func (r *T) Init(path string) (err error)

Init sets up the database with the loaded configuration.

func (*T) Marshal 

func (r *T) Marshal(ev *event.T, dst []byte) (b []byte)

Marshal an event using compact encoding if configured.

func (*T) Nuke 

func (r *T) Nuke() (err error)

Nuke wipes the database.

func (*T) Path 

func (r *T) Path() string

Path returns the path where the database files are stored.

func (*T) QueryEvents 

func (r *T) QueryEvents(c context.T, f *filter.T) (evs event.Ts, err error)

func (*T) QueryForIds 

func (r *T) QueryForIds(c context.T, f *filter.T) (founds []store.IdTsPk, err error)

func (*T) Rescan 

func (r *T) Rescan() (err error)

Rescan regenerates all indexes of events to add new indexes in a new version.

func (*T) SaveEvent 

func (r *T) SaveEvent(c context.T, ev *event.T) (err error)

func (*T) Serial 

func (r *T) Serial() (ser uint64, err error)

Serial returns the next monotonic conflict free unique serial on the database.

func (*T) SerialBytes 

func (r *T) SerialBytes() (ser []byte, err error)

SerialBytes returns a new serial value, used to store an event record with a conflict-free unique code (it is a monotonic, atomic, ascending counter).

func (*T) SerialKey 

func (r *T) SerialKey() (idx []byte, ser *serial.T)

SerialKey returns a key used for storing events, and the raw serial counter bytes to copy into index keys.

func (*T) SetConfiguration 

func (r *T) SetConfiguration(c *config.C) (err error)

SetConfiguration stores the store.C value to a provided setting.

func (*T) SetLogLevel 

func (r *T) SetLogLevel(level string)

func (*T) Sync 

func (r *T) Sync() (err error)

func (*T) Unmarshal 

func (r *T) Unmarshal(ev *event.T, evb []byte) (rem []byte, err error)

Unmarshal an event from bytes, using compact encoding if configured.

Source Files

View all Source files

Directories

Path Synopsis
Package del is a simple sorted list for database keys, primarily used to collect lists of events that need to be deleted either by expiration or for the garbage collector.
 Package del is a simple sorted list for database keys, primarily used to collect lists of events that need to be deleted either by expiration or for the garbage collector.
Package keys is a composable framework for constructing badger keys from fields of events.
 Package keys is a composable framework for constructing badger keys from fields of events.
arb
Package arb implements arbitrary length byte keys.Element.
 Package arb implements arbitrary length byte keys.Element.
count
Package count contains a series of data types for managing lists of indexes for garbage collection.
 Package count contains a series of data types for managing lists of indexes for garbage collection.
createdat
Package createdat implements a badger key index keys.Element for timestamps.
 Package createdat implements a badger key index keys.Element for timestamps.
fullid
Package fullid implements a keys.Element for a complete 32 byte event Ids.
 Package fullid implements a keys.Element for a complete 32 byte event Ids.
fullpubkey
Package fullpubkey implements a keys.Element for a complete 32 byte nostr pubkeys.
 Package fullpubkey implements a keys.Element for a complete 32 byte nostr pubkeys.
id
Package id implements a keys.Element for a truncated event Ids containing the first 8 bytes of an eventid.T.
 Package id implements a keys.Element for a truncated event Ids containing the first 8 bytes of an eventid.T.
index
Package index implements the single byte prefix of the database keys.
 Package index implements the single byte prefix of the database keys.
kinder
Package kinder implements a keys.Element for the 16 bit nostr 'kind' value for use in indexes.
 Package kinder implements a keys.Element for the 16 bit nostr 'kind' value for use in indexes.
pubkey
Package pubkey implements an 8 byte truncated public key implementation of a keys.Element.
 Package pubkey implements an 8 byte truncated public key implementation of a keys.Element.
serial
Package serial implements a keys.Element for encoding a serial (monotonic 64 bit counter) for stored events, used to link an index to the main data table.
 Package serial implements a keys.Element for encoding a serial (monotonic 64 bit counter) for stored events, used to link an index to the main data table.
tombstone
Package tombstone is a 16 byte truncated event Id for keys.Element used to mark an event as being deleted so it isn't saved again.
 Package tombstone is a 16 byte truncated event Id for keys.Element used to mark an event as being deleted so it isn't saved again.
Package prefixes provides a list of the index.P types that designate tables in the ratel event store, as well as enabling a simple syntax to assemble and decompose an index key into its keys.Element s.
 Package prefixes provides a list of the index.P types that designate tables in the ratel event store, as well as enabling a simple syntax to assemble and decompose an index key into its keys.Element s.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.