volume_badger

package
v0.51.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 9, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

README

Some useful notes from the Badger repository README:

Memory Usage

Badger's memory usage can be managed by tweaking several options available in the Options struct that is passed in when opening the database using DB.Open.

  • Options.ValueLogLoadingMode can be set to options.FileIO (instead of the default options.MemoryMap) to avoid memory-mapping log files. This can be useful in environments with low RAM.
  • Number of memtables (Options.NumMemtables)
    • If you modify Options.NumMemtables, also adjust Options.NumLevelZeroTables and Options.NumLevelZeroTablesStall accordingly.
  • Number of concurrent compactions (Options.NumCompactors)
  • Mode in which LSM tree is loaded (Options.TableLoadingMode)
  • Size of table (Options.MaxTableSize)
  • Size of value log file (Options.ValueLogFileSize)

If you want to decrease the memory usage of Badger instance, tweak these options (ideally one at a time) until you achieve the desired memory usage.

Documentation

Index

Constants

View Source 
const ControllerID = "hydra/volume/badger"

ControllerID identifies the Badger volume controller.

Variables

View Source 
var ConfigID = ControllerID

ConfigID is the id attached to the config objects.

View Source 
var Version = semver.MustParse("0.0.1")

Version is the version of the badger implementation.

Functions

This section is empty.

Types

type Badger 

type Badger = kvtx.Volume

Badger implements a BadgerDB backed volume.

func NewBadger 

func NewBadger(
	ctx context.Context,
	le *logrus.Entry,
	conf *Config,
) (*Badger, error)

NewBadger builds a new Badger volume, opening the database.

type Config 

type Config struct {

	// Dir is the directory to store the data in.
	// Should exist and be writable.
	Dir string `protobuf:"bytes,1,opt,name=dir,proto3" json:"dir,omitempty"`
	// ValueDir is the directory to store the value log in.
	// Can be the same as dir.
	// Should exist and be writable.
	// If empty, defaults to Dir.
	ValueDir string `protobuf:"bytes,2,opt,name=value_dir,json=valueDir,proto3" json:"valueDir,omitempty"`
	// KvKeyOpts are key/value options.
	KvKeyOpts *kvkey.Config `protobuf:"bytes,3,opt,name=kv_key_opts,json=kvKeyOpts,proto3" json:"kvKeyOpts,omitempty"`
	// NoGenerateKey indicates the controller should not generate a private key if
	// one is not already present. Setting this to false will cause the system to
	// create a new private key if one is not present in the store at startup. If
	// no key is in the store at startup and this is true, returns an error.
	NoGenerateKey bool `protobuf:"varint,4,opt,name=no_generate_key,json=noGenerateKey,proto3" json:"noGenerateKey,omitempty"`
	// NoWriteKey indicates the controller should not write a private key to
	// storage if it generates one. This results in an ephemeral volume peer
	// identity if there is no key present in the store already.
	//
	// Has no effect if the store has a peer private key.
	NoWriteKey bool `protobuf:"varint,25,opt,name=no_write_key,json=noWriteKey,proto3" json:"noWriteKey,omitempty"`
	// Verbose indicates we should log every operation.
	Verbose bool `protobuf:"varint,21,opt,name=verbose,proto3" json:"verbose,omitempty"`
	// BadgerDebug indicates to enable badger debug log messages.
	BadgerDebug bool `protobuf:"varint,24,opt,name=badger_debug,json=badgerDebug,proto3" json:"badgerDebug,omitempty"`
	// VolumeConfig is the volume controller config.
	VolumeConfig *controller.Config `protobuf:"bytes,22,opt,name=volume_config,json=volumeConfig,proto3" json:"volumeConfig,omitempty"`
	// StoreConfig is the store configuration for kvtx.
	StoreConfig *kvtx.Config `protobuf:"bytes,23,opt,name=store_config,json=storeConfig,proto3" json:"storeConfig,omitempty"`
	// NumVersionsToKeep indicates how many versions to keep per key.
	// Defaults to 1.
	NumVersionsToKeep uint32 `protobuf:"varint,7,opt,name=num_versions_to_keep,json=numVersionsToKeep,proto3" json:"numVersionsToKeep,omitempty"`
	// BaseTableSize is the base size for tables.
	// Defaults to 2 << 20.
	BaseTableSize uint64 `protobuf:"varint,26,opt,name=base_table_size,json=baseTableSize,proto3" json:"baseTableSize,omitempty"`
	// LevelSizeMultiplier is SizeOf(Li+1)/SizeOf(Li).
	// Defaults to 10.
	LevelSizeMultiplier uint32 `protobuf:"varint,9,opt,name=level_size_multiplier,json=levelSizeMultiplier,proto3" json:"levelSizeMultiplier,omitempty"`
	// MaxLevels is the maximum number of levels of compaction.
	// Defaults to 7
	MaxLevels uint32 `protobuf:"varint,10,opt,name=max_levels,json=maxLevels,proto3" json:"maxLevels,omitempty"`
	// ValueThreshold if value size >= threshold, only store offsets in tree.
	// Defaults to 1 << 20, 1MB
	ValueThreshold uint64 `protobuf:"varint,27,opt,name=value_threshold,json=valueThreshold,proto3" json:"valueThreshold,omitempty"`
	// NumMemtables is the Maximum number of tables to keep in memory, before
	// stalling.
	// Defaults to 5.
	NumMemtables uint32 `protobuf:"varint,12,opt,name=num_memtables,json=numMemtables,proto3" json:"numMemtables,omitempty"`
	// NumLevelZeroTables affects how LSM tree L0 is handled.
	// Maximum number of Level 0 tables before we start compacting.
	// Defaults to 5.
	NumLevelZeroTables uint32 `protobuf:"varint,13,opt,name=num_level_zero_tables,json=numLevelZeroTables,proto3" json:"numLevelZeroTables,omitempty"`
	// NumLevelZeroTablesStall is the number of level 0 tables to stall at until
	// l0 is compacted.
	// Defaults to 10.
	NumLevelZeroTablesStall uint32 `` /* 134-byte string literal not displayed */
	// BaseLevelSize is the base size for levels.
	// Defaults to 10 << 20.
	BaseLevelSize uint64 `protobuf:"varint,28,opt,name=base_level_size,json=baseLevelSize,proto3" json:"baseLevelSize,omitempty"`
	// ValueLogFileSize is the size of single value log file.
	// (2^30 - 1)*2 when mmapping < 2^31 - 1, max int32.
	// -1 so 2*ValueLogFileSize won't overflow on 32-bit systems.
	// Defaults to 1<<30 - 1
	ValueLogFileSize uint64 `protobuf:"varint,16,opt,name=value_log_file_size,json=valueLogFileSize,proto3" json:"valueLogFileSize,omitempty"`
	// ValueLogMaxEntries is the max number of entries a value log file can hold
	// (approximately). A value log file would be determined by the smaller of its
	// file size and max entries.
	// Defaults to 1000000
	ValueLogMaxEntries uint32 `protobuf:"varint,17,opt,name=value_log_max_entries,json=valueLogMaxEntries,proto3" json:"valueLogMaxEntries,omitempty"`
	// NumCompactors is the number of compaction workers to run concurrently.
	// Defaults to 3.
	NumCompactors uint32 `protobuf:"varint,18,opt,name=num_compactors,json=numCompactors,proto3" json:"numCompactors,omitempty"`
	// NoSyncWrites indicates all writes should not require disk sync before
	// returning. If set, writes will return before the filesystem has confirmed
	// the write is complete. Setting this to false will increase performance but
	// introduces risk of data loss.
	NoSyncWrites bool `protobuf:"varint,20,opt,name=no_sync_writes,json=noSyncWrites,proto3" json:"noSyncWrites,omitempty"`
	// contains filtered or unexported fields
}

Config is the badger volume controller config. Flag Dir is the only mandatory flag.

func (*Config) BuildBadgerOptions 

func (c *Config) BuildBadgerOptions() (*bdb.Options, error)

BuildBadgerOptions builds badger options from the config.

func (*Config) CloneMessageVT 

func (m *Config) CloneMessageVT() protobuf_go_lite.CloneMessage

func (*Config) CloneVT 

func (m *Config) CloneVT() *Config

func (*Config) EqualMessageVT 

func (this *Config) EqualMessageVT(thatMsg any) bool

func (*Config) EqualVT 

func (this *Config) EqualVT(that *Config) bool

func (*Config) EqualsConfig 

func (c *Config) EqualsConfig(other config.Config) bool

EqualsConfig checks if the config is equal to another.

func (*Config) GetBadgerDebug 

func (x *Config) GetBadgerDebug() bool

func (*Config) GetBaseLevelSize 

func (x *Config) GetBaseLevelSize() uint64

func (*Config) GetBaseTableSize 

func (x *Config) GetBaseTableSize() uint64

func (*Config) GetConfigID 

func (c *Config) GetConfigID() string

GetConfigID returns the unique string for this configuration type. This string is stored with the encoded config.

func (*Config) GetDebugVals 

func (c *Config) GetDebugVals() config.DebugValues

GetDebugVals returns the directive arguments as key/value pairs. This should be something like param1="test", param2="test". This is not necessarily unique, and is primarily intended for display.

func (*Config) GetDir 

func (x *Config) GetDir() string

func (*Config) GetKvKeyOpts 

func (x *Config) GetKvKeyOpts() *kvkey.Config

func (*Config) GetLevelSizeMultiplier 

func (x *Config) GetLevelSizeMultiplier() uint32

func (*Config) GetMaxLevels 

func (x *Config) GetMaxLevels() uint32

func (*Config) GetNoGenerateKey 

func (x *Config) GetNoGenerateKey() bool

func (*Config) GetNoSyncWrites 

func (x *Config) GetNoSyncWrites() bool

func (*Config) GetNoWriteKey 

func (x *Config) GetNoWriteKey() bool

func (*Config) GetNumCompactors 

func (x *Config) GetNumCompactors() uint32

func (*Config) GetNumLevelZeroTables 

func (x *Config) GetNumLevelZeroTables() uint32

func (*Config) GetNumLevelZeroTablesStall 

func (x *Config) GetNumLevelZeroTablesStall() uint32

func (*Config) GetNumMemtables 

func (x *Config) GetNumMemtables() uint32

func (*Config) GetNumVersionsToKeep 

func (x *Config) GetNumVersionsToKeep() uint32

func (*Config) GetStoreConfig 

func (x *Config) GetStoreConfig() *kvtx.Config

func (*Config) GetValueDir 

func (x *Config) GetValueDir() string

func (*Config) GetValueLogFileSize 

func (x *Config) GetValueLogFileSize() uint64

func (*Config) GetValueLogMaxEntries 

func (x *Config) GetValueLogMaxEntries() uint32

func (*Config) GetValueThreshold 

func (x *Config) GetValueThreshold() uint64

func (*Config) GetVerbose 

func (x *Config) GetVerbose() bool

func (*Config) GetVolumeConfig 

func (x *Config) GetVolumeConfig() *controller.Config

func (*Config) MarshalJSON 

func (x *Config) MarshalJSON() ([]byte, error)

MarshalJSON marshals the Config to JSON.

func (*Config) MarshalProtoJSON 

func (x *Config) MarshalProtoJSON(s *json.MarshalState)

MarshalProtoJSON marshals the Config message to JSON.

func (*Config) MarshalProtoText 

func (x *Config) MarshalProtoText() string

func (*Config) MarshalToSizedBufferVT 

func (m *Config) MarshalToSizedBufferVT(dAtA []byte) (int, error)

func (*Config) MarshalToVT 

func (m *Config) MarshalToVT(dAtA []byte) (int, error)

func (*Config) MarshalVT 

func (m *Config) MarshalVT() (dAtA []byte, err error)

func (*Config) ProtoMessage 

func (*Config) ProtoMessage()

func (*Config) Reset 

func (x *Config) Reset()

func (*Config) SizeVT 

func (m *Config) SizeVT() (n int)

func (*Config) String 

func (x *Config) String() string

func (*Config) UnmarshalJSON 

func (x *Config) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals the Config from JSON.

func (*Config) UnmarshalProtoJSON 

func (x *Config) UnmarshalProtoJSON(s *json.UnmarshalState)

UnmarshalProtoJSON unmarshals the Config message from JSON.

func (*Config) UnmarshalVT 

func (m *Config) UnmarshalVT(dAtA []byte) error

func (*Config) Validate 

func (c *Config) Validate() error

Validate validates the configuration. This is a cursory validation to see if the values "look correct."

type Factory 

type Factory struct {
	// contains filtered or unexported fields
}

Factory constructs a Badger volume.

func NewFactory 

func NewFactory(bus bus.Bus) *Factory

NewFactory builds a Badger volume factory.

func (*Factory) Construct 

func (t *Factory) Construct(
	ctx context.Context,
	conf config.Config,
	opts controller.ConstructOpts,
) (controller.Controller, error)

Construct constructs the associated controller given configuration.

func (*Factory) ConstructConfig 

func (t *Factory) ConstructConfig() config.Config

ConstructConfig constructs an instance of the controller configuration.

func (*Factory) GetConfigID 

func (t *Factory) GetConfigID() string

GetConfigID returns the unique ID for the config.

func (*Factory) GetControllerID 

func (t *Factory) GetControllerID() string

GetControllerID returns the unique ID for the controller.

func (*Factory) GetVersion 

func (t *Factory) GetVersion() semver.Version

GetVersion returns the version of this controller.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.