blobman

package
v0.0.0-...-b080474 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 21, 2025 License: GPL-3.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	TableFileExt = ".slot"
	PackFileExt  = ".pack"
)
View Source 
const (
	// TableEntrySize is the size of a row in the table.
	// It actually uses less space than this, but we want to align to 4096 bytes.
	TableEntrySize  = 32
	TableHeaderSize = 32
	PageSize        = 4096
	EntriesPerPage  = PageSize / TableEntrySize
	// DefaultMaxTableLen is the maximum length of a table in rows.
	DefaultMaxTableLen = (1<<20 - TableHeaderSize) / TableEntrySize
)
View Source 
const DefaultMaxPackSize = 1 << 26

Variables

This section is empty.

Functions

func Copy 

func Copy(srcTab Table, srcPack Pack, dstTab Table, dstPack Pack) (int, error)

Copy copies entries from src{Tab,Pack} to dst{Tab,Pack}. If there is not enough space available in either the table or the pack, then an error is returned. Copy will not copy tombstones, and will sort all the entries by key before copying.

func CreatePackFile 

func CreatePackFile(root *os.Root, prefix Prefix120, maxSize uint32) (*os.File, error)

CreatePackFile creates a file configured for a pack in the filesystem, and returns it.

func CreateTableFile 

func CreateTableFile(root *os.Root, prefix Prefix120, maxSize uint32) (*os.File, error)

CreateTableFile creates a file configured for a table in the filesystem, and returns it. maxSize is the maximum size of the table in bytes, NOT the number of rows.

func KeyCompare 

func KeyCompare(a, b Key) int

func LoadPackFile 

func LoadPackFile(root *os.Root, prefix Prefix120) (*os.File, error)

func LoadTableFile 

func LoadTableFile(root *os.Root, prefix Prefix120) (*os.File, error)

Types

type BitMap 

type BitMap []uint64

func (BitMap) Get 

func (bm BitMap) Get(i int) bool

func (*BitMap) Set 

func (bm *BitMap) Set(i int, v bool)

type Key 

type Key [2]uint64

Key is a 128 bit key. The 0th bit is considered the first bit, and that is at k[0] & (1 << 0).

func KeyFromBytes 

func KeyFromBytes(b []byte) Key

func (Key) Bytes 

func (k Key) Bytes() []byte

func (Key) Data 

func (k Key) Data() (ret [16]byte)

func (Key) IsZero 

func (k Key) IsZero() bool

func (Key) Rotate 

func (k Key) Rotate(i int) Key

func (Key) ShiftIn 

func (k Key) ShiftIn(i int) Key

ShiftIn shifts the key into 0. The lowest bits are discarded, zeros are shifted in to the highest bits.

func (Key) ToPrefix 

func (k Key) ToPrefix(numBits uint8) Prefix120

ToPrefix takes the first numBits bits of the key and includes those in a prefix. The last 7 bits of the key must be dropped. ToPrefix will panic, the same as NewPrefix120, if numBits is greater than 120.

func (Key) Uint16 

func (k Key) Uint16(i int) uint16

func (Key) Uint64 

func (k Key) Uint64(i int) uint64

Uint64 returns the 64 bit integer at the given index. The index is 0 or 1.

func (Key) Uint8 

func (k Key) Uint8(i int) uint8

func (Key) Uint8Len 

func (k Key) Uint8Len() int

Uint8Len returns the number of 8 bit integers in the key.

type Pack 

type Pack struct {
	// contains filtered or unexported fields
}

Pack is an append-only file on disk.

func NewPack 

func NewPack(f *os.File, nextOffset uint32) (Pack, error)

NewPack calls f.Stat() to determine the max size Then it mmaps the file.

func (*Pack) Append 

func (pk *Pack) Append(data []byte) uint32

Append appends data to the pack and returns the offset of the data.

func (*Pack) CanAppend 

func (pk *Pack) CanAppend(dataLen uint32) bool

func (Pack) Close 

func (pk Pack) Close() error

func (*Pack) Flush 

func (pk *Pack) Flush() error

func (Pack) FreeSpace 

func (pk Pack) FreeSpace() uint32

func (*Pack) Get 

func (pk *Pack) Get(offset, length uint32, fn func(data []byte)) error

Get reads data from the pack by offset and size.

type Prefix120 

type Prefix120 struct {
	// contains filtered or unexported fields
}

Prefix120 is a prefix of at most 120 bits. Prefix120 takes up 128 bits. A prefix refers to a set of keys.

func NewPrefix121 

func NewPrefix121(data [15]byte, numBits uint8) Prefix120

func (Prefix120) ChildrenDir 

func (prefix Prefix120) ChildrenDir() (string, error)

ChildrenDir returns the directory that contains all the children of this prefix.

func (Prefix120) Data 

func (p Prefix120) Data() (ret [15]byte)

func (Prefix120) Len 

func (p Prefix120) Len() int

func (Prefix120) PackPath 

func (p Prefix120) PackPath() (string, error)

func (Prefix120) Path 

func (p Prefix120) Path() (string, error)

func (Prefix120) ShiftIn 

func (p Prefix120) ShiftIn(i int) Prefix120

func (Prefix120) TablePath 

func (p Prefix120) TablePath() (string, error)

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store stores blobs on in the filesystem.

func New 

func New(root *os.Root) *Store

func (*Store) Close 

func (db *Store) Close() error

func (*Store) Delete 

func (db *Store) Delete(key Key) error

Delete overwrites any tables containing key with a tombstone.

func (*Store) Flush 

func (db *Store) Flush() error

func (*Store) Get 

func (db *Store) Get(key Key, fn func(data []byte)) (bool, error)

Get finds key if it exists and calls fn with the data. The data must not be used outside the callback.

func (*Store) Maintain 

func (db *Store) Maintain() error

Maintain performs background maintenance tasks on the trie.

func (*Store) Put 

func (db *Store) Put(key Key, data []byte) (bool, error)

Put finds a spot for key, and writes data to it. If the key already exists, then the write is ignored and false is returned. If some data with key does not exist in the system after this call, then an error is returned.

type Table 

type Table struct {
	// contains filtered or unexported fields
}

Table is an unordered append-only list of entries. Each entry points into a pack, and all entries are the same size.

func NewTable 

func NewTable(f *os.File) (Table, error)

NewTable mmaps a file and returns a Table. count should be the number of rows in the table.

func (*Table) Append 

func (idx *Table) Append(ent TableEntry) uint32

func (*Table) CanAppend 

func (idx *Table) CanAppend() bool

func (Table) Capacity 

func (idx Table) Capacity() uint32

Capacity is the total number of rows that can be stored in the table.

func (*Table) Close 

func (idx *Table) Close() error

func (*Table) Flush 

func (idx *Table) Flush() error

func (*Table) Len 

func (idx *Table) Len() uint32

Len returns the number of rows in the table.

func (*Table) SetSlot 

func (idx *Table) SetSlot(slot uint32, ent TableEntry)

func (*Table) Slot 

func (idx *Table) Slot(i uint32) (ret TableEntry)

func (*Table) SlotOffset 

func (idx *Table) SlotOffset(i uint32) uint32

SlotOffset returns the offset of the i-th slot in the table. Slot 0 starts at 4 bytes, to make room for the row count at the beginning.

func (*Table) SlotsLeft 

func (idx *Table) SlotsLeft() uint32

func (*Table) Tombstone 

func (idx *Table) Tombstone(slot uint32)

Tombstone writes a tombstone at the given slot.

type TableEntry 

type TableEntry struct {
	Key Key

	Offset uint32
	Len    uint32
}

func (*TableEntry) IsTombstone 

func (ent *TableEntry) IsTombstone() bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.