cephprovider

Documentation

Index

• Constants
• func UUIDSliceToArr(id []byte) [16]byte
• type CacheItem
• type CephCache
• type CephSegment
  • func (seg *CephSegment) BaseAddress() uint64
  • func (seg *CephSegment) Flush()
  • func (seg *CephSegment) Unlock()
  • func (seg *CephSegment) Write(uuid []byte, address uint64, data []byte) (uint64, error)
• type CephStorageProvider
  • func (sp *CephStorageProvider) CreateDatabase(cfg configprovider.Configuration) error
  • func (sp *CephStorageProvider) GetRH() int
  • func (sp *CephStorageProvider) GetStreamVersion(uuid []byte) uint64
  • func (sp *CephStorageProvider) Initialize(cfg configprovider.Configuration)
  • func (sp *CephStorageProvider) LockSegment(uuid []byte) bprovider.Segment
  • func (sp *CephStorageProvider) Read(uuid []byte, address uint64, buffer []byte) []byte
  • func (sp *CephStorageProvider) ReadSuperBlock(uuid []byte, version uint64, buffer []byte) []byte
  • func (sp *CephStorageProvider) SetStreamVersion(uuid []byte, version uint64)
  • func (sp *CephStorageProvider) WriteSuperBlock(uuid []byte, version uint64, buffer []byte)

Constants

const ADDR_LOCK_SIZE = 0x1000000000

4096 blocks per addr lock

const ADDR_OBJ_SIZE = 0x0001000000

const MAX_EXPECTED_OBJECT_SIZE = 20485

Just over the DBSIZE

const METADATA_BASE = 0xFF00000000000000

We know we won't get any addresses here, because this is the relocation base as well

const NUM_RHANDLES = 128

const NUM_WHANDLES = 128

const OFFSET_MASK = 0xFFFFFF

const RADOS_CACHE_SIZE = NUM_RHANDLES * 2

The number of RADOS blocks to cache (up to 16MB each, probably only 1.6MB each)

const R_ADDRMASK = ^((uint64(1) << 20) - 1)

We are caching 1MB blocks for read, so the address should have the bottom 20 bits clear

const R_CHUNKSIZE = 1 << 20

const R_OFFSETMASK = (uint64(1) << 20) - 1

const SBLOCKS_PER_CHUNK = 1 << SBLOCK_CHUNK_SHIFT

const SBLOCK_CHUNK_MASK = 0xFFFFF

const SBLOCK_CHUNK_SHIFT = 20

Makes 16MB for 16B sblocks

const SBLOCK_SIZE = 16

const SEGCACHE_SIZE = 1024

const WCACHE_SIZE = 1 << 20

1MB for write cache, I doubt we will ever hit this tbh

const WORTH_CACHING = OFFSET_MASK - MAX_EXPECTED_OBJECT_SIZE

This is how many uuid/address pairs we will keep to facilitate appending to segments instead of creating new ones.

Functions

func UUIDSliceToArr

func UUIDSliceToArr(id []byte) [16]byte

Types

type CacheItem

type CacheItem struct {
	// contains filtered or unexported fields
}

type CephCache

type CephCache struct {
	// contains filtered or unexported fields
}

type CephSegment

type CephSegment struct {
	// contains filtered or unexported fields
}

func (*CephSegment) BaseAddress

func (seg *CephSegment) BaseAddress() uint64

Returns the address of the first free word in the segment when it was locked

func (*CephSegment) Flush

func (seg *CephSegment) Flush()

Block until all writes are complete. Note this does not imply a flush of the underlying files.

func (*CephSegment) Unlock

func (seg *CephSegment) Unlock()

Unlocks the segment for the StorageProvider to give to other consumers Implies a flush

func (*CephSegment) Write

func (seg *CephSegment) Write(uuid []byte, address uint64, data []byte) (uint64, error)

Writes a slice to the segment, returns immediately Returns nil if op is OK, otherwise ErrNoSpace or ErrInvalidArgument It is up to the implementer to work out how to report no space immediately The uint64 is the address to be used for the next write

type CephStorageProvider

type CephStorageProvider struct {
	// contains filtered or unexported fields
}

func (*CephStorageProvider) CreateDatabase

func (sp *CephStorageProvider) CreateDatabase(cfg configprovider.Configuration) error

Called to create the database for the first time This doesn't lock, but nobody else would be trying to do the same thing at the same time, so...

func (*CephStorageProvider) GetRH

func (sp *CephStorageProvider) GetRH() int

func (*CephStorageProvider) GetStreamVersion

func (sp *CephStorageProvider) GetStreamVersion(uuid []byte) uint64

Gets the version of a stream. Returns 0 if none exists.

func (*CephStorageProvider) Initialize

func (sp *CephStorageProvider) Initialize(cfg configprovider.Configuration)

Called at startup of a normal run

func (*CephStorageProvider) LockSegment

func (sp *CephStorageProvider) LockSegment(uuid []byte) bprovider.Segment

Lock a segment, or block until a segment can be locked Returns a Segment struct Implicit unchecked assumption: you cannot lock more than one segment for a given uuid (without unlocking them in between). It will break segcache

func (*CephStorageProvider) Read

func (sp *CephStorageProvider) Read(uuid []byte, address uint64, buffer []byte) []byte

Read the blob into the given buffer

func (*CephStorageProvider) ReadSuperBlock

func (sp *CephStorageProvider) ReadSuperBlock(uuid []byte, version uint64, buffer []byte) []byte

Read the given version of superblock into the buffer. mebbeh we want to cache this?

func (*CephStorageProvider) SetStreamVersion

func (sp *CephStorageProvider) SetStreamVersion(uuid []byte, version uint64)

Sets the version of a stream. If it is in the past, it is essentially a rollback, and although no space is freed, the consecutive version numbers can be reused note to self: you must make sure not to call ReadSuperBlock on versions higher than you get from GetStreamVersion because they might succeed

func (*CephStorageProvider) WriteSuperBlock

func (sp *CephStorageProvider) WriteSuperBlock(uuid []byte, version uint64, buffer []byte)

Writes a superblock of the given version TODO I think the storage will need to chunk this, because sb logs of gigabytes are possible