eris

package
v0.0.0-...-bc42473 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 6, 2024 License: BSD-3-Clause Imports: 17 Imported by: 0

Documentation

Overview

Encoding for Robust Immutable Storage (ERIS).

Index

Constants

View Source 
const (
	SmallChunkSize = 1 << 10
	LargeChunkSize = 1 << 15
)
View Source 
const (
	// CBOR tag registered for ERIS capability data.
	CapabilityCBORTag = 276
)

Variables

View Source 
var (
	ErrChunkSizeMismatch = errors.New("chunk size mismatch")
	ErrChunkNotFound     = errors.New("chunk not found")
	ErrInvalidChunkSize  = errors.New("invalid chunk size")
	ErrNotPermitted      = errors.New("operation not permitted")
	ErrChunkCorrupt      = errors.New("corrupt chunk detected")
)

Functions

func ChunkBinaryLogarithm 

func ChunkBinaryLogarithm(cap ReadCapability) int

Return the binary logarithm of the chunk size for a given ReadCapability. For 1KiB this is 10 (0xa) and for 32KiB this is 15 (0xf).

func Get 

func Get(store Store, ctx context.Context, chunk []byte, pair Pair, level Level) (err error)

Get the plaintext chunk from a store for ref and key at the given tree level.

func Merge 

func Merge(ctx context.Context, dst Store, src IteratorStore) (n int64, err error)

Merge all chunks from src Store into dst Store and return the total number of bytes copied.

func TreeArity 

func TreeArity(cap ReadCapability) int

Return the arity of the ERIS tree for this capability. This is the maximum number of ref-key pairs in any block of the tree.

func Verify 

func Verify(chunk []byte, ref Ref) (err error)

Verify the correspondence between chunk and ref.

Types

type AllowOps 

type AllowOps struct {
	Get, Put bool
}

Type for disallowing operations when opening a store.

type ChunkIteration 

type ChunkIteration func(ctx context.Context, chunk []byte, ref Ref) error

ChunkIteration is the type of the function called by IterateBlocks on blocks within a store. If a non-nil error is returned the iteration is process is halted.

type ChunkSize 

type ChunkSize int
const (
	Size1KiB  ChunkSize = 1 << 10
	Size32KiB ChunkSize = 32 << 10
)

type DirectoryStore 

type DirectoryStore struct {
	// contains filtered or unexported fields
}

The DirectoryStore stores ERIS chunks on the local file-system.

func OpenDirectoryStore 

func OpenDirectoryStore(path string, allow AllowOps) (*DirectoryStore, error)

func (*DirectoryStore) Allowed 

func (store *DirectoryStore) Allowed() AllowOps

func (*DirectoryStore) Close 

func (store *DirectoryStore) Close() (err error)

func (*DirectoryStore) Drop 

func (store *DirectoryStore) Drop(ref Ref) (err error)

func (*DirectoryStore) Get 

func (store *DirectoryStore) Get(ctx context.Context, chunk []byte, ref Ref) (err error)

func (*DirectoryStore) Has 

func (store *DirectoryStore) Has(ctx context.Context, ref Ref, chunkSize int) (present bool)

func (*DirectoryStore) Iterate 

func (store *DirectoryStore) Iterate(ctx context.Context, iter ChunkIteration) (err error)

func (*DirectoryStore) Put 

func (store *DirectoryStore) Put(ctx context.Context, chunk []byte, ref Ref) (err error)

type DisableVerificationContextKey 

type DisableVerificationContextKey struct{}

Type of key to set on a Context to disable chunk verification.

type DiscardStore 

type DiscardStore struct {
}

The DiscardStore simply discards any blocks that are Put to it.

func (DiscardStore) Allowed 

func (store DiscardStore) Allowed() AllowOps

func (DiscardStore) Close 

func (store DiscardStore) Close() error

func (DiscardStore) Drop 

func (store DiscardStore) Drop(ref Ref) error

func (DiscardStore) Get 

func (store DiscardStore) Get(ctx context.Context, chunk []byte, ref Ref) error

func (DiscardStore) Has 

func (store DiscardStore) Has(ctx context.Context, ref Ref, chunkSize int) bool

func (DiscardStore) Put 

func (store DiscardStore) Put(ctx context.Context, chunk []byte, ref Ref) error

type IteratorStore 

type IteratorStore interface {
	Store

	// Call iter at least once for every chunk and ref pair in a Store.
	Iterate(ctx context.Context, iter ChunkIteration) error
}

An IteratorStore is a variant of a store that supports iteration over its chunks.

type Key 

type Key [32]byte

type Level 

type Level uint8

type MemoryStore 

type MemoryStore struct {
	// contains filtered or unexported fields
}

The MemoryStore holds ERIS chunks in memory. Chunks are not verified on retrieval.

func NewMemoryStore 

func NewMemoryStore() *MemoryStore

func (*MemoryStore) Allowed 

func (store *MemoryStore) Allowed() AllowOps

func (*MemoryStore) Close 

func (store *MemoryStore) Close() error

func (*MemoryStore) Drop 

func (store *MemoryStore) Drop(ref Ref) error

func (*MemoryStore) Get 

func (store *MemoryStore) Get(ctx context.Context, chunk []byte, ref Ref) (err error)

func (*MemoryStore) Has 

func (store *MemoryStore) Has(ctx context.Context, ref Ref, chunkSize int) (present bool)

func (*MemoryStore) Iterate 

func (store *MemoryStore) Iterate(ctx context.Context, iter ChunkIteration) (err error)

func (*MemoryStore) Put 

func (store *MemoryStore) Put(ctx context.Context, chunk []byte, ref Ref) (err error)

type Multiplexer 

type Multiplexer struct {
	// contains filtered or unexported fields
}

func NewMultiplexer 

func NewMultiplexer(stores ...Store) (m *Multiplexer)

Create a new Store Multiplexer that combines operations over multiple stores. Put operations are issued to each store. Get operations are issued to each store in order of lowest average latency until the Get succeedes.

All Stores passed here are closed when Close is called at the Multiplexer.

func (*Multiplexer) Add 

func (m *Multiplexer) Add(store Store, cache bool)

Add a Store to the Multiplexer.

If cache is true then any chunks that are retrieved from a non-cache store will be replicated into the passed store. The store will be closed when Close is called at the Multiplexer.

func (*Multiplexer) Allowed 

func (m *Multiplexer) Allowed() (result AllowOps)

func (*Multiplexer) Close 

func (m *Multiplexer) Close() (err error)

func (*Multiplexer) Drop 

func (m *Multiplexer) Drop(ref Ref) (err error)

func (*Multiplexer) Get 

func (m *Multiplexer) Get(ctx context.Context, chunk []byte, ref Ref) (err error)

func (*Multiplexer) Has 

func (m *Multiplexer) Has(ctx context.Context, ref Ref, chunkSize int) bool

func (*Multiplexer) Put 

func (m *Multiplexer) Put(ctx context.Context, chunk []byte, ref Ref) (err error)

type Pair 

type Pair struct {
	Ref Ref
	Key Key
}

func Put 

func Put(store Store, ctx context.Context, chunk []byte, secret Secret, level Level) (Pair, error)

Put the plaintext chunk to a store for the given convergence secret and tree level.

type ReadCapability 

type ReadCapability struct {
	Pair      Pair
	Level     Level
	ChunkSize ChunkSize
}

func ConvergentReaderCapability 

func ConvergentReaderCapability(r io.Reader) (cap ReadCapability, err error)

Determine the convergent ERIS read capability for the contents of an io.Reader.

func Copy 

func Copy(store Store, r io.Reader, secret *Secret) (copied int64, cap ReadCapability, err error)

Copy the contents of a Reader to a Store using the specified convergence secret. The ERIS block size is selected automatically. If less than 16KiB can be read from the Reader then the block size is 1KiB, if 16KiB or more then the block size is 32KiB.

func CopyConvergent 

func CopyConvergent(store Store, r io.Reader) (int64, ReadCapability, error)

Copy the contents of a Reader to a Store using the null convergence secret.

func CopyUnique 

func CopyUnique(store Store, r io.Reader) (int64, ReadCapability, error)

Copy the contents of a Reader to a Store using a unique convergence secret.

func DecodeCap 

func DecodeCap(data []byte) (cap ReadCapability, err error)

func ParseUrn 

func ParseUrn(urn string) (cap ReadCapability, err error)

func (ReadCapability) Bytes 

func (cap ReadCapability) Bytes() []byte

func (*ReadCapability) MarshalBinary 

func (cap *ReadCapability) MarshalBinary() (data []byte, err error)

func (ReadCapability) MarshalCBOR 

func (cap ReadCapability) MarshalCBOR() (data []byte, err error)

MarshalCBOR encodes a read capability to a CBOR tagged byte sequence. CDDL: #6.276(bstr), see https://eris.codeberg.page/eer/cbor.xml

func (*ReadCapability) MarshalText 

func (cap *ReadCapability) MarshalText() (text []byte, err error)

func (ReadCapability) String 

func (cap ReadCapability) String() (urn string)

func (*ReadCapability) UnmarshalBinary 

func (cap *ReadCapability) UnmarshalBinary(data []byte) error

func (*ReadCapability) UnmarshalCBOR 

func (cap *ReadCapability) UnmarshalCBOR(data []byte) error

UnmarshalCBOR decodes a CBOR tagged byte sequence to a read capability. CDDL: #6.276(bstr), see https://eris.codeberg.page/eer/cbor.xml

func (*ReadCapability) UnmarshalText 

func (cap *ReadCapability) UnmarshalText(text []byte) error

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

func NewReader 

func NewReader(store Store, cap ReadCapability) (dec *Reader, err error)

Create a new io.Reader that decodes the given ReadCapability from the given Store.

func (*Reader) Close 

func (dec *Reader) Close() error

func (*Reader) Read 

func (dec *Reader) Read(p []byte) (n int, err error)

func (*Reader) Seek 

func (dec *Reader) Seek(offset int64, whence int) (pos int64, err error)

func (*Reader) WithContext 

func (dec *Reader) WithContext(ctx context.Context)

Set a Context for this Reader. The reader will otherwise use the background context by default.

type Ref 

type Ref [32]byte

func RefOf 

func RefOf(chunk []byte) (ref Ref, err error)

Hash a chunk to produce a Ref.

func (Ref) String 

func (r Ref) String() string

type Secret 

type Secret [32]byte

type Store 

type Store interface {
	// Return a set ofallowed operations.
	Allowed() AllowOps

	// Get a chunk from the the store.
	// Store implementations are responsible for
	// verifying chunks before returning them, unless
	// DisableVerificationContextKey is set in the Context
	// with a non-nil value.
	Get(ctx context.Context, chunk []byte, ref Ref) error

	// Return true if the Store has a Ref.
	Has(ctx context.Context, ref Ref, chunkSize int) bool

	// Put a chunk into the store.
	// Store implementations do not verify a correspondence
	// between ref and chunk, that is the caller's responsibility.
	Put(ctx context.Context, chunk []byte, ref Ref) error

	// Drop a chunk from the Store.
	Drop(ref Ref) error

	// Flush and close the resources of a Store.
	Close() error
}

A Store represents some service for storing and retrieving raw ERIS blocks.

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

func NewConvergentWriter 

func NewConvergentWriter(store Store, chunkSize ChunkSize) (*Writer, error)

Create a new Writer using the null convergence secret. This writer will produce an identical ReadCapability given the same ChunkSize and content.

func NewUniqueWriter 

func NewUniqueWriter(store Store, chunkSize ChunkSize) (*Writer, error)

Create a new Writer using a convergence secret generated from a system source of randomness.

func NewWriter 

func NewWriter(store Store, secret *Secret, chunkSize ChunkSize) (enc *Writer, err error)

func (*Writer) Cap 

func (enc *Writer) Cap() (cap ReadCapability, err error)

func (*Writer) Write 

func (enc *Writer) Write(data []byte) (int, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.