store

Documentation

Index

• Variables
• type FileReferenceBlockStore
  • func (i *FileReferenceBlockStore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)
  • func (i *FileReferenceBlockStore) DeleteBlock(ctx context.Context, cid cid.Cid) error
  • func (i *FileReferenceBlockStore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, error)
  • func (i *FileReferenceBlockStore) GetSize(ctx context.Context, c cid.Cid) (int, error)
  • func (i *FileReferenceBlockStore) Has(ctx context.Context, cid cid.Cid) (bool, error)
  • func (i *FileReferenceBlockStore) HashOnRead(enabled bool)
  • func (i *FileReferenceBlockStore) Put(ctx context.Context, block blocks.Block) error
  • func (i *FileReferenceBlockStore) PutMany(ctx context.Context, i2 []blocks.Block) error
• type PieceReader
  • func NewPieceReader(ctx context.Context, car model.Car, storage model.Storage, ...) (*PieceReader, error)
  • func (pr *PieceReader) Clone() *PieceReader
  • func (pr *PieceReader) Close() error
  • func (pr *PieceReader) Read(p []byte) (n int, err error)
  • func (pr *PieceReader) Seek(offset int64, whence int) (int64, error)

Variables

var ErrFileHasChanged = errors.New("file has changed")

var ErrFileNotProvided = errors.New("file not provided")

var ErrIncontiguousBlocks = errors.New("Blocks must be contiguous")

var ErrInvalidEndOffset = errors.New("last block must end at car end")

var ErrInvalidStartOffset = errors.New("first block must start at car Header")

var ErrInvalidVarintLength = errors.New("varint read does not match varint length")

var ErrInvalidWhence = errors.New("invalid whence")

var ErrNegativeOffset = errors.New("negative offset")

var ErrNoCarBlocks = errors.New("no Blocks provided")

var ErrOffsetOutOfRange = errors.New("position past end of file")

var ErrTruncated = errors.New("original file has been truncated")

var ErrVarintDoesNotMatchBlockLength = errors.New("varint does not match block length")

Types

type FileReferenceBlockStore

type FileReferenceBlockStore struct {
	DBNoContext *gorm.DB
}

FileReferenceBlockStore is a struct that represents a block store backed by file references. It uses a GORM database.

Fields:

• DBNoContext: The GORM database used for storage. This should be initialized and connected to a database before use.

func (*FileReferenceBlockStore) AllKeysChan

func (i *FileReferenceBlockStore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)

func (*FileReferenceBlockStore) DeleteBlock

func (i *FileReferenceBlockStore) DeleteBlock(ctx context.Context, cid cid.Cid) error

func (*FileReferenceBlockStore) Get

func (i *FileReferenceBlockStore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, error)

func (*FileReferenceBlockStore) GetSize

func (i *FileReferenceBlockStore) GetSize(ctx context.Context, c cid.Cid) (int, error)

GetSize is a method on the FileReferenceBlockStore struct that retrieves the size of a block with the specified CID from the store. It uses the context for the database operation and returns an error if the operation fails.

Parameters:

• ctx: The context for the database operation. This can be used to cancel the operation or set a deadline.
• c: The CID of the block whose size is to be retrieved.

Returns:

• The size of the block in bytes, and an error if the operation failed. If the block does not exist in the store, it returns a

func (*FileReferenceBlockStore) Has

func (i *FileReferenceBlockStore) Has(ctx context.Context, cid cid.Cid) (bool, error)

Has is a method on the FileReferenceBlockStore struct that checks if a block with the specified CID exists in the store. It uses the context for the database operation and returns an error if the operation fails.

Parameters:

• ctx: The context for the database operation. This can be used to cancel the operation or set a deadline.
• cid: The CID of the block to check for.

Returns:

• A boolean indicating whether the block exists in the store, and an error if the operation failed.

func (*FileReferenceBlockStore) HashOnRead

func (i *FileReferenceBlockStore) HashOnRead(enabled bool)

func (*FileReferenceBlockStore) Put

func (i *FileReferenceBlockStore) Put(ctx context.Context, block blocks.Block) error

func (*FileReferenceBlockStore) PutMany

func (i *FileReferenceBlockStore) PutMany(ctx context.Context, i2 []blocks.Block) error

type PieceReader

type PieceReader struct {
	// contains filtered or unexported fields
}

PieceReader is a struct that represents a reader for pieces of data.

Fields:

• ctx: The context in which the PieceReader operates. This can be used to cancel operations or set deadlines.
• fileSize: The size of the file being read.
• header: A byte slice representing the header of the file.
• handlerMap: Handler map from storage ID to the actual RCloneHandler
• carBlocks: A slice of CarBlocks. These represent the blocks of data in the CAR (Content Addressable Archive) format.
• files: A map where the keys are file ID. This represents the files of data being read.
• reader: An io.ReadCloser that is used to read the data and close the reader when done.
• readerFor: A uint64 file ID that represents the current file being read.
• pos: An int64 that represents the current position in the data being read.
• blockIndex: An integer that represents the index of the current block being read.

func NewPieceReader

func NewPieceReader(
	ctx context.Context,
	car model.Car,
	storage model.Storage,
	carBlocks []model.CarBlock,
	files []model.File,
) (
	*PieceReader,
	error,
)

NewPieceReader is a function that creates a new PieceReader. It takes a context, a Car model, a Source model, a slice of CarBlock models, a slice of File models, and a HandlerResolver as input. It validates the input data and returns an error if any of it is invalid. The returned PieceReader starts at the beginning of the data (position 0).

Parameters:

• ctx: The context for the new PieceReader. This can be used to cancel operations or set deadlines.
• car: A Car model that represents the CAR (Content Addressable Archive) file being read.
• source: A Source model that represents the source of the data.
• carBlocks: A slice of CarBlock models that represent the blocks of data in the CAR file.
• files: A slice of File models that represent the files of data being read.
• resolver: A HandlerResolver that is used to resolve the handler for the source of the data.

Returns:

• A new PieceReader that has been initialized with the provided data, and an error if the initialization failed.

func (*PieceReader) Clone

func (pr *PieceReader) Clone() *PieceReader

Clone is a method on the PieceReader struct that creates a new PieceReader with the same state as the original. The new PieceReader starts at the beginning of the data (position 0).

Returns:

• A new PieceReader that has the same state as the original, but starting at position 0.

func (*PieceReader) Close

func (pr *PieceReader) Close() error

func (*PieceReader) Read

func (pr *PieceReader) Read(p []byte) (n int, err error)

Read is a method on the PieceReader struct that reads data into the provided byte slice.

• It reads data from the current position of the PieceReader and advances the position accordingly.
• If the context of the PieceReader has been cancelled, it returns an error immediately.
• If the end of the file has been reached, it returns io.EOF.
• If the PieceReader is currently at a block boundary, it advances to the next block before reading data.
• If the PieceReader is currently at a varint or CID boundary within a block, it reads the varint or CID data.
• If the PieceReader is currently at a raw block boundary within a block, it reads the raw block data.
• If the PieceReader is currently at an file boundary within a block, it reads the file data.
• If the PieceReader encounters an error while reading data, it returns the error.

Parameters:

• p: The byte slice to read data into.

Returns:

• The number of bytes read, and an error if the read operation failed.

func (*PieceReader) Seek

func (pr *PieceReader) Seek(offset int64, whence int) (int64, error)

Seek is a method on the PieceReader struct that changes the position of the reader. It takes an offset and a 'whence' value as input, similar to the standard io.Seeker interface. The offset is added to the position determined by 'whence'.

• If 'whence' is io.SeekStart, the offset is from the start of the file.
• If 'whence' is io.SeekCurrent, the offset is from the current position.
• If 'whence' is io.SeekEnd, the offset is from the end of the file.
• If the resulting position is negative or beyond the end of the file, an error is returned.
• If a reader is currently open, it is closed before the position is changed.

Parameters:

• offset: The offset to move the position by. Can be negative.
• whence: The position to move the offset from. Must be one of io.SeekStart, io.SeekCurrent, or io.SeekEnd.

Returns:

• The new position after seeking, and an error if the seek operation failed.