pack

Documentation

Index

• Variables
• func GetCommp(calc *commp.Calc, targetPieceSize uint64) (cid.Cid, uint64, error)
• func Pack(ctx context.Context, db *gorm.DB, job model.Job) (*model.Car, error)
• type Assembler
  • func NewAssembler(ctx context.Context, reader storagesystem.Reader, fileRanges []model.FileRange, ...) *Assembler
  • func (a *Assembler) Close() error
  • func (a *Assembler) Read(p []byte) (int, error)

Variables

var ErrFileModified = errors.New("file has been modified")

var ErrNoContent = errors.New("no content to pack")

Functions

func GetCommp

func GetCommp(calc *commp.Calc, targetPieceSize uint64) (cid.Cid, uint64, error)

GetCommp calculates the data commitment (CommP) and the piece size based on the provided commp.Calc instance and target piece size. It ensures that the calculated piece size matches the target piece size specified. If necessary, it pads the data to meet the target piece size.

Parameters:

• calc: A pointer to a commp.Calc instance, which has been used to write data and will be used to calculate the piece commitment for that data.
• targetPieceSize: The desired size of the piece, specified in bytes.

Returns:

• cid.Cid: A CID (Content Identifier) representing the data commitment (CommP).
• uint64: The size of the piece, in bytes, after potential padding.
• error: An error indicating issues during the piece commitment calculation, padding, or CID conversion, or nil if the operation was successful.

func Pack

func Pack(
	ctx context.Context,
	db *gorm.DB,
	job model.Job,
) (*model.Car, error)

Pack takes in a Job and processes its attachment by reading it, possibly encrypting it, splitting it into manageable chunks, and then storing those chunks into a designated storage. If the job's attachment requires encryption, it will be encrypted using the specified encryption method. The function returns a slice of Car objects which represent the stored chunks and an error if any occurred.

Parameters:

• ctx: The context which controls the lifetime of the operation.
• db: The gorm database instance used for querying and updating database records.
• job: The Job model instance which contains information about the attachment to be processed.

Returns:

• A slice of model.Car instances representing stored chunks.
• An error, if any occurred during the operation.

Types

type Assembler

type Assembler struct {
	// contains filtered or unexported fields
}

Assembler assembles various objects and data streams into a coherent output stream. It uses the provided file ranges, and internal buffers to produce the desired output.

func NewAssembler

func NewAssembler(ctx context.Context, reader storagesystem.Reader,
	fileRanges []model.FileRange, noInline bool, skipInaccessibleFiles bool) *Assembler

NewAssembler initializes a new Assembler instance with the given parameters.

func (*Assembler) Close

func (a *Assembler) Close() error

Close closes the assembler and all of its underlying readers

func (*Assembler) Read

func (a *Assembler) Read(p []byte) (int, error)

Read reads data from the buffer, or fetches the next chunk from fileRanges if the buffer is empty. It will assemble links if needed and respect the context's cancellation or deadline.