core

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 23, 2025 License: MIT Imports: 14 Imported by: 6

Documentation

Overview

Package core defines structure to hold data and relative functions.

Index

Constants

View Source 
const (
	// DefaultChunkSize represents the default chunk size in bytes.
	DefaultChunkSize = 256

	// DefaultSegmentMaxChunks represents the default maximum number of chunks within a segment.
	DefaultSegmentMaxChunks = 1024

	// DefaultSegmentSize represents the default segment size in bytes.
	DefaultSegmentSize = DefaultChunkSize * DefaultSegmentMaxChunks
)

Variables

View Source 
var (
	EmptyChunk     = make([]byte, DefaultChunkSize)
	EmptyChunkHash = crypto.Keccak256Hash(EmptyChunk)
)
View Source 
var (
	// ErrFileRequired is returned when manipulate on a folder.
	ErrFileRequired = errors.New("file required")

	// ErrFileEmpty is returned when empty file opened.
	ErrFileEmpty = errors.New("file is empty")
)

Functions

func ComputePaddedSize 

func ComputePaddedSize(chunks uint64) (uint64, uint64)

func Exists 

func Exists(name string) (bool, error)

func IteratorPaddedSize 

func IteratorPaddedSize(dataSize int64, flowPadding bool) uint64

func MerkleRoot added in v0.6.0 

func MerkleRoot(filename string) (common.Hash, error)

MerkleRoot returns the merkle root hash of a file on disk

func MerkleTree 

func MerkleTree(data IterableData) (*merkle.Tree, error)

MerkleTree create merkle tree of the data.

func NextPow2 added in v0.6.2 

func NextPow2(input uint64) uint64

func NumSegmentsPadded 

func NumSegmentsPadded(data IterableData) int

NumSegmentsPadded return the number of segments of padded data

func NumSplits 

func NumSplits(total int64, unit int) uint64

func PaddedSegmentRoot added in v0.6.2 

func PaddedSegmentRoot(segmentIndex uint64, chunks []byte, fileSize int64) (common.Hash, uint64)

PaddedSegmentRoot calculates the Merkle root for a given segment based on its index, the chunk data, and the file size. It handles the logic of padding empty chunks if the segment is the last one and doesn't have enough data to form a complete segment.

func ReadAt 

func ReadAt(data IterableData, readSize int, offset int64, paddedSize uint64) ([]byte, error)

ReadAt read data at specified offset, paddedSize is the size of data after padding.

func SegmentRange added in v0.6.2 

func SegmentRange(startChunkIndex, fileSize uint64) (startSegmentIndex, endSegmentIndex uint64)

SegmentRange calculates the start and end flow segment index for a file based on the file's start chunk index and file size.

func SegmentRoot 

func SegmentRoot(chunks []byte, emptyChunksPadded ...uint64) common.Hash

SegmentRoot return the merkle root of given chunks

Types

type DataInMemory 

type DataInMemory struct {
	// contains filtered or unexported fields
}

DataInMemory implement of IterableData, the underlying is memory data

func NewDataInMemory 

func NewDataInMemory(data []byte) (*DataInMemory, error)

NewDataInMemory creates DataInMemory from given data

func (*DataInMemory) NumChunks 

func (data *DataInMemory) NumChunks() uint64

func (*DataInMemory) NumSegments 

func (data *DataInMemory) NumSegments() uint64

func (*DataInMemory) Offset added in v0.6.2 

func (data *DataInMemory) Offset() int64

func (*DataInMemory) PaddedSize 

func (data *DataInMemory) PaddedSize() uint64

func (*DataInMemory) Read 

func (data *DataInMemory) Read(buf []byte, offset int64) (int, error)

func (*DataInMemory) Size 

func (data *DataInMemory) Size() int64

func (*DataInMemory) Split added in v0.6.2 

func (data *DataInMemory) Split(fragmentSize int64) []IterableData

type File 

type File struct {
	os.FileInfo
	// contains filtered or unexported fields
}

File implement of IterableData, the underlying is a file on disk

func Open 

func Open(name string) (*File, error)

Open create a File from a file on disk

func (*File) Close 

func (file *File) Close() error

func (*File) NumChunks 

func (file *File) NumChunks() uint64

func (*File) NumSegments 

func (file *File) NumSegments() uint64

func (*File) Offset added in v0.6.2 

func (file *File) Offset() int64

func (*File) PaddedSize 

func (file *File) PaddedSize() uint64

func (*File) Read 

func (file *File) Read(buf []byte, offset int64) (int, error)

func (*File) Size added in v0.6.2 

func (file *File) Size() int64

func (*File) Split added in v0.6.2 

func (file *File) Split(fragmentSize int64) []IterableData

type Flow 

type Flow struct {
	// contains filtered or unexported fields
}

func NewFlow 

func NewFlow(data IterableData, tags []byte, opts ...common.LogOption) *Flow

func (*Flow) CreateSubmission 

func (flow *Flow) CreateSubmission() (*contract.Submission, error)

type IterableData 

type IterableData interface {
	NumChunks() uint64
	NumSegments() uint64
	Offset() int64
	Size() int64
	PaddedSize() uint64
	Read(buf []byte, offset int64) (int, error)
	Split(fragmentSize int64) []IterableData
}

IterableData defines the data interface to upload to 0g storage network.

type Iterator 

type Iterator interface {
	Next() (bool, error)
	Current() []byte
}

type TreeBuilderInitializer 

type TreeBuilderInitializer struct {
	// contains filtered or unexported fields
}

func (*TreeBuilderInitializer) ParallelCollect 

func (t *TreeBuilderInitializer) ParallelCollect(result *parallel.Result) error

ParallelCollect implements parallel.Interface.

func (*TreeBuilderInitializer) ParallelDo 

func (t *TreeBuilderInitializer) ParallelDo(ctx context.Context, routine int, task int) (interface{}, error)

ParallelDo implements parallel.Interface.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.