renter

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 27, 2019 License: MIT Imports: 27 Imported by: 0

Documentation

Overview

Package renter provides formats for contracts and files.

Index

Constants

View Source 
const (
	// ContractMagic is the magic string that identifies contract files.
	ContractMagic = "us-contract"

	// ContractHeaderSize is the size in bytes of the contract file header.
	// It is also the offset at which the contract revision data begins.
	ContractHeaderSize = 11 + 1 + 32 + 32 + 32

	// ContractSize is the maximum size in bytes of a contract file.
	ContractSize = 1024

	// ContractVersion is the current version of the contract file format. It is
	// incremented after each change to the format.
	ContractVersion uint8 = 3
)
View Source 
const (
	// MetaFileVersion is the current version of the metafile format. It is
	// incremented after each change to the format.
	MetaFileVersion = 2
)
View Source 
const SectorSliceSize = 64

SectorSliceSize is the encoded size of a SectorSlice.

Variables

View Source 
var ErrBadChecksum = errors.New("sector data failed checksum validation")

ErrBadChecksum indicates that a piece of sector data failed checksum validation.

Functions

func ConvertContract added in v0.2.0 

func ConvertContract(filename string) error

ConvertContract converts a contract file to the latest version of the contract format. The operation is atomic: if conversion fails, the old contract file will be unchanged.

func MetaFileCanDownload 

func MetaFileCanDownload(filename string) (bool, error)

MetaFileCanDownload reads a metafile without extracting it, reporting whether it can be downloaded.

func MetaFileFullyUploaded 

func MetaFileFullyUploaded(filename string) (bool, error)

MetaFileFullyUploaded reads a metafile without extracting it, reporting whether it has been fully uploaded.

func ReadContractRevision added in v0.2.0 

func ReadContractRevision(filename string) (proto.ContractRevision, error)

ReadContractRevision reads, decodes, and returns the ContractRevision of a contract file. The ContractRevision is not validated.

func SaveContract 

func SaveContract(contract proto.ContractRevision, key ed25519.PrivateKey, filename string) error

SaveContract creates a new contract file using the provided contract.

func SaveRenewedContract 

func SaveRenewedContract(oldContract *Contract, newContract proto.ContractRevision, filename string) error

SaveRenewedContract creates a new contract file using the provided contract.

Types

type Contract 

type Contract struct {
	proto.ContractRevision // for convenience
	// contains filtered or unexported fields
}

A Contract represents an open contract file. Contract files contain all the data necessary to revise a file contract.

func LoadContract 

func LoadContract(filename string) (_ *Contract, err error)

LoadContract loads a contract file into memory.

func (*Contract) Close 

func (c *Contract) Close() error

Close closes the contract file.

func (*Contract) HostKey added in v0.2.0 

func (c *Contract) HostKey() hostdb.HostPublicKey

HostKey returns the public key of the contract's host.

func (*Contract) Key added in v0.2.0 

func (c *Contract) Key() ed25519.PrivateKey

Key returns the renter's signing key.

func (*Contract) Revision added in v0.2.0 

func (c *Contract) Revision() proto.ContractRevision

Revision returns the latest revision of the file contract.

func (*Contract) SetRevision added in v0.2.0 

func (c *Contract) SetRevision(rev proto.ContractRevision) error

SetRevision sets the current revision of the file contract.

type ContractHeader 

type ContractHeader struct {
	// contains filtered or unexported fields
}

ContractHeader contains the data encoded within the first ContractHeaderSize bytes of the contract file.

func (*ContractHeader) Validate 

func (h *ContractHeader) Validate() error

Validate validates a ContractHeader, checking its magic bytes and version.

type ContractSet 

type ContractSet map[hostdb.HostPublicKey]*Contract

A ContractSet is a map of Contracts keyed by their host public key.

func LoadContracts 

func LoadContracts(dir string) (ContractSet, error)

LoadContracts loads a set of contract files stored in a directory and returns a map that keys the contracts by their host key. Files not ending in .contract are ignored. If multiple contracts have the same host key, LoadContracts returns an error.

func (ContractSet) Close 

func (set ContractSet) Close() error

Close closes all the Contracts in the set.

type ErasureCoder 

type ErasureCoder interface {
	// Encode encodes data into shards. The resulting shards do not constitute
	// a single matrix, but a series of matrices, each with a shard size of
	// merkletree.Segmentsize. The supplied shards must each have a capacity
	// of at least len(data)/m.
	Encode(data []byte, shards [][]byte)
	// Reconstruct recalculates any missing shards in the input. Missing
	// shards must have the same capacity as a normal shard, but a length of
	// zero.
	Reconstruct(shards [][]byte) error
	// Recover recalculates any missing data shards and writes them to w,
	// stopping after n bytes.
	Recover(w io.Writer, shards [][]byte, n int) error
}

An ErasureCoder encodes and decodes data to/from a set of shards. The encoding is done piecewise, such that every segment can be decoded individually.

func NewRSCode 

func NewRSCode(m, n int) ErasureCoder

NewRSCode returns an m-of-n ErasureCoder. It panics if m <= 0 or n < m.

type HostKeyResolver added in v0.2.0 

type HostKeyResolver interface {
	ResolveHostKey(pubkey hostdb.HostPublicKey) (modules.NetAddress, error)
}

A HostKeyResolver resolves a host's public key to the most recent NetAddress it announced on the blockchain.

type KeySeed added in v0.2.0 

type KeySeed [32]byte

A KeySeed derives subkeys and uses them to encrypt and decrypt messages.

func (KeySeed) MarshalJSON added in v0.2.0 

func (s KeySeed) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

func (*KeySeed) UnmarshalJSON added in v0.2.0 

func (s *KeySeed) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*KeySeed) XORKeyStream added in v0.2.0 

func (s *KeySeed) XORKeyStream(msg []byte, nonce []byte, startIndex uint64)

XORKeyStream xors msg with the keystream derived from s, using startIndex as the starting offset within the stream. The nonce must be 24 bytes.

type MetaFile 

type MetaFile struct {
	MetaIndex

	Workdir string
	// contains filtered or unexported fields
}

A MetaFile represents an extracted metafile archive.

func NewMetaFile 

func NewMetaFile(filename string, mode os.FileMode, size int64, contracts ContractSet, minShards int) (*MetaFile, error)

NewMetaFile creates a metafile using the specified contracts and erasure- coding parameters. The metafile is returned in extracted state, meaning a temporary directory will be created to hold the archive contents. This directory will be removed when Archive is called on the metafile.

func OpenMetaFile added in v0.2.0 

func OpenMetaFile(filename string) (_ *MetaFile, err error)

OpenMetaFile extracts an existing metafile archive. Like NewMetaFile, it creates a temporary directory to hold the extracted files.

func (*MetaFile) Archive 

func (m *MetaFile) Archive(filename string) error

Archive concatenates the metafile index with its referenced shard files and writes the resulting gzipped tar archive to filename.

func (*MetaFile) Close added in v0.2.0 

func (m *MetaFile) Close() error

Close re-archives the MetaFile and writes it to disk using the default filename, which is the same as the filename passed to NewMetaFile or OpenMetaFile.

func (*MetaFile) HostIndex 

func (m *MetaFile) HostIndex(hostKey hostdb.HostPublicKey) int

HostIndex returns the index of the shard that references data stored on the specified host. If m does not reference any data on the host, HostIndex returns -1.

func (*MetaFile) ReplaceHost 

func (m *MetaFile) ReplaceHost(oldHostKey, newHostKey hostdb.HostPublicKey) bool

ReplaceHost replaces a host within the metafile. The shard file of the replaced host is not immediately deleted, but it will not be included in the new archive when Close or Archive is called.

func (*MetaFile) ShardPath 

func (m *MetaFile) ShardPath(hostKey hostdb.HostPublicKey) string

ShardPath returns the canonical path on disk of a shard associated with the given hostKey.

type MetaIndex 

type MetaIndex struct {
	Version   int
	Filesize  int64       // original file size
	Mode      os.FileMode // mode bits
	ModTime   time.Time   // set when Archive is called
	MasterKey KeySeed     // seed from which shard encryption keys are derived
	MinShards int         // number of shards required to recover file
	Hosts     []hostdb.HostPublicKey
}

A MetaIndex contains the metadata that ties shards together into a single object with file semantics.

func ReadMetaFileContents 

func ReadMetaFileContents(filename string) (MetaIndex, [][]SectorSlice, error)

ReadMetaFileContents returns the metafile's index and shard slice data.

func ReadMetaIndex 

func ReadMetaIndex(filename string) (MetaIndex, error)

ReadMetaIndex returns the index of a metafile without extracting it.

func (*MetaIndex) ErasureCode 

func (m *MetaIndex) ErasureCode() ErasureCoder

ErasureCode returns the erasure code used to encode and decode the shards of m.

func (*MetaIndex) MaxChunkSize 

func (m *MetaIndex) MaxChunkSize() int64

MaxChunkSize returns the maximum amount of file data that can fit into a chunk. A chunk is a buffer of file data pre-erasure coding. When the chunk is encoded, it is split into len(m.Hosts) shards of equal size. Thus the MaxChunkSize is the size of such a buffer that results in shards equal to renterhost.SectorSize. MaxChunkSize is NOT guaranteed to match the actual chunk size used in the shard files of m.

func (*MetaIndex) MinChunkSize added in v0.2.0 

func (m *MetaIndex) MinChunkSize() int64

MinChunkSize is the size of the smallest possible chunk. When this chunk is erasure-encoded into shards, each shard will have a length of merkle.SegmentSize.

func (*MetaIndex) MinChunks 

func (m *MetaIndex) MinChunks() int64

MinChunks returns the minimum number of chunks required to fully upload the file. It assumes that each SectorSlice will reference a full sector (renterhost.SectorSize bytes).

type SectorBuilder 

type SectorBuilder struct {
	// contains filtered or unexported fields
}

A SectorBuilder facilitates the construction of sectors for later upload. SectorBuilders are particularly useful when packing data from multiple sources into a single sector. The zero value for a SectorBuilder is an empty sector.

func (*SectorBuilder) Append 

func (sb *SectorBuilder) Append(data []byte, key KeySeed)

Append appends data to the sector being constructed, encrypting it with the given key and chunkIndex. The data must be a multiple of merkle.SegmentSize.

Each call to Append creates a SectorSlice that is accessible via the Slices method. This SectorSlice reflects the length and checksum of the original (unencrypted) data.

Append panics if len(data) > sb.Remaining().

func (*SectorBuilder) Finish 

func (sb *SectorBuilder) Finish() *[renterhost.SectorSize]byte

Finish fills the remaining capacity of the sector with random bytes and returns it. The MerkleRoot fields of the SectorSlices tracked by sb are also set to the Merkle root of the resulting sector.

After calling Finish, Len returns renterhost.SectorSize and Remaining returns 0; no more data can be appended until Reset is called.

Finish returns a pointer to sb's internal buffer, so the standard warnings regarding such pointers apply. In particular, the pointer should not be retained after Reset is called.

func (*SectorBuilder) Len 

func (sb *SectorBuilder) Len() int

Len returns the number of bytes appended to the sector. Note that, due to padding, it is not guaranteed that Len equals the sum of the slices passed to Append.

func (*SectorBuilder) Remaining 

func (sb *SectorBuilder) Remaining() int

Remaining returns the number of bytes remaining in the sector. It is equivalent to renterhost.SectorSize - sb.Len().

func (*SectorBuilder) Reset 

func (sb *SectorBuilder) Reset()

Reset resets the SectorBuilder to its initial state.

Reset does not allocate a new sector buffer; since Finish returns a pointer to the buffer, this pointer should not be retained after Reset is called.

func (*SectorBuilder) Slices 

func (sb *SectorBuilder) Slices() []SectorSlice

Slices returns the SectorSlices present in the sector. One SectorSlice is returned for each call to Append since the last call to Reset. Slices should only be called after calling Finish; otherwise the MerkleRoot field of each SectorSlice will be unset.

type SectorSlice 

type SectorSlice struct {
	MerkleRoot   crypto.Hash
	SegmentIndex uint32
	NumSegments  uint32
	Nonce        [24]byte
}

A SectorSlice is the unit element of a shard file. Each SectorSlice uniquely identifies a contiguous slice of data stored on a host.

func ReadShard 

func ReadShard(filename string) ([]SectorSlice, error)

ReadShard loads the slices of a shard file into memory.

type Shard 

type Shard struct {
	// contains filtered or unexported fields
}

A Shard is a shard file open for writing.

func OpenShard 

func OpenShard(filename string) (*Shard, error)

OpenShard opens a shard file for writing, creating it if necessary. If the file's size is not a multiple of SectorSliceSize, it is truncated accordingly.

func (*Shard) Close 

func (s *Shard) Close() error

Close closes the shard file.

func (*Shard) WriteSlice 

func (s *Shard) WriteSlice(slice SectorSlice, index int64) error

WriteSlice writes slice at the specified index, growing the underlying file as necessary.

type ShardDownloader 

type ShardDownloader struct {
	Downloader *proto.Session
	Slices     []SectorSlice
	Key        KeySeed
	// contains filtered or unexported fields
}

A ShardDownloader wraps a proto.Session to provide SectorSlice-based data retrieval, transparently decrypting and validating the received data.

func NewShardDownloader 

func NewShardDownloader(m *MetaFile, contract *Contract, hkr HostKeyResolver) (*ShardDownloader, error)

NewShardDownloader connects to a host and returns a ShardDownloader capable of downloading the SectorSlices of m.

func (*ShardDownloader) Close 

func (d *ShardDownloader) Close() error

Close closes the connection to the host.

func (*ShardDownloader) CopySection added in v0.2.0 

func (d *ShardDownloader) CopySection(w io.Writer, offset, length int64) error

CopySection downloads the requested section of the Shard, decrypts it, and writes it to w.

func (*ShardDownloader) DownloadAndDecrypt 

func (d *ShardDownloader) DownloadAndDecrypt(chunkIndex int64) ([]byte, error)

DownloadAndDecrypt downloads the SectorSlice associated with chunkIndex. The data is decrypted and validated before it is returned. The returned slice is only valid until the next call to DownloadAndDecrypt.

func (*ShardDownloader) HostKey 

func (d *ShardDownloader) HostKey() hostdb.HostPublicKey

HostKey returns the public key of the host.

type ShardUploader 

type ShardUploader struct {
	Uploader *proto.Session
	Shard    *Shard
	Key      KeySeed
	Sector   SectorBuilder
}

A ShardUploader wraps a proto.Session to provide SectorSlice-based data storage, transparently encrypting and checksumming all data before transferring it to the host.

func NewShardUploader 

func NewShardUploader(m *MetaFile, contract *Contract, hkr HostKeyResolver, currentHeight types.BlockHeight) (*ShardUploader, error)

NewShardUploader connects to a host and returns a ShardUploader capable of uploading m's data and writing to one of m's Shard files.

func (*ShardUploader) Close 

func (u *ShardUploader) Close() error

Close closes the connection to the host and the Shard file.

func (*ShardUploader) EncryptAndUpload 

func (u *ShardUploader) EncryptAndUpload(data []byte, chunkIndex int64) (SectorSlice, error)

EncryptAndUpload uploads the data associated with chunkIndex, creating a SectorSlice. The data is encrypted and padded to renterhost.SectorSize before it is uploaded. The resulting SectorSlice is written to u.Shard.

func (*ShardUploader) HostKey 

func (u *ShardUploader) HostKey() hostdb.HostPublicKey

HostKey returns the public key of the host.

func (*ShardUploader) Upload added in v0.2.0 

func (u *ShardUploader) Upload(chunkIndex int64) error

Upload uploads u.Sector, writing the resulting SectorSlice(s) to u.Shard, starting at offset chunkIndex. Upload does not call Reset on u.Sector.

Source Files

View all Source files

Directories

Path Synopsis
Package proto implements the renter side of the Sia renter-host protocol.
 Package proto implements the renter side of the Sia renter-host protocol.
Package renterutil provides convenience functions for common renter actions.
 Package renterutil provides convenience functions for common renter actions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.