storage

package
v1.9.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 10, 2025 License: MPL-2.0 Imports: 40 Imported by: 0

Documentation

Overview

Package storage implements storage backends for package torrent.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CreateNativeZeroLengthFile 

func CreateNativeZeroLengthFile(name string) error

A helper to create zero-length files which won't appear for file-orientated storage since no writes will ever occur to them (no torrent data is associated with a zero-length file). The caller should make sure the file name provided is safe/sanitized.

func GetPieceCompletionRange 

func GetPieceCompletionRange(pc PieceCompletion, ih infohash.T, begin, end int) iter.Seq[Completion]

Get piece completion as an iterator. Should be faster for long sequences of Gets. Uses optional interface PieceCompletionGetRanger if implemented.

func NewMMapWithCompletion 

func NewMMapWithCompletion(baseDir string, completion PieceCompletion) *mmapClientImpl

func NewSqlitePieceCompletion 

func NewSqlitePieceCompletion(dir string) (ret *sqlitePieceCompletion, err error)

func ToSafeFilePath 

func ToSafeFilePath(fileInfoComponents ...string) (string, error)

Combines file info path components, ensuring the result won't escape into parent directories.

Types

type ChunksReaderer 

type ChunksReaderer interface {
	ChunksReader(dir string) (PieceReader, error)
}

type Client 

type Client struct {
	// contains filtered or unexported fields
}

func NewClient 

func NewClient(cl ClientImpl) *Client

func (Client) OpenTorrent 

func (cl Client) OpenTorrent(
	ctx context.Context,
	info *metainfo.Info,
	infoHash metainfo.Hash,
) (*Torrent, error)

type ClientImpl 

type ClientImpl interface {
	OpenTorrent(ctx context.Context, info *metainfo.Info, infoHash metainfo.Hash) (TorrentImpl, error)
}

Represents data storage for an unspecified torrent.

func NewResourcePieces 

func NewResourcePieces(p PieceProvider) ClientImpl

func NewResourcePiecesOpts 

func NewResourcePiecesOpts(p PieceProvider, opts ResourcePiecesOpts) ClientImpl

type ClientImplCloser 

type ClientImplCloser interface {
	ClientImpl
	Close() error
}

func NewBoltDB 

func NewBoltDB(filePath string) ClientImplCloser

func NewFile 

func NewFile(baseDir string) ClientImplCloser

All Torrent data stored in this baseDir. The info names of each torrent are used as directories.

func NewFileByInfoHash 

func NewFileByInfoHash(baseDir string) ClientImplCloser

File storage with data partitioned by infohash.

func NewFileOpts 

func NewFileOpts(opts NewFileClientOpts) ClientImplCloser

NewFileOpts creates a new ClientImplCloser that stores files using the OS native filesystem.

func NewFileWithCompletion 

func NewFileWithCompletion(baseDir string, completion PieceCompletion) ClientImplCloser

func NewFileWithCustomPathMaker deprecated 

func NewFileWithCustomPathMaker(baseDir string, pathMaker func(baseDir string, info *metainfo.Info, infoHash metainfo.Hash) string) ClientImplCloser

Deprecated: Allows passing a function to determine the path for storing torrent data. The function is responsible for sanitizing the info if it uses some part of it (for example sanitizing info.Name).

func NewFileWithCustomPathMakerAndCompletion deprecated 

func NewFileWithCustomPathMakerAndCompletion(
	baseDir string,
	pathMaker TorrentDirFilePathMaker,
	completion PieceCompletion,
) ClientImplCloser

Deprecated: Allows passing custom PieceCompletion

func NewMMap 

func NewMMap(baseDir string) ClientImplCloser

TODO: Support all the same native filepath configuration that NewFileOpts provides.

type Completion 

type Completion struct {
	Err error
	// The state is known or cached.
	Ok bool
	// If Ok, whether the data is correct. TODO: Check all callsites test Ok first.
	Complete bool
}

Completion state of a piece.

type ConsecutiveChunkReader 

type ConsecutiveChunkReader interface {
	ReadConsecutiveChunks(prefix string) (io.ReadCloser, error)
}

type FileMapping 

type FileMapping = mmapSpan.Mmap

func WrapFileMapping 

func WrapFileMapping(region mmap.MMap, file *os.File) FileMapping

Combines a mmapped region and file into a storage Mmap abstraction, which handles closing the mmap file handle.

type FilePathMaker 

type FilePathMaker func(opts FilePathMakerOpts) string

Determines the filepath to be used for each file in a torrent.

type FilePathMakerOpts 

type FilePathMakerOpts struct {
	Info *metainfo.Info
	File *metainfo.FileInfo
}

Info passed to a FilePathMaker.

type MovePrefixer 

type MovePrefixer interface {
	MovePrefix(old, new string) error
}

type NewFileClientOpts 

type NewFileClientOpts struct {
	// The base directory for all downloads.
	ClientBaseDir   string
	FilePathMaker   FilePathMaker
	TorrentDirMaker TorrentDirFilePathMaker
	// If part files are enabled, this will default to inferring completion from file names at
	// startup, and keep the rest in memory.
	PieceCompletion PieceCompletion
	UsePartFiles    g.Option[bool]
	Logger          *slog.Logger
}

type Piece 

type Piece struct {
	PieceImpl
	// contains filtered or unexported fields
}

func (Piece) NewReader 

func (p Piece) NewReader() (PieceReader, error)

func (Piece) ReadAt 

func (p Piece) ReadAt(b []byte, off int64) (n int, err error)

If you're calling this you're probably doing something very inefficient. Consider WriteTo which handles data spread across multiple objects in storage.

func (Piece) WriteAt 

func (p Piece) WriteAt(b []byte, off int64) (n int, err error)

func (Piece) WriteTo 

func (p Piece) WriteTo(w io.Writer) (_ int64, err error)

Why do we have this wrapper? Well PieceImpl doesn't implement io.Reader, so we can't let io.Copy and friends check for io.WriterTo and fallback for us since they expect an io.Reader.

type PieceCompletion 

type PieceCompletion interface {
	PieceCompletionGetSetter
	// Piece completion is maintained between storage instances. We can use this to avoid flushing
	// pieces when they're marked complete if there's some other mechanism to ensure correctness.
	Close() error
}

Implementations track the completion of pieces. It must be concurrent-safe.

func NewBoltPieceCompletion 

func NewBoltPieceCompletion(dir string) (ret PieceCompletion, err error)

func NewDefaultPieceCompletionForDir 

func NewDefaultPieceCompletionForDir(dir string) (PieceCompletion, error)

sqlite is always the default when available.

func NewMapPieceCompletion 

func NewMapPieceCompletion() PieceCompletion

type PieceCompletionGetRanger 

type PieceCompletionGetRanger interface {
	GetRange(_ infohash.T, begin, end int) iter.Seq[Completion]
}

Optional interface with optimized Get for ranges. Use GetPieceCompletionRange wrapper to abstract over it not being implemented.

type PieceCompletionGetSetter 

type PieceCompletionGetSetter interface {
	// I think the extra error parameter is vestigial. Looks like you should put your error in
	// Completion.Err.
	Get(metainfo.PieceKey) (Completion, error)
	Set(_ metainfo.PieceKey, complete bool) error
}

type PieceCompletionPersistenter 

type PieceCompletionPersistenter interface {
	Persistent() bool
}

type PieceImpl 

type PieceImpl interface {
	// These interfaces are not as strict as normally required. They can
	// assume that the parameters are appropriate for the dimensions of the
	// piece.
	io.ReaderAt
	io.WriterAt
	// Called when the client believes the piece data will pass a hash check.
	// The storage can move or mark the piece data as read-only as it sees
	// fit.
	MarkComplete() error
	MarkNotComplete() error
	// Returns the state of a piece. Typically, this is implemented in some kind of storage to avoid
	// rehashing, and cheap checks are performed here. (The implementation maintains a cache in
	// Torrent).
	Completion() Completion
}

Interacts with torrent piece data. Optional interfaces to implement include:// 

	io.WriterTo, such as when a piece supports a more efficient way to write out incomplete chunks.
	SelfHashing, such as when a piece supports a more efficient way to hash its contents.
 	PieceReaderer when it has a stateful Reader interface that is more efficient.

type PieceProvider 

type PieceProvider interface {
	resource.Provider
}

type PieceReader 

type PieceReader interface {
	io.ReaderAt
	io.Closer
}

type PieceReaderer 

type PieceReaderer interface {
	NewReader() (PieceReader, error)
}

Piece supports dedicated reader.

type PrefixDeleter 

type PrefixDeleter interface {
	DeletePrefix(prefix string) error
}

type ResourcePiecesOpts 

type ResourcePiecesOpts struct {
	Capacity TorrentCapacity
	// After marking a piece complete, don't bother deleting its incomplete blobs.
	LeaveIncompleteChunks bool
	// Sized puts require being able to stream from a statement executed on another connection.
	// Without them, we buffer the entire read and then put that.
	NoSizedPuts bool
}

type SelfHashing 

type SelfHashing interface {
	SelfHash() (metainfo.Hash, error)
}

Allows a storage backend to override hashing (i.e. if it can do it more efficiently than the torrent client can).

type SizedPutter 

type SizedPutter interface {
	PutSized(io.Reader, int64) error
}

type Torrent 

type Torrent struct {
	TorrentImpl
}

func (*Torrent) Piece 

func (t *Torrent) Piece(p metainfo.Piece) Piece

Deprecated. Use PieceWithHash, as this doesn't work with pure v2 torrents.

func (*Torrent) PieceWithHash 

func (t *Torrent) PieceWithHash(p metainfo.Piece, pieceHash g.Option[[]byte]) Piece

type TorrentCapacity 

type TorrentCapacity *func() (cap int64, capped bool)

Returning a negative cap, can we indicate there's no specific cap? If this is not-nil we use it as a key into piece request order. The capped bool also needs to be true to be truly capped though.

type TorrentDirFilePathMaker 

type TorrentDirFilePathMaker func(baseDir string, info *metainfo.Info, infoHash metainfo.Hash) string

Determines the directory for a given torrent within a storage client.

type TorrentImpl 

type TorrentImpl struct {
	// v2 infos might not have the piece hash available even if we have the info. The
	// metainfo.Piece.Hash method was removed to enforce this.
	Piece func(p metainfo.Piece) PieceImpl
	// Preferred over PieceWithHash. Called with the piece hash if it's available.
	PieceWithHash func(p metainfo.Piece, pieceHash g.Option[[]byte]) PieceImpl
	Close         func() error
	// Storages that share the same space, will provide equal pointers. The function is called once
	// to determine the storage for torrents sharing the same function pointer, and mutated in
	// place.
	Capacity TorrentCapacity

	NewReader      func() TorrentReader
	NewPieceReader func(p Piece) PieceReader
}

Data storage bound to a torrent.

type TorrentReader 

type TorrentReader interface {
	io.ReaderAt
	io.Closer
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.