unixfs

package
Version: v0.0.0-...-ce94876 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 1, 2019 License: MIT, MIT Imports: 5 Imported by: 0

README

go-unixfs

Coverage Status Travis CI

go-unixfs implements unix-like filesystem utilities on top of an ipld merkledag

Table of Contents

Package Directory

This package contains many subpackages, each of which can be very large on its own.

Top Level

The top level unixfs package defines the unixfs format datastructures, and some helper methods around it.

importers

The importer subpackage is what you'll use when you want to turn a normal file into a unixfs file.

io

The io subpackage provides helpers for reading files and manipulating directories. The DagReader takes a reference to a unixfs file and returns a file handle that can be read from and seeked through. The Directory interface allows you to easily read items in a directory, add items to a directory, and do lookups.

mod

The mod subpackage implements a DagModifier type that can be used to write to an existing unixfs file, or create a new one. The logic for this is significantly more complicated than for the dagreader, so its a separate type. (TODO: maybe it still belongs in the io subpackage though?)

hamt

The hamt subpackage implements a CHAMP hamt that is used in unixfs directory sharding.

archive

The archive subpackage implements a tar importer and exporter. The objects created here are not officially unixfs, but in the future, this may be integrated more directly.

test

The test subpackage provides several utilities to make testing unixfs related things easier.

Install

go get github.com/ipfs/go-unixfs

Contribute

PRs are welcome!

Small note: If editing the Readme, please conform to the standard-readme specification.

License

MIT © Juan Batiz-Benet

Documentation

Overview

Package unixfs implements a data format for files in the IPFS filesystem It is not the only format in ipfs, but it is the one that the filesystem assumes

Index

Constants

View Source
const (
	TRaw       = pb.Data_Raw
	TFile      = pb.Data_File
	TDirectory = pb.Data_Directory
	TMetadata  = pb.Data_Metadata
	TSymlink   = pb.Data_Symlink
	THAMTShard = pb.Data_HAMTShard
)

Shorthands for protobuffer types

Variables

View Source
var (
	ErrMalformedFileFormat = errors.New("malformed data in file format")
	ErrUnrecognizedType    = errors.New("unrecognized node type")
)

Common errors

Functions

func BytesForMetadata

func BytesForMetadata(m *Metadata) ([]byte, error)

BytesForMetadata wraps the given Metadata as a profobuf message of Data type, setting the DataType to Metadata. The wrapped bytes are itself the result of calling m.Bytes().

func DataSize

func DataSize(data []byte) (uint64, error)

DataSize returns the size of the contents in protobuf wrapped slice. For raw data it simply provides the length of it. For Data_Files, it will return the associated filesize. Note that Data_Directories will return an error.

func EmptyDirNode

func EmptyDirNode() *dag.ProtoNode

EmptyDirNode creates an empty folder Protonode.

func FilePBData

func FilePBData(data []byte, totalsize uint64) []byte

FilePBData creates a protobuf File with the given byte slice and returns the marshaled protobuf bytes representing it.

func FolderPBData

func FolderPBData() []byte

FolderPBData returns Bytes that represent a Directory.

func FromBytes deprecated

func FromBytes(data []byte) (*pb.Data, error)

FromBytes unmarshals a byte slice as protobuf Data. Deprecated: Use `FSNodeFromBytes` instead to avoid direct manipulation of `pb.Data`.

func HAMTShardData

func HAMTShardData(data []byte, fanout uint64, hashType uint64) ([]byte, error)

HAMTShardData return a `Data_HAMTShard` protobuf message

func SymlinkData

func SymlinkData(path string) ([]byte, error)

SymlinkData returns a `Data_Symlink` protobuf message for the path you specify.

func UnwrapData

func UnwrapData(data []byte) ([]byte, error)

UnwrapData unmarshals a protobuf messages and returns the contents.

func WrapData

func WrapData(b []byte) []byte

WrapData marshals raw bytes into a `Data_Raw` type protobuf message.

Types

type FSNode

type FSNode struct {
	// contains filtered or unexported fields
}

An FSNode represents a filesystem object using the UnixFS specification.

The `NewFSNode` constructor should be used instead of just calling `new(FSNode)` to guarantee that the required (`Type` and `Filesize`) fields in the `format` structure are initialized before marshaling (in `GetBytes()`).

func FSNodeFromBytes

func FSNodeFromBytes(b []byte) (*FSNode, error)

FSNodeFromBytes unmarshal a protobuf message onto an FSNode.

func NewFSNode

func NewFSNode(dataType pb.Data_DataType) *FSNode

NewFSNode creates a new FSNode structure with the given `dataType`.

It initializes the (required) `Type` field (that doesn't have a `Set()` accessor so it must be specified at creation), otherwise the `Marshal()` method in `GetBytes()` would fail (`required field "Type" not set`).

It also initializes the `Filesize` pointer field to ensure its value is never nil before marshaling, this is not a required field but it is done to be backwards compatible with previous `go-ipfs` versions hash. (If it wasn't initialized there could be cases where `Filesize` could have been left at nil, when the `FSNode` was created but no data or child nodes were set to adjust it, as is the case in `NewLeaf()`.)

func (*FSNode) AddBlockSize

func (n *FSNode) AddBlockSize(s uint64)

AddBlockSize adds the size of the next child block of this node

func (*FSNode) BlockSize

func (n *FSNode) BlockSize(i int) uint64

BlockSize returns the block size indexed by `i`. TODO: Evaluate if this function should be bounds checking.

func (*FSNode) BlockSizes

func (n *FSNode) BlockSizes() []uint64

BlockSizes gets blocksizes of format

func (*FSNode) Data

func (n *FSNode) Data() []byte

Data retrieves the `Data` field from the internal `format`.

func (*FSNode) Fanout

func (n *FSNode) Fanout() uint64

Fanout gets fanout of format

func (*FSNode) FileSize

func (n *FSNode) FileSize() uint64

FileSize returns the total size of this tree. That is, the size of the data in this node plus the size of all its children.

func (*FSNode) GetBytes

func (n *FSNode) GetBytes() ([]byte, error)

GetBytes marshals this node as a protobuf message.

func (*FSNode) HashType

func (n *FSNode) HashType() uint64

HashType gets hash type of format

func (*FSNode) IsDir

func (n *FSNode) IsDir() bool

IsDir checks whether the node represents a directory

func (*FSNode) NumChildren

func (n *FSNode) NumChildren() int

NumChildren returns the number of child blocks of this node

func (*FSNode) RemoveAllBlockSizes

func (n *FSNode) RemoveAllBlockSizes()

RemoveAllBlockSizes removes all the child block sizes of this node.

func (*FSNode) RemoveBlockSize

func (n *FSNode) RemoveBlockSize(i int)

RemoveBlockSize removes the given child block's size.

func (*FSNode) SetData

func (n *FSNode) SetData(newData []byte)

SetData sets the `Data` field from the internal `format` updating its `Filesize`.

func (*FSNode) Type

func (n *FSNode) Type() pb.Data_DataType

Type retrieves the `Type` field from the internal `format`.

func (*FSNode) UpdateFilesize

func (n *FSNode) UpdateFilesize(filesizeDiff int64)

UpdateFilesize updates the `Filesize` field from the internal `format` by a signed difference (`filesizeDiff`). TODO: Add assert to check for `Filesize` > 0?

type LinkResult

type LinkResult struct {
	Link *ipld.Link
	Err  error
}

A LinkResult for any parallel enumeration of links TODO: Should this live in go-ipld-format?

type Metadata

type Metadata struct {
	MimeType string
	Size     uint64
}

Metadata is used to store additional FSNode information.

func MetadataFromBytes

func MetadataFromBytes(b []byte) (*Metadata, error)

MetadataFromBytes Unmarshals a protobuf Data message into Metadata. The provided slice should have been encoded with BytesForMetadata().

func (*Metadata) Bytes

func (m *Metadata) Bytes() ([]byte, error)

Bytes marshals Metadata as a protobuf message of Metadata type.

Source Files

Directories

Path Synopsis
Package archive provides utilities to archive and compress a [Unixfs] DAG.
Package archive provides utilities to archive and compress a [Unixfs] DAG.
tar
Package tar provides functionality to write a unixfs merkledag as a tar archive.
Package tar provides functionality to write a unixfs merkledag as a tar archive.
Package hamt implements a Hash Array Mapped Trie over ipfs merkledag nodes.
Package hamt implements a Hash Array Mapped Trie over ipfs merkledag nodes.
Package importer implements utilities used to create IPFS DAGs from files and readers.
Package importer implements utilities used to create IPFS DAGs from files and readers.
balanced
Package balanced provides methods to build balanced DAGs, which are generalistic DAGs in which all leaves (nodes representing chunks of data) are at the same distance from the root.
Package balanced provides methods to build balanced DAGs, which are generalistic DAGs in which all leaves (nodes representing chunks of data) are at the same distance from the root.
trickle
Package trickle allows to build trickle DAGs.
Package trickle allows to build trickle DAGs.
Package io implements convenience objects for working with the ipfs unixfs data format.
Package io implements convenience objects for working with the ipfs unixfs data format.
Package mod provides DAG modification utilities to, for example, insert additional nodes in a unixfs DAG or truncate them.
Package mod provides DAG modification utilities to, for example, insert additional nodes in a unixfs DAG or truncate them.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL