mfs

README

go-mfs

go-mfs implements an in-memory model of a mutable IPFS filesystem.

Table of Contents

• Install
• Usage
• Contribute
• License

Install

go-mfs works like a regular Go module:

> go get github.com/ipfs/go-mfs

It uses Gx to manage dependencies. You can use make all to build it with the gx dependencies.

Usage

import "github.com/ipfs/go-mfs"

Check the GoDoc documentation

Contribute

PRs accepted.

Small note: If editing the README, please conform to the standard-readme specification.

License

MIT © Protocol Labs, Inc.

Documentation

Overview

package mfs implements an in memory model of a mutable IPFS filesystem.

It consists of four main structs:

1. The Filesystem The filesystem serves as a container and entry point for various mfs filesystems
2. Root Root represents an individual filesystem mounted within the mfs system as a whole
3. Directories
4. Files

Index

• Constants
• Variables
• func FlushPath(rt *Root, pth string) error
• func IsDir(fsn FSNode) bool
• func IsFile(fsn FSNode) bool
• func Mkdir(r *Root, pth string, opts MkdirOpts) error
• func Mv(r *Root, src, dst string) error
• func PutNode(r *Root, path string, nd ipld.Node) error
• type Directory
  • func NewDirectory(ctx context.Context, name string, node ipld.Node, parent childCloser, ...) (*Directory, error)
  • func (d *Directory) AddChild(name string, nd ipld.Node) error
  • func (d *Directory) AddUnixFSChild(name string, node ipld.Node) error
  • func (d *Directory) Child(name string) (FSNode, error)
  • func (d *Directory) Flush() error
  • func (d *Directory) ForEachEntry(ctx context.Context, f func(NodeListing) error) error
  • func (d *Directory) GetCidBuilder() cid.Builder
  • func (d *Directory) GetNode() (ipld.Node, error)
  • func (d *Directory) List(ctx context.Context) ([]NodeListing, error)
  • func (d *Directory) ListNames(ctx context.Context) ([]string, error)
  • func (d *Directory) Mkdir(name string) (*Directory, error)
  • func (d *Directory) Path() string
  • func (d *Directory) SetCidBuilder(b cid.Builder)
  • func (d *Directory) Type() NodeType
  • func (d *Directory) Uncache(name string)
  • func (d *Directory) Unlink(name string) error
• type FSNode
  • func DirLookup(d *Directory, pth string) (FSNode, error)
  • func Lookup(r *Root, path string) (FSNode, error)
• type File
  • func NewFile(name string, node ipld.Node, parent childCloser, dserv ipld.DAGService) (*File, error)
  • func (fi *File) Flush() error
  • func (fi *File) GetNode() (ipld.Node, error)
  • func (fi *File) Open(flags int, sync bool) (FileDescriptor, error)
  • func (fi *File) Size() (int64, error)
  • func (fi *File) Sync() error
  • func (fi *File) Type() NodeType
• type FileDescriptor
• type MkdirOpts
• type NodeListing
• type NodeType
• type PubFunc
• type Republisher
  • func NewRepublisher(ctx context.Context, pf PubFunc, tshort, tlong time.Duration) *Republisher
  • func (p *Republisher) Close() error
  • func (np *Republisher) Run()
  • func (np *Republisher) Update(c cid.Cid)
  • func (p *Republisher) WaitPub()
• type Root
  • func NewRoot(parent context.Context, ds ipld.DAGService, node *dag.ProtoNode, pf PubFunc) (*Root, error)
  • func (kr *Root) Close() error
  • func (kr *Root) Flush() error
  • func (kr *Root) FlushMemFree(ctx context.Context) error
  • func (kr *Root) GetDirectory() *Directory

Constants

const (
	OpenReadOnly = iota
	OpenWriteOnly
	OpenReadWrite
)

Variables

var ErrDirExists = errors.New("directory already has entry by that name")

var ErrInvalidChild = errors.New("invalid child node")

var ErrIsDirectory = errors.New("error: is a directory")

var ErrNotExist = errors.New("no such rootfs")

var ErrNotYetImplemented = errors.New("not yet implemented")

Functions

func FlushPath

func FlushPath(rt *Root, pth string) error

func IsDir

func IsDir(fsn FSNode) bool

IsDir checks whether the FSNode is dir type

func IsFile

func IsFile(fsn FSNode) bool

IsFile checks whether the FSNode is file type

func Mkdir

func Mkdir(r *Root, pth string, opts MkdirOpts) error

Mkdir creates a directory at 'path' under the directory 'd', creating intermediary directories as needed if 'mkparents' is set to true

func Mv

func Mv(r *Root, src, dst string) error

Mv moves the file or directory at 'src' to 'dst'

func PutNode

func PutNode(r *Root, path string, nd ipld.Node) error

PutNode inserts 'nd' at 'path' in the given mfs

Types

type Directory

type Directory struct {
	// contains filtered or unexported fields
}

func NewDirectory

func NewDirectory(ctx context.Context, name string, node ipld.Node, parent childCloser, dserv ipld.DAGService) (*Directory, error)

NewDirectory constructs a new MFS directory.

You probably don't want to call this directly. Instead, construct a new root using NewRoot.

func (*Directory) AddChild

func (d *Directory) AddChild(name string, nd ipld.Node) error

AddChild adds the node 'nd' under this directory giving it the name 'name'

func (*Directory) AddUnixFSChild

func (d *Directory) AddUnixFSChild(name string, node ipld.Node) error

AddUnixFSChild adds a child to the inner UnixFS directory and transitions to a HAMT implementation if needed.

func (*Directory) Child

func (d *Directory) Child(name string) (FSNode, error)

Child returns the child of this directory by the given name

func (*Directory) Flush

func (d *Directory) Flush() error

func (*Directory) ForEachEntry

func (d *Directory) ForEachEntry(ctx context.Context, f func(NodeListing) error) error

func (*Directory) GetCidBuilder

func (d *Directory) GetCidBuilder() cid.Builder

GetCidBuilder gets the CID builder of the root node

func (*Directory) GetNode

func (d *Directory) GetNode() (ipld.Node, error)

func (*Directory) List

func (d *Directory) List(ctx context.Context) ([]NodeListing, error)

func (*Directory) ListNames

func (d *Directory) ListNames(ctx context.Context) ([]string, error)

func (*Directory) Mkdir

func (d *Directory) Mkdir(name string) (*Directory, error)

func (*Directory) Path

func (d *Directory) Path() string

func (*Directory) SetCidBuilder

func (d *Directory) SetCidBuilder(b cid.Builder)

SetCidBuilder sets the CID builder

func (*Directory) Type

func (d *Directory) Type() NodeType

func (*Directory) Uncache

func (d *Directory) Uncache(name string)

func (*Directory) Unlink

func (d *Directory) Unlink(name string) error

type FSNode

type FSNode interface {
	GetNode() (ipld.Node, error)
	Flush() error
	Type() NodeType
}

FSNode represents any node (directory, root, or file) in the mfs filesystem.

func DirLookup

func DirLookup(d *Directory, pth string) (FSNode, error)

DirLookup will look up a file or directory at the given path under the directory 'd'

func Lookup

func Lookup(r *Root, path string) (FSNode, error)

Lookup extracts the root directory and performs a lookup under it. TODO: Now that the root is always a directory, can this function be collapsed with `DirLookup`? Or at least be made a method of `Root`?

type File

type File struct {
	RawLeaves bool
	// contains filtered or unexported fields
}

func NewFile

func NewFile(name string, node ipld.Node, parent childCloser, dserv ipld.DAGService) (*File, error)

NewFile returns a NewFile object with the given parameters. If the Cid version is non-zero RawLeaves will be enabled.

func (*File) Flush

func (fi *File) Flush() error

func (*File) GetNode

func (fi *File) GetNode() (ipld.Node, error)

GetNode returns the dag node associated with this file

func (*File) Open

func (fi *File) Open(flags int, sync bool) (FileDescriptor, error)

func (*File) Size

func (fi *File) Size() (int64, error)

Size returns the size of this file

func (*File) Sync

func (fi *File) Sync() error

func (*File) Type

func (fi *File) Type() NodeType

Type returns the type FSNode this is

type FileDescriptor

type FileDescriptor interface {
	io.Reader
	CtxReadFull(context.Context, []byte) (int, error)

	io.Writer
	io.WriterAt

	io.Closer
	io.Seeker

	Truncate(int64) error
	Size() (int64, error)
	Sync() error
	Flush() error
}

type MkdirOpts

type MkdirOpts struct {
	Mkparents  bool
	Flush      bool
	CidBuilder cid.Builder
}

MkdirOpts is used by Mkdir

type NodeListing

type NodeListing struct {
	Name string
	Type int
	Size int64
	Hash string
}

type NodeType

type NodeType int

const (
	TFile NodeType = iota
	TDir
)

type PubFunc

type PubFunc func(context.Context, cid.Cid) error

PubFunc is the function used by the `publish()` method.

type Republisher

type Republisher struct {
	TimeoutLong  time.Duration
	TimeoutShort time.Duration
	Publish      chan struct{}
	// contains filtered or unexported fields
}

Republisher manages when to publish a given entry.

func NewRepublisher

func NewRepublisher(ctx context.Context, pf PubFunc, tshort, tlong time.Duration) *Republisher

NewRepublisher creates a new Republisher object to republish the given root using the given short and long time intervals.

func (*Republisher) Close

func (p *Republisher) Close() error

func (*Republisher) Run

func (np *Republisher) Run()

Run is the main republisher loop.

func (*Republisher) Update

func (np *Republisher) Update(c cid.Cid)

Touch signals that an update has occurred since the last publish. Multiple consecutive touches may extend the time period before the next Publish occurs in order to more efficiently batch updates.

func (*Republisher) WaitPub

func (p *Republisher) WaitPub()

WaitPub Returns immediately if `lastpub` value is consistent with the current value `val`, else will block until `val` has been published.

type Root

type Root struct {
	// contains filtered or unexported fields
}

Root represents the root of a filesystem tree.

func NewRoot

func NewRoot(parent context.Context, ds ipld.DAGService, node *dag.ProtoNode, pf PubFunc) (*Root, error)

NewRoot creates a new Root and starts up a republisher routine for it.

func (*Root) Close

func (kr *Root) Close() error

func (*Root) Flush

func (kr *Root) Flush() error

Flush signals that an update has occurred since the last publish, and updates the Root republisher.

func (*Root) FlushMemFree

func (kr *Root) FlushMemFree(ctx context.Context) error

FlushMemFree flushes the root directory and then uncaches all of its links. This has the effect of clearing out potentially stale references and allows them to be garbage collected. CAUTION: Take care not to ever call this while holding a reference to any child directories. Those directories will be bad references and using them may have unintended racy side effects. A better implemented mfs system (one that does smarter internal caching and refcounting) shouldnt need this method.

func (*Root) GetDirectory

func (kr *Root) GetDirectory() *Directory

GetDirectory returns the root directory.