README
Documentation
¶
Overview ¶
cafs is a "content-addressed-file-systen", which is a generalized interface for working with content-addressed filestores. real-on-the-real, this is a wrapper for IPFS. It looks a lot like the ipfs datastore interface, except the datastore itself determines keys.
This file is pulled from github.com/ipfs/go-ipfs/commands/files It's hoisted here to generalize for all cafs implementations.
Index ¶
- Variables
- func Walk(root File, depth int, visit func(f File, depth int) error) (err error)
- type AddedFile
- type Adder
- type Fetcher
- type File
- type FileInfo
- type Filestore
- type MapStore
- func (m *MapStore) AddConnection(other *MapStore)
- func (m MapStore) Delete(key string) error
- func (m *MapStore) Fetch(source Source, key string) (File, error)
- func (m *MapStore) Get(key string) (File, error)
- func (m MapStore) Has(key string) (exists bool, err error)
- func (m MapStore) NewAdder(pin, wrap bool) (Adder, error)
- func (m MapStore) PathPrefix() string
- func (m *MapStore) Pin(key string, recursive bool) error
- func (m MapStore) Print() (string, error)
- func (m *MapStore) Put(file File, pin bool) (key string, err error)
- func (m *MapStore) Unpin(key string, recursive bool) error
- type Memdir
- func (d *Memdir) AddChildren(fs ...File)
- func (d *Memdir) ChildDir(dirname string) *Memdir
- func (Memdir) Close() error
- func (m Memdir) FileName() string
- func (m Memdir) FullPath() string
- func (Memdir) IsDirectory() bool
- func (d *Memdir) MakeDirP(f File) *Memdir
- func (d *Memdir) NextFile() (File, error)
- func (Memdir) Read([]byte) (int, error)
- func (d *Memdir) SetPath(path string)
- type Memfile
- type PathSetter
- type PeekFile
- type Pinner
- type SizeFile
- type Source
- type StatFile
Constants ¶
This section is empty.
Variables ¶
var ( ErrNotDirectory = errors.New("Couldn't call NextFile(), this isn't a directory") ErrNotReader = errors.New("This file is a directory, can't use Reader functions") )
var ( // ErrNotFound is the canonical error for not finding a value ErrNotFound = errors.New("cafs: path not found") )
var (
// SourceAny specifies that content can come from anywhere
SourceAny = source("any")
)
Functions ¶
Types ¶
type AddedFile ¶
AddedFile reports on the results of adding a file to the store TODO - add filepath to this struct
type Adder ¶
type Adder interface {
// AddFile adds a file or directory of files to the store
// this function will return immideately, consumers should read
// from the Added() channel to see the results of file addition.
AddFile(File) error
// Added gives a channel to read added files from.
Added() chan AddedFile
// In IPFS land close calls adder.Finalize() and adder.PinRoot()
// (files will only be pinned if the pin flag was set on NewAdder)
// Close will close the underlying
Close() error
}
Adder is the interface for adding files to a Filestore. The addition process is parallelized. Implementers must make all required AddFile calls, then call Close to finalize the addition process. Progress can be monitored through the Added() channel
type Fetcher ¶
type Fetcher interface {
// Fetch gets a file from a source
Fetch(source Source, key string) (File, error)
}
Fetcher is the interface for getting files from a remote source filestores can opt into the fetcher interface
type File ¶
type File interface {
// Files implement ReadCloser, but can only be read from or closed if
// they are not directories
io.ReadCloser
// FileName returns a filename associated with this file
FileName() string
// FullPath returns the full path used when adding this file
FullPath() string
// IsDirectory returns true if the File is a directory (and therefore
// supports calling `NextFile`) and false if the File is a normal file
// (and therefor supports calling `Read` and `Close`)
IsDirectory() bool
// NextFile returns the next child file available (if the File is a
// directory). It will return (nil, io.EOF) if no more files are
// available. If the file is a regular file (not a directory), NextFile
// will return a non-nil error.
NextFile() (File, error)
}
File is an interface that provides functionality for handling files/directories as values that can be supplied to commands. For directories, child files are accessed serially by calling `NextFile()`.
type Filestore ¶
type Filestore interface {
// Put places a file or a directory in the store.
// The most notable difference from a standard file store is the store itself determines
// the resulting key (google "content addressing" for more info ;)
// keys returned by put must be prefixed with the PathPrefix,
// eg. /ipfs/QmZ3KfGaSrb3cnTriJbddCzG7hwQi2j6km7Xe7hVpnsW5S
Put(file File, pin bool) (key string, err error)
// Get retrieves the object `value` named by `key`.
// Get will return ErrNotFound if the key is not mapped to a value.
Get(key string) (file File, err error)
// Has returns whether the `key` is mapped to a `value`.
// In some contexts, it may be much cheaper only to check for existence of
// a value, rather than retrieving the value itself. (e.g. HTTP HEAD).
// The default implementation is found in `GetBackedHas`.
Has(key string) (exists bool, err error)
// Delete removes the value for given `key`.
Delete(key string) error
// NewAdder allocates an Adder instance for adding files to the filestore
// Adder gives a higher degree of control over the file adding process at the
// cost of being harder to work with.
// "pin" is a flag for recursively pinning this object
// "wrap" sets weather the top level should be wrapped in a directory
// expect this to change to something like:
// NewAdder(opt map[string]interface{}) (Adder, error)
NewAdder(pin, wrap bool) (Adder, error)
// PathPrefix is a top-level identifier to distinguish between filestores,
// for exmple: the "ipfs" in /ipfs/QmZ3KfGaSrb3cnTriJbddCzG7hwQi2j6km7Xe7hVpnsW5S
// a Filestore implementation should always return the same
PathPrefix() string
}
Filestore is an interface for working with a content-addressed file system. This interface is under active development, expect it to change lots. It's currently form-fitting around IPFS (ipfs.io), with far-off plans to generalize toward compatibility with git (git-scm.com), then maybe other stuff, who knows.
type MapStore ¶
MapStore implements Filestore in-memory as a map
An example pulled from tests will create a tree of "cafs" with directories & cafs, with paths properly set: NewMemdir("/a",
NewMemfileBytes("a.txt", []byte("foo")),
NewMemfileBytes("b.txt", []byte("bar")),
NewMemdir("/c",
NewMemfileBytes("d.txt", []byte("baz")),
NewMemdir("/e",
NewMemfileBytes("f.txt", []byte("bat")),
),
),
) File is an interface that provides functionality for handling cafs/directories as values that can be supplied to commands.
This is pretty close to things that already exist in ipfs and might not be necessary in most situations, but provides a sensible degree of modularity for our purposes: * memdir: github.com/ipfs/go-ipfs/commands/SerialFile * memfs: github.com/ipfs/go-ipfs/commands/ReaderFile
Network simulates IPFS-like behavior, where nodes can connect to each other to retrieve data from other machines
func (*MapStore) AddConnection ¶
AddConnection sets up pointers from this MapStore to that, and vice versa.
func (MapStore) PathPrefix ¶
PathPrefix returns the prefix on paths in the store
type Memdir ¶
type Memdir struct {
// contains filtered or unexported fields
}
Memdir is an in-memory directory Currently it only supports either Memfile & Memdir as links
func (*Memdir) AddChildren ¶
AddChildren allows any sort of file to be added, but only implementations that implement the PathSetter interface will have properly configured paths.
func (Memdir) IsDirectory ¶
type Memfile ¶
type Memfile struct {
// contains filtered or unexported fields
}
Memfile is an in-memory file
func NewMemfileBytes ¶
NewMemfileBytes creates a file from a byte slice
func NewMemfileReader ¶
NewMemfileBytes creates a file from an io.Reader
func (Memfile) IsDirectory ¶
type PathSetter ¶
type PathSetter interface {
SetPath(path string)
}
PathSetter adds the capacity to modify a path property
type Pinner ¶
type Pinner interface {
Pin(key string, recursive bool) error
Unpin(key string, recursive bool) error
}
Pinner interface for content stores that support the concept of pinning (originated by IPFS). Necessarily asynchronous, with no stateful guarantees, currently not testable.
Source Files
Directories