fs_cablobstore

package
v0.0.0-...-ba1c585 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 29, 2017 License: BSD-3-Clause Imports: 22 Imported by: 0

Documentation

Overview

Package fs_cablobstore implements a content addressable blob store on top of a file system. It assumes that either os.Link() or os.Rename() is available.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BlobReader 

type BlobReader struct {
	// contains filtered or unexported fields
}

A BlobReader allows a blob to be read using the standard ReadAt(), Read(), and Seek() calls. A BlobReader can be created with NewBlobReader(), and should be closed with the Close() method to avoid leaking file handles.

func (*BlobReader) Close 

func (br *BlobReader) Close() error

Close() indicates that the client will perform no further operations on *br. It closes any open file handles within a BlobReader.

func (*BlobReader) Hash 

func (br *BlobReader) Hash() []byte

Hash() returns *br's hash. It may be nil if the blob is not finalized.

func (*BlobReader) IsFinalized 

func (br *BlobReader) IsFinalized() bool

IsFinalized() returns whether *br has been finalized.

func (*BlobReader) Name 

func (br *BlobReader) Name() string

Name() returns *br's name.

func (*BlobReader) Read 

func (br *BlobReader) Read(b []byte) (n int, err error)

Read() fills b[] with up to len(b) bytes of data starting at the current seek position of *br within the blob that *br indicates, and then both returns the number of bytes read and advances *br's seek position by that amount.

func (*BlobReader) ReadAt 

func (br *BlobReader) ReadAt(b []byte, at int64) (n int, err error)

ReadAt() fills b[] with up to len(b) bytes of data starting at position "at" within the blob that *br indicates, and returns the number of bytes read.

func (*BlobReader) Seek 

func (br *BlobReader) Seek(offset int64, whence int) (result int64, err error)

Seek() sets the seek position of *br to offset if whence==0, offset+current_seek_position if whence==1, and offset+end_of_blob if whence==2, and then returns the current seek position.

func (*BlobReader) Size 

func (br *BlobReader) Size() int64

Size() returns *br's size.

type BlobWriter 

type BlobWriter struct {
	// contains filtered or unexported fields
}

A BlobWriter allows a blob to be written. If a blob has not yet been finalized, it also allows that blob to be extended. A BlobWriter may be created with NewBlobWriter(), and should be closed with Close() or CloseWithoutFinalize().

func (*BlobWriter) AppendBlob 

func (bw *BlobWriter) AppendBlob(blobName string, size int64, offset int64) (err error)

AppendBlob() adds a (substring of a) pre-existing blob to the blob being written by *bw. The fragments of the pre-existing blob are not physically copied; they are referenced by both blobs.

func (*BlobWriter) AppendBytes 

func (bw *BlobWriter) AppendBytes(item ...localblobstore.BlockOrFile) (err error)

AppendBytes() tentatively appends bytes to the blob being written by *bw, where the bytes are composed of the byte vectors described by the elements of item[]. The implementation may choose when to commit these bytes to disc, except that they will be committed before the return of a subsequent call to Close() or CloseWithoutFinalize(). Uncomitted bytes may be lost during a crash, and will not be returned by a concurrent reader until they are committed.

func (*BlobWriter) Close 

func (bw *BlobWriter) Close() (err error)

Close() finalizes *bw, and indicates that the client will perform no further append operations on *bw. Any internal open file handles are closed.

func (*BlobWriter) CloseWithoutFinalize 

func (bw *BlobWriter) CloseWithoutFinalize() (err error)

CloseWithoutFinalize() indicates that the client will perform no further append operations on *bw, but does not finalize the blob. Any internal open file handles are closed. Clients are expected to need this operation infrequently.

func (*BlobWriter) Hash 

func (bw *BlobWriter) Hash() []byte

Hash() returns *bw's hash, reflecting the bytes written so far.

func (*BlobWriter) IsFinalized 

func (bw *BlobWriter) IsFinalized() bool

IsFinalized() returns whether *bw has been finalized.

func (*BlobWriter) Name 

func (bw *BlobWriter) Name() string

Name() returns *bw's name.

func (*BlobWriter) Size 

func (bw *BlobWriter) Size() int64

Size() returns *bw's size.

type FsCaBlobStore 

type FsCaBlobStore struct {
	// contains filtered or unexported fields
}

An FsCaBlobStore represents a simple, content-addressable store.

func Create 

func Create(ctx *context.T, stEngine, rootName string) (fscabs *FsCaBlobStore, err error)

Create() returns a pointer to an FsCaBlobStore stored in the file system at "rootName". If the directory rootName does not exist, it is created.

func (*FsCaBlobStore) BlobChunkStream 

func (fscabs *FsCaBlobStore) BlobChunkStream(ctx *context.T, blobName string) (cs localblobstore.ChunkStream)

BlobChunkStream() returns a ChunkStream that can be used to read the ordered list of content hashes of chunks in blob blobName. It is expected that this list will be presented to RecipeFromChunks() on another device, to create a recipe for transmitting the blob efficiently to that other device.

func (*FsCaBlobStore) Close 

func (fscabs *FsCaBlobStore) Close() error

Close() closes the FsCaBlobStore. {

func (*FsCaBlobStore) DeleteBlob 

func (fscabs *FsCaBlobStore) DeleteBlob(ctx *context.T, blobName string) (err error)

DeleteBlob() deletes the named blob from *fscabs.

func (*FsCaBlobStore) DeleteBlobMetadata 

func (fscabs *FsCaBlobStore) DeleteBlobMetadata(ctx *context.T, blobID wire.BlobRef) error

DeleteBlobMetadata() deletes the BlobMetadata for the specified blob.

func (*FsCaBlobStore) DeletePerSyncgroup 

func (fscabs *FsCaBlobStore) DeletePerSyncgroup(ctx *context.T, sgId interfaces.GroupId) error

DeletePerSyncgroup() deletes the PerSyncgroup for the specified blob.

func (*FsCaBlobStore) DeleteSignpost 

func (fscabs *FsCaBlobStore) DeleteSignpost(ctx *context.T, blobID wire.BlobRef) error

DeleteSignpost() deletes the Signpost for the specified blob.

func (*FsCaBlobStore) GC 

func (fscabs *FsCaBlobStore) GC(ctx *context.T) (err error)

GC() removes old temp files and content-addressed blocks that are no longer referenced by any blob. It may be called concurrently with other calls to GC(), and with uses of BlobReaders and BlobWriters.

func (*FsCaBlobStore) GetBlobMetadata 

func (fscabs *FsCaBlobStore) GetBlobMetadata(ctx *context.T, blobID wire.BlobRef, bmd *localblobstore.BlobMetadata) (err error)

GetBlobMetadata() yields in *bmd the BlobMetadata associated with a blob. If there is an error, *bmd is set to a canonical empty BlobMetadata. On return, it is guaranteed that any maps in *bmd are non-nil.

func (*FsCaBlobStore) GetPerSyncgroup 

func (fscabs *FsCaBlobStore) GetPerSyncgroup(ctx *context.T, sgId interfaces.GroupId, psg *localblobstore.PerSyncgroup) error

GetPerSyncgroup() yields in *psg the PerSyncgroup associated with a syncgroup. If there is an error, *psg is set to a canonical empty PerSyncgroup. On return, it is guaranteed that any maps in *psg are non-nil.

func (*FsCaBlobStore) GetSignpost 

func (fscabs *FsCaBlobStore) GetSignpost(ctx *context.T, blobID wire.BlobRef, sp *interfaces.Signpost) error

GetSignpost() yields in *sp the Signpost associated with a blob. If there is an error, *sp is set to a canonical empty Signpost. On return, it is guaranteed that any maps in *sp are non-nil.

func (*FsCaBlobStore) ListBlobIds 

func (fscabs *FsCaBlobStore) ListBlobIds(ctx *context.T) localblobstore.Stream

ListBlobIds() returns an iterator that can be used to enumerate the blobs in an FsCaBlobStore. Expected use is: 

fscabsi := fscabs.ListBlobIds(ctx)
for fscabsi.Advance() {
  // Process fscabsi.Value() here.
}
if fscabsi.Err() != nil {
  // The loop terminated early due to an error.
}

func (*FsCaBlobStore) ListCAIds 

func (fscabs *FsCaBlobStore) ListCAIds(ctx *context.T) localblobstore.Stream

ListCAIds() returns an iterator that can be used to enumerate the content-addressable fragments in an FsCaBlobStore. Expected use is: 

fscabsi := fscabs.ListCAIds(ctx)
for fscabsi.Advance() {
  // Process fscabsi.Value() here.
}
if fscabsi.Err() != nil {
  // The loop terminated early due to an error.
}

func (*FsCaBlobStore) LookupChunk 

func (fscabs *FsCaBlobStore) LookupChunk(ctx *context.T, chunkHash []byte) (loc localblobstore.Location, err error)

LookupChunk returns the location of a chunk with the specified chunk hash within the store.

func (*FsCaBlobStore) NewBlobMetadataStream 

func (fscabs *FsCaBlobStore) NewBlobMetadataStream(ctx *context.T) localblobstore.BlobMetadataStream

NewBlobMetadataStream() returns a pointer to a BlobMetadataStream that allows the client to iterate over each blob for which a BlobMetadata has been specified.

func (*FsCaBlobStore) NewBlobReader 

func (fscabs *FsCaBlobStore) NewBlobReader(ctx *context.T, blobName string) (br localblobstore.BlobReader, err error)

NewBlobReader() returns a pointer to a newly allocated BlobReader on the specified blobName. BlobReaders should not be used concurrently by multiple threads. Returned handles should be closed with Close().

func (*FsCaBlobStore) NewBlobWriter 

func (fscabs *FsCaBlobStore) NewBlobWriter(ctx *context.T, name string) (localblobstore.BlobWriter, error)

NewBlobWriter() returns a pointer to a newly allocated BlobWriter on a newly created blob. If "name" is non-empty, it is used to name the blob, and it must be in the format of a name returned by this interface (probably by another instance on another device). Otherwise, a new name is created, which can be found using the Name() method. It is an error to attempt to overwrite a blob that already exists in this blob store. BlobWriters should not be used concurrently by multiple threads. The returned handle should be closed with either the Close() or CloseWithoutFinalize() method to avoid leaking file handles.

func (*FsCaBlobStore) NewPerSyncgroupStream 

func (fscabs *FsCaBlobStore) NewPerSyncgroupStream(ctx *context.T) localblobstore.PerSyncgroupStream

NewPerSyncgroupStream() returns a pointer to a PerSyncgroupStream that allows the client to iterate over each syncgroup for which a PerSyncgroup has been specified.

func (*FsCaBlobStore) NewSignpostStream 

func (fscabs *FsCaBlobStore) NewSignpostStream(ctx *context.T) localblobstore.SignpostStream

NewSignpostStream() returns a pointer to a SignpostStream that allows the client to iterate over each blob for which a Signpost has been specified.

func (*FsCaBlobStore) RecipeStreamFromChunkStream 

func (fscabs *FsCaBlobStore) RecipeStreamFromChunkStream(ctx *context.T, chunkStream localblobstore.ChunkStream) localblobstore.RecipeStream

RecipeStreamFromChunkStream() returns a pointer to a RecipeStream that allows the client to iterate over each RecipeStep needed to create the blob formed by the chunks in chunkStream.

func (*FsCaBlobStore) ResumeBlobWriter 

func (fscabs *FsCaBlobStore) ResumeBlobWriter(ctx *context.T, blobName string) (localblobstore.BlobWriter, error)

ResumeBlobWriter() returns a pointer to a newly allocated BlobWriter on an old, but unfinalized blob name.

func (*FsCaBlobStore) Root 

func (fscabs *FsCaBlobStore) Root() string

Root() returns the name of the root directory where *fscabs is stored.

func (*FsCaBlobStore) SetBlobMetadata 

func (fscabs *FsCaBlobStore) SetBlobMetadata(ctx *context.T, blobID wire.BlobRef, bmd *localblobstore.BlobMetadata) error

SetBlobMetadata() sets the BlobMetadata associated with a blob to *bmd.

func (*FsCaBlobStore) SetPerSyncgroup 

func (fscabs *FsCaBlobStore) SetPerSyncgroup(ctx *context.T, sgId interfaces.GroupId, psg *localblobstore.PerSyncgroup) error

SetPerSyncgroup() sets the PerSyncgroup associated with a syncgroup to *psg.

func (*FsCaBlobStore) SetSignpost 

func (fscabs *FsCaBlobStore) SetSignpost(ctx *context.T, blobID wire.BlobRef, sp *interfaces.Signpost) error

SetSignpost() sets the Signpost associated with a blob to *sp.

type FsCasIter 

type FsCasIter struct {
	// contains filtered or unexported fields
}

An FsCasIter represents an iterator that allows the client to enumerate all the blobs or fragments in a FsCaBlobStore.

func (*FsCasIter) Advance 

func (fscabsi *FsCasIter) Advance() (advanced bool)

Advance() stages an item so that it may be retrieved via Value. Returns true iff there is an item to retrieve. Advance must be called before Value is called.

func (*FsCasIter) Cancel 

func (fscabsi *FsCasIter) Cancel()

Cancel() indicates that the iteration stream should terminate early. Never blocks. May be called concurrently with other methods on fscabsi.

func (*FsCasIter) Err 

func (fscabsi *FsCasIter) Err() error

Err() returns any error encountered by Advance. Never blocks.

func (*FsCasIter) Value 

func (fscabsi *FsCasIter) Value() (name string)

Value() returns the item that was staged by Advance. May panic if Advance returned false or was not called. Never blocks.

type RecipeStream 

type RecipeStream struct {
	// contains filtered or unexported fields
}

A RecipeStream implements localblobstore.RecipeStream. It allows the client to iterate over the recipe steps to recreate a blob identified by a stream of chunk hashes (from chunkStream), but using parts of blobs in the current blob store where possible.

func (*RecipeStream) Advance 

func (rs *RecipeStream) Advance() (ok bool)

Advance() stages an item so that it may be retrieved via Value(). Returns true iff there is an item to retrieve. Advance() must be called before Value() is called. The caller is expected to read until Advance() returns false, or to call Cancel().

func (*RecipeStream) Cancel 

func (rs *RecipeStream) Cancel()

Cancel() indicates that the client wishes to cease reading from the stream. It causes the next call to Advance() to return false. Never blocks. It may be called concurrently with other calls on the stream.

func (*RecipeStream) Err 

func (rs *RecipeStream) Err() error

Err() returns any error encountered by Advance. Never blocks.

func (*RecipeStream) Value 

func (rs *RecipeStream) Value() localblobstore.RecipeStep

Value() returns the item that was staged by Advance(). May panic if Advance() returned false or was not called. Never blocks.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.