README
¶
juju/blobstore
This package provides a Mongo GridFS-backed blob storage engine.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrRequestExpired = fmt.Errorf("request expired")
ErrRequestExpired is used to indicate that a put request has already expired when an attempt is made to supply a response.
var ErrResourceDeleted = fmt.Errorf("resource was deleted")
ErrResourceDeleted is used to indicate that a resource was deleted before the put response could be acted on.
var ErrResponseMismatch = fmt.Errorf("response checksums do not match")
ErrResponseMismatch is used to indicate that a put response did not contain the expected checksums.
var ErrUploadPending = fmt.Errorf("Resource not available because upload is not yet complete")
ErrUploadPending is used to indicate that the underlying resource for a catalog entry is not yet fully uploaded.
Functions ¶
func NewPutResponse ¶
NewPutResponse creates a new putResponse for the given requestId and hashes.
Types ¶
type ManagedResource ¶
ManagedResource is a catalog entry for stored data. The data may be associated with a specified environment and/or user. The data is logically considered to be stored at the specified path.
type ManagedStorage ¶
type ManagedStorage interface {
// GetForEnvironment returns a reader for data at path, namespaced to the environment.
// If the data is still being uploaded and is not fully written yet,
// an ErrUploadPending error is returned. This means the path is valid but the caller
// should try again to retrieve the data.
GetForEnvironment(envUUID, path string) (r io.ReadCloser, length int64, err error)
// PutForEnvironment stores data from reader at path, namespaced to the environment.
PutForEnvironment(envUUID, path string, r io.Reader, length int64) error
// RemoveForEnvironment deletes data at path, namespaced to the environment.
RemoveForEnvironment(envUUID, path string) error
// PutForEnvironmentRequest requests that data, which may already exist in storage,
// be saved at path, namespaced to the environment. It allows callers who can
// demonstrate proof of ownership of the data to store a reference to it without
// having to upload it all. If no such data exists, a NotFound error is returned
// and a call to EnvironmentPut is required. If matching data is found, the caller
// is returned a response indicating the random byte range to for which they must
// provide checksums to complete the process.
PutForEnvironmentRequest(envUUID, path string, hash ResourceHash) (*RequestResponse, error)
// ProofOfAccessResponse is called to respond to a Put..Request call in order to
// prove ownership of data for which a storage reference is created.
ProofOfAccessResponse(putResponse) error
}
ManagedStorage instances persist data for an environment, for a user, or globally. (Only environment storage is currently implemented).
func NewManagedStorage ¶
func NewManagedStorage(db *mgo.Database, rs ResourceStorage) ManagedStorage
NewManagedStorage creates a new ManagedStorage using the transaction runner, storing resource entries in the specified database, and resource data in the specified resource storage.
type PutRequest ¶
type PutRequest struct {
// contains filtered or unexported fields
}
PutRequest is to record a request to put a file pending proof of access.
type RequestResponse ¶
RequestResponse is returned by a put request to inform the caller the data range over which to calculate the hashes for the response.
type Resource ¶
type Resource struct {
ResourceHash
Path string
Length int64
}
Resource is a catalog entry for stored data. It contains the path where the data is stored as well as hashes of the data which are used for de-duping.
type ResourceCatalog ¶
type ResourceCatalog interface {
// Get fetches a Resource with the given id.
Get(id string) (*Resource, error)
// Find returns the resource id for the Resource with the given hashes.
Find(rh *ResourceHash) (id string, err error)
// Put ensures a Resource entry exists for the given ResourceHash, returning the id, path,
// and a flag indicating if this is a new record.
// If the Resource exists, its reference count is incremented, otherwise a new entry is created.
Put(rh *ResourceHash, length int64) (id, path string, isNew bool, err error)
// UploadComplete records that the underlying resource described by the Resource entry with id
// is now fully uploaded and the resource is available for use.
UploadComplete(id string) error
// Remove decrements the reference count for a Resource with the given id, deleting it
// if the reference count reaches zero. The path of the Resource is returned.
// If the Resource is deleted, wasDeleted is returned as true.
Remove(id string) (wasDeleted bool, path string, err error)
}
ResourceCatalog instances persist Resources. Resources with the same hash values are not duplicated; instead a reference count is incremented. Similarly, when a Resource is removed, the reference count is decremented. When the reference count reaches zero, the Resource is deleted.
type ResourceHash ¶
ResourceHash contains hashes which are used to unambiguously identify stored data.
type ResourceStorage ¶
type ResourceStorage interface {
// Get returns a reader for the resource located at path.
Get(path string) (io.ReadCloser, error)
// Put writes data from the specified reader to path and returns a checksum of the data written.
Put(path string, r io.Reader, length int64) (checksum string, err error)
// Remove deletes the data at the specified path.
Remove(path string) error
}
ResourceStorage instances save and retrieve data from an underlying storage implementation.
Source Files