s3

Documentation

Overview

Package s3 implements an S3-backed repository. Objects are stored within a prefix in a bucket.

Index

• func Dial(u *url.URL) (reflow.Repository, error)
• func SetClient(client s3iface.S3API)
• type Repository
  • func (r *Repository) Collect(ctx context.Context, live reflow.Liveset) error
  • func (r *Repository) Get(ctx context.Context, id digest.Digest) (io.ReadCloser, error)
  • func (r *Repository) GetFile(ctx context.Context, id digest.Digest, w io.WriterAt) (int64, error)
  • func (r *Repository) Put(ctx context.Context, body io.Reader) (digest.Digest, error)
  • func (r *Repository) PutFile(ctx context.Context, file reflow.File, body io.Reader) error
  • func (r *Repository) ReadFrom(_ context.Context, id digest.Digest, u *url.URL) error
  • func (r *Repository) ShortString() string
  • func (r *Repository) Stat(ctx context.Context, id digest.Digest) (reflow.File, error)
  • func (r *Repository) String() string
  • func (r *Repository) URL() *url.URL
  • func (r *Repository) WriteTo(_ context.Context, id digest.Digest, u *url.URL) error

Functions

func Dial

func Dial(u *url.URL) (reflow.Repository, error)

Dial dials an s3 repository. The URL must have the form:

s3r://bucket/prefix

TODO(marius): we should support shipping authentication information in the URL also.

func SetClient

func SetClient(client s3iface.S3API)

SetClient sets the default s3 client to use for dialling repositories. If non-nil, it is used when there is not a more specific per-bucket client.

Types

type Repository

type Repository struct {
	Client s3iface.S3API
	Bucket string
	Prefix string
}

Repository implements an S3-backed Repository. Objects are stored in the given bucket under the given prefix, followed by "objects":

s3://bucket/<prefix>/objects/sha256:<hex>...

In-progress uploads are stored under "uploads":

s3://bucket/<prefix>/uploads/<hex>

func (*Repository) Collect

func (r *Repository) Collect(ctx context.Context, live reflow.Liveset) error

Collect is not supported on S3.

func (*Repository) Get

func (r *Repository) Get(ctx context.Context, id digest.Digest) (io.ReadCloser, error)

Get retrieves an object from the repository.

func (*Repository) GetFile

func (r *Repository) GetFile(ctx context.Context, id digest.Digest, w io.WriterAt) (int64, error)

GetFile retrieves an object from the repository directly to the a io.WriterAt. This uses the S3 download manager to download chunks concurrently.

func (*Repository) Put

func (r *Repository) Put(ctx context.Context, body io.Reader) (digest.Digest, error)

Put installs an object into the repository; its digest ID is returned.

func (*Repository) PutFile

func (r *Repository) PutFile(ctx context.Context, file reflow.File, body io.Reader) error

PutFile installs a file into the repository. PutFile uses the S3 upload manager directly.

func (*Repository) ReadFrom

func (r *Repository) ReadFrom(_ context.Context, id digest.Digest, u *url.URL) error

ReadFrom is unsupported by the S3 repository.

TODO(marius): we can support other s3r here by performing CopyObjects.

func (*Repository) ShortString

func (r *Repository) ShortString() string

func (*Repository) Stat

func (r *Repository) Stat(ctx context.Context, id digest.Digest) (reflow.File, error)

Stat queries the repository for object metadata.

func (*Repository) String

func (r *Repository) String() string

String returns the repository URL.

func (*Repository) URL

func (r *Repository) URL() *url.URL

URL returns the URL for this repository. It is of the form:

s3r://bucket/prefix

func (*Repository) WriteTo

func (r *Repository) WriteTo(_ context.Context, id digest.Digest, u *url.URL) error

WriteTo is unsupported by the S3 repository.

TODO(marius): we can support other s3r here by performing CopyObjects.