s3

README

gateway

None of this would be possible, or at the very least require a huge amount of R&D if it wasn't for the amazing people at Protocol Labs and other community contributors at the ipfs/go-ds-s3 repository.

It is being modified to work better with Storj as the datastore, avoiding the limitations and restrictions imposed by using an S3 service with a company like AWS.

Documentation

Overview

Package s3 is our client with storj minio gateways

Index

• type Config
  • func NewConfig(accessKey, secretKey, logPath string) Config
• type Datastore
  • func NewDatastore(cfg Config, dev bool) (*Datastore, error)
  • func (d *Datastore) Batch() (ds.Batch, error)
  • func (d *Datastore) BucketExists(name string) error
  • func (d *Datastore) Close() error
  • func (d *Datastore) CreateBucket(name string) error
  • func (d *Datastore) Delete(k ds.Key) error
  • func (d *Datastore) DeleteBucket(name string) error
  • func (d *Datastore) Get(k ds.Key) ([]byte, error)
  • func (d *Datastore) GetSize(k ds.Key) (size int, err error)
  • func (d *Datastore) Has(k ds.Key) (exists bool, err error)
  • func (d *Datastore) Put(k ds.Key, value []byte) error
  • func (d *Datastore) Query(q dsq.Query) (dsq.Results, error)

Types

type Config

type Config struct {
	AccessKey string
	SecretKey string
	//	SessionToken   string
	Bucket        string
	Region        string
	Endpoint      string
	RootDirectory string
	LogPath       string
	Secure        bool
	Workers       int
}

Config is used to configure our gateway

func NewConfig

func NewConfig(accessKey, secretKey, logPath string) Config

NewConfig is used to generate a config with defaults

type Datastore

type Datastore struct {
	S3 *s3.S3

	Config
	// contains filtered or unexported fields
}

Datastore is our interface to minio

func NewDatastore

func NewDatastore(cfg Config, dev bool) (*Datastore, error)

NewDatastore is used to create our datastore against the minio gateway powered by storj

func (*Datastore) Batch

func (d *Datastore) Batch() (ds.Batch, error)

Batch is a batched datastore operations

func (*Datastore) BucketExists

func (d *Datastore) BucketExists(name string) error

BucketExists is used to lookup if the designated bucket exists

func (*Datastore) Close

func (d *Datastore) Close() error

Close is needed to satisfy the datastore interface

func (*Datastore) CreateBucket

func (d *Datastore) CreateBucket(name string) error

CreateBucket is used to create a bucket

func (*Datastore) Delete

func (d *Datastore) Delete(k ds.Key) error

Delete is used to remove an object from our datastore

func (*Datastore) DeleteBucket

func (d *Datastore) DeleteBucket(name string) error

DeleteBucket is used to remove the specified bucket

func (*Datastore) Get

func (d *Datastore) Get(k ds.Key) ([]byte, error)

Get is used to retrieve data from our storj backed s3 datastore we do not log errors here since checking if an object exists is part of the IPFS operation process when adding a file it avoids having to needlessly operate n files

func (*Datastore) GetSize

func (d *Datastore) GetSize(k ds.Key) (size int, err error)

GetSize is used to retrieve the size of an object

func (*Datastore) Has

func (d *Datastore) Has(k ds.Key) (exists bool, err error)

Has is used to check if we already have an object matching this key we do not log errors here since checking if an object exists is part of the IPFS operation process when adding a file it avoids having to needlessly operate n files

func (*Datastore) Put

func (d *Datastore) Put(k ds.Key, value []byte) error

Put is used to store some data

func (*Datastore) Query

func (d *Datastore) Query(q dsq.Query) (dsq.Results, error)

Query is used to examine our s3 datastore and pull any objects matching our given query