badgers3

README

Updated S3 for CertMagic Storage

This library allows you to use any S3-compatible provider as key/certificate storage backend for your Certmagic-enabled HTTPS server. To protect your keys from unwanted attention, client-side encryption using secretbox is possible.

See example/ for an exemplary integration.

Why have we made this fork?

Whilst using this plugin, Certmagic itself calls the Load and other functions quite a lot and there is not any level of caching on those functions for the library. We've chosen BadgerDB which is a proven database that has been able to handle millions of concurrent reads and writes on our systems. We've learned that the default S3 cache library simply cannot cut it and handle the amount of requests we receive.

The aim of this fork is to improve performance and scalability when it comes to using the AWS S3 storage with Certmagic to store certificates.

What is a S3-compatible service?

In the current state, any service must support the following:

• v4 Signatures
• HTTPS
• A few basic operations:
  • Bucket Exists
  • Get Object
  • Put Object
  • Remove Object
  • Stat Object
  • List Objects

Known good providers/software:

• Minio (with HTTPS enabled)
• Backblaze
• AWS

For development

Our caching key format is as follows

• <key> - Just a regular S3 file
• `<key_ki> - The key info for a S3 file

Documentation

Index

• Variables
• type CleartextIO
  • func (ci *CleartextIO) ByteReader(buf []byte) Reader
  • func (ci *CleartextIO) WrapReader(r io.Reader) io.Reader
• type IO
• type Reader
  • func (r *Reader) Len() int64
  • func (r Reader) Read(buf []byte) (int, error)
• type S3Opts
• type S3Storage
  • func NewS3Storage(opts S3Opts) (*S3Storage, error)
  • func (gs *S3Storage) Delete(ctx context.Context, key string) error
  • func (gs *S3Storage) Exists(ctx context.Context, key string) bool
  • func (gs *S3Storage) List(ctx context.Context, prefix string, recursive bool) ([]string, error)
  • func (gs *S3Storage) Load(ctx context.Context, key string) ([]byte, error)
  • func (gs *S3Storage) Lock(ctx context.Context, key string) error
  • func (gs *S3Storage) Stat(ctx context.Context, key string) (certmagic.KeyInfo, error)
  • func (gs *S3Storage) Store(ctx context.Context, key string, value []byte) error
  • func (gs *S3Storage) Unlock(ctx context.Context, key string) error
• type SecretBoxIO
  • func (sb *SecretBoxIO) ByteReader(msg []byte) Reader
  • func (sb *SecretBoxIO) WrapReader(r io.Reader) io.Reader

Variables

var (
	LockExpiration   = 2 * time.Minute
	LockPollInterval = 1 * time.Second
	LockTimeout      = 15 * time.Second
)

Types

type CleartextIO

type CleartextIO struct{}

func (*CleartextIO) ByteReader

func (ci *CleartextIO) ByteReader(buf []byte) Reader

func (*CleartextIO) WrapReader

func (ci *CleartextIO) WrapReader(r io.Reader) io.Reader

type IO

type IO interface {
	WrapReader(io.Reader) io.Reader
	ByteReader([]byte) Reader
}

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

func (*Reader) Len

func (r *Reader) Len() int64

func (Reader) Read

func (r Reader) Read(buf []byte) (int, error)

type S3Opts

type S3Opts struct {
	Endpoint        string
	Bucket          string
	AccessKeyID     string
	SecretAccessKey string

	ObjPrefix string

	// EncryptionKey is optional. If you do not wish to encrypt your certficates and key inside the S3 bucket, leave it empty.
	EncryptionKey []byte
}

type S3Storage

type S3Storage struct {
	// contains filtered or unexported fields
}

func NewS3Storage

func NewS3Storage(opts S3Opts) (*S3Storage, error)

func (*S3Storage) Delete

func (gs *S3Storage) Delete(ctx context.Context, key string) error

func (*S3Storage) Exists

func (gs *S3Storage) Exists(ctx context.Context, key string) bool

func (*S3Storage) List

func (gs *S3Storage) List(ctx context.Context, prefix string, recursive bool) ([]string, error)

func (*S3Storage) Load

func (gs *S3Storage) Load(ctx context.Context, key string) ([]byte, error)

func (*S3Storage) Lock

func (gs *S3Storage) Lock(ctx context.Context, key string) error

func (*S3Storage) Stat

func (gs *S3Storage) Stat(ctx context.Context, key string) (certmagic.KeyInfo, error)

func (*S3Storage) Store

func (gs *S3Storage) Store(ctx context.Context, key string, value []byte) error

func (*S3Storage) Unlock

func (gs *S3Storage) Unlock(ctx context.Context, key string) error

type SecretBoxIO

type SecretBoxIO struct {
	SecretKey [32]byte
}

func (*SecretBoxIO) ByteReader

func (sb *SecretBoxIO) ByteReader(msg []byte) Reader

func (*SecretBoxIO) WrapReader

func (sb *SecretBoxIO) WrapReader(r io.Reader) io.Reader