storage

README

storage

Package storage offers generic (cloud) storage drivers.

Drivers

Supported drivers are as follows:

Driver	Description
disk	Local disk.
ftp	FTP storage.
google	Google Drive storage.
minio	Minio storage.
s3	Amazon S3 storage.
sftp	SFTP server storage (over SSH).
stack	TransIP Stack storage.
webdav	WebDAV server storage.

Usage

In order to use one of the supported storage backends, you need to import the appropriate driver and supply a configuration like follows:

package main

import (
    "fmt"

    "maze.io/x/storage"
    _ "maze.io/x/storage/disk" // Local disk driver
)

func main() {
    config := storage.Config{
        Type:   "disk",
        Config: map[string]interface{}{
            "path": "/tmp",
        },
    }

    backend, err := config.Open()
    if err != nil {
        panic(err)
    }

    fmt.Printf("using storage %s\n", backend)
}

The storage.Config struct exists to facilitate loading storage configurations from structured configuration formats, such as JSON or YAML:

package main

import (
    "encoding/json"
    "fmt"

    "maze.io/x/storage"
    _ "maze.io/x/storage/disk" // Local disk driver
)

func main() {
    var config storage.Config
    if err := json.Unmarshal([]byte(configFile), &config); err != nil {
        panic(err)
    }

    backend, err := config.Open()
    if err != nil {
        panic(err)
    }

    fmt.Printf("using storage %s\n", backend)
}

const configFile = `{
    "type": "disk",
    "config": {
        "path": "/"
    }
}`

You can also use the driver directly:

package main

import (
    "fmt"

    "maze.io/x/storage/disk" // Local disk driver
)

func main() {
    config := disk.Config{
        Path: "/tmp",
    }

    backend, err := config.Open()
    if err != nil {
        panic(err)
    }

    fmt.Printf("using storage %s\n", backend)
}

Documentation

Overview

Package storage offers generic (cloud) storage drivers.

Directories

Note that there are no mkdir/rmdir operations, although storage drivers should support subdirectories. Directory management is abstracted away in the backends that should attempt to purge empty directories.

Index

• Variables
• func ExistError(op, path string) error
• func Glob(ctx context.Context, storage Storage, pattern string) (matches []string, err error)
• func NotExistError(op, path string) error
• func PermissonError(op, path string) error
• func Register(kind string, fn func() Opener)
• type Config
  • func (c *Config) Open() (Storage, error)
• type Opener
• type Storage
  • func OpenMemory() Storage

Variables

var ErrBadPattern = errors.New("syntax error in pattern")

ErrBadPattern indicates a pattern was malformed.

var (
	ErrConfigNil = errors.New("config can't be nil")
)

Common errors.

Functions

func ExistError

func ExistError(op, path string) error

ExistError returns an error indicating that the file already exists. The error will pass os.IsExist.

func Glob

func Glob(ctx context.Context, storage Storage, pattern string) (matches []string, err error)

Glob returns the names of all files matching pattern or nil if there is no matching file. The pattern may describe hierarchical names such as /usr/*/bin/ed.

Glob ignores file system errors such as I/O errors reading directories. The only possible returned error is ErrBadPattern, when pattern is malformed.

The pattern syntax is:

pattern:
	{ term }

term:
	'*'         matches any sequence of non-Separator characters
	'?'         matches any single non-Separator character
	'[' [ '^' ] { character-range } ']'
	            character class (must be non-empty)
	c           matches character c (c != '*', '?', '\\', '[')
	'\\' c      matches character c

character-range:
	c           matches character c (c != '\\', '-', ']')
	'\\' c      matches character c
	lo '-' hi   matches character c for lo <= c <= hi

func NotExistError

func NotExistError(op, path string) error

NotExistError returns an error indicating that a file doesn't exist. The error will pass os.IsNotExist.

func PermissonError

func PermissonError(op, path string) error

PermissonError returns an error indicating that there is a permission error accessing the file. The error will pass os.IsPermission.

func Register

func Register(kind string, fn func() Opener)

Register a new storage type.

Types

type Config

type Config struct {
	Type   string                 `json:"type" yaml:"type"`
	Config map[string]interface{} `json:"config" yaml:"config"`
}

Config for a storage backend.

func (*Config) Open

func (c *Config) Open() (Storage, error)

Open the storage backend.

type Opener

type Opener interface {
	Open() (Storage, error)
}

Opener can open a new Storage.

type Storage

type Storage interface {
	// Name gives a description of the storage.
	Name() string

	// Close the storage.
	Close() error

	// Save a file to the storage.
	Save(ctx context.Context, name string, r io.Reader) error

	// Open a file from storage for reading.
	Open(ctx context.Context, name string) (io.ReadCloser, error)

	// Create a file in storage for writing.
	Create(ctx context.Context, name string) (io.WriteCloser, error)

	// Readdir reads directory contents.
	Readdir(ctx context.Context, name string) ([]os.FileInfo, error)

	// Stat a file in storage.
	Stat(ctx context.Context, name string) (os.FileInfo, error)

	// Remove a file from storage.
	Remove(ctx context.Context, name string) error
}

Storage for named files. All paths in the supported storage use a backward slash ("/") as path separator.

func OpenMemory

func OpenMemory() Storage

OpenMemory is an in-memory storage (for testing).