README

storage

Package storage offers generic (cloud) storage drivers.

Drivers

Supported drivers are as follows:

Driver Description
disk Local disk.
ftp FTP storage.
google Google Drive storage.
minio Minio storage.
s3 Amazon S3 storage.
sftp SFTP server storage (over SSH).
stack TransIP Stack storage.
webdav WebDAV server storage.

Usage

In order to use one of the supported storage backends, you need to import the appropriate driver and supply a configuration like follows:

package main

import (
    "fmt"

    "maze.io/x/storage"
    _ "maze.io/x/storage/disk" // Local disk driver
)

func main() {
    config := storage.Config{
        Type:   "disk",
        Config: map[string]interface{}{
            "path": "/tmp",
        },
    }

    backend, err := config.Open()
    if err != nil {
        panic(err)
    }

    fmt.Printf("using storage %s\n", backend)
}

The storage.Config struct exists to facilitate loading storage configurations from structured configuration formats, such as JSON or YAML:

package main

import (
    "encoding/json"
    "fmt"

    "maze.io/x/storage"
    _ "maze.io/x/storage/disk" // Local disk driver
)

func main() {
    var config storage.Config
    if err := json.Unmarshal([]byte(configFile), &config); err != nil {
        panic(err)
    }

    backend, err := config.Open()
    if err != nil {
        panic(err)
    }

    fmt.Printf("using storage %s\n", backend)
}

const configFile = `{
    "type": "disk",
    "config": {
        "path": "/"
    }
}`

You can also use the driver directly:

package main

import (
    "fmt"

    "maze.io/x/storage/disk" // Local disk driver
)

func main() {
    config := disk.Config{
        Path: "/tmp",
    }

    backend, err := config.Open()
    if err != nil {
        panic(err)
    }

    fmt.Printf("using storage %s\n", backend)
}

Documentation

Overview

Package storage offers generic (cloud) storage drivers.

Directories

Note that there are no mkdir/rmdir operations, although storage drivers should support subdirectories. Directory management is abstracted away in the backends that should attempt to purge empty directories.

Index

Constants

This section is empty.

Variables

View Source
var ErrBadPattern = errors.New("syntax error in pattern")

    ErrBadPattern indicates a pattern was malformed.

    View Source
    var (
    	ErrConfigNil = errors.New("config can't be nil")
    )

      Common errors.

      Functions

      func ExistError

      func ExistError(op, path string) error

        ExistError returns an error indicating that the file already exists. The error will pass os.IsExist.

        func Glob

        func Glob(ctx context.Context, storage Storage, pattern string) (matches []string, err error)

          Glob returns the names of all files matching pattern or nil if there is no matching file. The pattern may describe hierarchical names such as /usr/*/bin/ed.

          Glob ignores file system errors such as I/O errors reading directories. The only possible returned error is ErrBadPattern, when pattern is malformed.

          The pattern syntax is:

          pattern:
          	{ term }
          
          term:
          	'*'         matches any sequence of non-Separator characters
          	'?'         matches any single non-Separator character
          	'[' [ '^' ] { character-range } ']'
          	            character class (must be non-empty)
          	c           matches character c (c != '*', '?', '\\', '[')
          	'\\' c      matches character c
          
          character-range:
          	c           matches character c (c != '\\', '-', ']')
          	'\\' c      matches character c
          	lo '-' hi   matches character c for lo <= c <= hi
          

          func NotExistError

          func NotExistError(op, path string) error

            NotExistError returns an error indicating that a file doesn't exist. The error will pass os.IsNotExist.

            func PermissonError

            func PermissonError(op, path string) error

              PermissonError returns an error indicating that there is a permission error accessing the file. The error will pass os.IsPermission.

              func Register

              func Register(kind string, fn func() Opener)

                Register a new storage type.

                Types

                type Config

                type Config struct {
                	Type   string                 `json:"type" yaml:"type"`
                	Config map[string]interface{} `json:"config" yaml:"config"`
                }

                  Config for a storage backend.

                  func (*Config) Open

                  func (c *Config) Open() (Storage, error)

                    Open the storage backend.

                    type Opener

                    type Opener interface {
                    	Open() (Storage, error)
                    }

                      Opener can open a new Storage.

                      type Storage

                      type Storage interface {
                      	// Name gives a description of the storage.
                      	Name() string
                      
                      	// Close the storage.
                      	Close() error
                      
                      	// Save a file to the storage.
                      	Save(ctx context.Context, name string, r io.Reader) error
                      
                      	// Open a file from storage for reading.
                      	Open(ctx context.Context, name string) (io.ReadCloser, error)
                      
                      	// Create a file in storage for writing.
                      	Create(ctx context.Context, name string) (io.WriteCloser, error)
                      
                      	// Readdir reads directory contents.
                      	Readdir(ctx context.Context, name string) ([]os.FileInfo, error)
                      
                      	// Stat a file in storage.
                      	Stat(ctx context.Context, name string) (os.FileInfo, error)
                      
                      	// Remove a file from storage.
                      	Remove(ctx context.Context, name string) error
                      }

                        Storage for named files. All paths in the supported storage use a backward slash ("/") as path separator.

                        func OpenMemory

                        func OpenMemory() Storage

                          OpenMemory is an in-memory storage (for testing).

                          Directories

                          Path Synopsis
                          Package all loads all supported storage drivers.
                          Package all loads all supported storage drivers.
                          cmd
                          Package disk provides local disk storage.
                          Package disk provides local disk storage.
                          Package ftp provides FTP storage.
                          Package ftp provides FTP storage.
                          Package google provides Google Drive storage.
                          Package google provides Google Drive storage.
                          Package oauth2 contains helper functions for provisioning OAuth2 tokens.
                          Package oauth2 contains helper functions for provisioning OAuth2 tokens.
                          Package s3 provides S3 (and compatible) storage.
                          Package s3 provides S3 (and compatible) storage.
                          Package sftp provides SFTP storage.
                          Package sftp provides SFTP storage.
                          Package webdav provides WebDAV (and compatible) storage.
                          Package webdav provides WebDAV (and compatible) storage.
                          internal
                          obscure
                          Package obscure contains the Obscure and Reveal commands
                          Package obscure contains the Obscure and Reveal commands