Back to godoc.org
gocloud.dev / blob / fileblob

Package fileblob

v0.20.0
Latest Go to latest

The latest major version is .

Published: Jun 10, 2020 | License: Apache-2.0 | Module: gocloud.dev

Overview

Package fileblob provides a blob implementation that uses the filesystem. Use OpenBucket to construct a *blob.Bucket.

URLs

For blob.OpenBucket, fileblob registers for the scheme "file". To customize the URL opener, or for more details on the URL format, see URLOpener. See https://gocloud.dev/concepts/urls/ for background information.

Escaping

Go CDK supports all UTF-8 strings; to make this work with services lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for fileblob:

- Blob keys: ASCII characters 0-31 are escaped to "__0x<hex>__".
  If os.PathSeparator != "/", it is also escaped.
  Additionally, the "/" in "../", the trailing "/" in "//", and a trailing
  "/" is key names are escaped in the same way.
  On Windows, the characters "<>:"|?*" are also escaped.

As

fileblob exposes the following types for As:

- Error: *os.PathError
Example (OpenBucketFromURL)

Code:

package main

import (
	"context"
	"fmt"
	"gocloud.dev/blob"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
	"strings"
)

func main() {
	// Create a temporary directory.
	dir, err := ioutil.TempDir("", "go-cloud-fileblob-example")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(dir)

	// On Unix, append the dir to "file://".
	// On Windows, convert "\" to "/" and add a leading "/":
	dirpath := filepath.ToSlash(dir)
	if os.PathSeparator != '/' && !strings.HasPrefix(dirpath, "/") {
		dirpath = "/" + dirpath
	}

	// blob.OpenBucket creates a *blob.Bucket from a URL.
	ctx := context.Background()
	b, err := blob.OpenBucket(ctx, "file://"+dirpath)
	if err != nil {
		log.Fatal(err)
	}
	defer b.Close()

	// Now we can use b to read or write files to the container.
	err = b.WriteAll(ctx, "my-key", []byte("hello world"), nil)
	if err != nil {
		log.Fatal(err)
	}
	data, err := b.ReadAll(ctx, "my-key")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(data))

}
hello world

Index

Examples

Constants

const Scheme = "file"

Scheme is the URL scheme fileblob registers its URLOpener under on blob.DefaultMux.

func OpenBucket

func OpenBucket(dir string, opts *Options) (*blob.Bucket, error)

OpenBucket creates a *blob.Bucket backed by the filesystem and rooted at dir, which must exist. See the package documentation for an example.

Example

Code:

package main

import (
	"gocloud.dev/blob/fileblob"
	"log"
	"os"
)

func main() {
	// PRAGMA: This example is used on gocloud.dev; PRAGMA comments adjust how it is shown and can be ignored.

	// The directory you pass to fileblob.OpenBucket must exist first.
	const myDir = "path/to/local/directory"
	if err := os.MkdirAll(myDir, 0777); err != nil {
		log.Fatal(err)
	}

	// Create a file-based bucket.
	bucket, err := fileblob.OpenBucket(myDir, nil)
	if err != nil {
		log.Fatal(err)
	}
	defer bucket.Close()
}

type Options

type Options struct {
	// URLSigner implements signing URLs (to allow access to a resource without
	// further authorization) and verifying that a given URL is unexpired and
	// contains a signature produced by the URLSigner.
	// URLSigner is only required for utilizing the SignedURL API.
	URLSigner URLSigner
}

Options sets options for constructing a *blob.Bucket backed by fileblob.

type URLOpener

type URLOpener struct {
	// Options specifies the default options to pass to OpenBucket.
	Options Options
}

URLOpener opens file bucket URLs like "file:///foo/bar/baz".

The URL's host is ignored unless it is ".", which is used to signal a relative path. For example, "file://./../.." uses "../.." as the path.

If os.PathSeparator != "/", any leading "/" from the path is dropped and remaining '/' characters are converted to os.PathSeparator.

The following query parameters are supported:

- base_url: the base URL to use to construct signed URLs; see URLSignerHMAC
- secret_key_path: path to read for the secret key used to construct signed URLs;
  see URLSignerHMAC

If either of these is provided, both must be.

- file:///a/directory
  -> Passes "/a/directory" to OpenBucket.
- file://localhost/a/directory
  -> Also passes "/a/directory".
- file://./../..
  -> The hostname is ".", signaling a relative path; passes "../..".
- file:///c:/foo/bar on Windows.
  -> Passes "c:\foo\bar".
- file://localhost/c:/foo/bar on Windows.
  -> Also passes "c:\foo\bar".
- file:///a/directory?base_url=/show&secret_key_path=secret.key
  -> Passes "/a/directory" to OpenBucket, and sets Options.URLSigner
     to a URLSignerHMAC initialized with base URL "/show" and secret key
     bytes read from the file "secret.key".

func (*URLOpener) OpenBucketURL

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens a blob.Bucket based on u.

type URLSigner

type URLSigner interface {
	// URLFromKey defines how the bucket's object key will be turned
	// into a signed URL. URLFromKey must be safe to call from multiple goroutines.
	URLFromKey(ctx context.Context, key string, opts *driver.SignedURLOptions) (*url.URL, error)

	// KeyFromURL must be able to validate a URL returned from URLFromKey.
	// KeyFromURL must only return the object if if the URL is
	// both unexpired and authentic. KeyFromURL must be safe to call from
	// multiple goroutines. Implementations of KeyFromURL should not modify
	// the URL argument.
	KeyFromURL(ctx context.Context, surl *url.URL) (string, error)
}

URLSigner defines an interface for creating and verifying a signed URL for objects in a fileblob bucket. Signed URLs are typically used for granting access to an otherwise-protected resource without requiring further authentication, and callers should take care to restrict the creation of signed URLs as is appropriate for their application.

type URLSignerHMAC

type URLSignerHMAC struct {
	// contains filtered or unexported fields
}

URLSignerHMAC signs URLs by adding the object key, expiration time, and a hash-based message authentication code (HMAC) into the query parameters. Values of URLSignerHMAC with the same secret key will accept URLs produced by others as valid.

func NewURLSignerHMAC

func NewURLSignerHMAC(baseURL *url.URL, secretKey []byte) *URLSignerHMAC

NewURLSignerHMAC creates a URLSignerHMAC. If the secret key is empty, then NewURLSignerHMAC panics.

func (*URLSignerHMAC) KeyFromURL

func (h *URLSignerHMAC) KeyFromURL(ctx context.Context, sURL *url.URL) (string, error)

KeyFromURL checks expiry and signature, and returns the object key only if the signed URL is both authentic and unexpired.

func (*URLSignerHMAC) URLFromKey

func (h *URLSignerHMAC) URLFromKey(ctx context.Context, key string, opts *driver.SignedURLOptions) (*url.URL, error)

URLFromKey creates a signed URL by copying the baseURL and appending the object key, expiry, and signature as a query params.

Package Files

Documentation was rendered with GOOS=linux and GOARCH=amd64.

Jump to identifier

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to identifier