Back to / blob / fileblob

Package fileblob

Latest Go to latest

The latest major version is .

Published: Jun 10, 2020 | License: Apache-2.0 | Module:


Package fileblob provides a blob implementation that uses the filesystem. Use OpenBucket to construct a *blob.Bucket.


For blob.OpenBucket, fileblob registers for the scheme "file". To customize the URL opener, or for more details on the URL format, see URLOpener. See for background information.


Go CDK supports all UTF-8 strings; to make this work with services lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for fileblob:

- Blob keys: ASCII characters 0-31 are escaped to "__0x<hex>__".
  If os.PathSeparator != "/", it is also escaped.
  Additionally, the "/" in "../", the trailing "/" in "//", and a trailing
  "/" is key names are escaped in the same way.
  On Windows, the characters "<>:"|?*" are also escaped.


fileblob exposes the following types for As:

- Error: *os.PathError
Example (OpenBucketFromURL)


package main

import (

func main() {
	// Create a temporary directory.
	dir, err := ioutil.TempDir("", "go-cloud-fileblob-example")
	if err != nil {
	defer os.RemoveAll(dir)

	// On Unix, append the dir to "file://".
	// On Windows, convert "\" to "/" and add a leading "/":
	dirpath := filepath.ToSlash(dir)
	if os.PathSeparator != '/' && !strings.HasPrefix(dirpath, "/") {
		dirpath = "/" + dirpath

	// blob.OpenBucket creates a *blob.Bucket from a URL.
	ctx := context.Background()
	b, err := blob.OpenBucket(ctx, "file://"+dirpath)
	if err != nil {
	defer b.Close()

	// Now we can use b to read or write files to the container.
	err = b.WriteAll(ctx, "my-key", []byte("hello world"), nil)
	if err != nil {
	data, err := b.ReadAll(ctx, "my-key")
	if err != nil {

hello world




const Scheme = "file"

Scheme is the URL scheme fileblob registers its URLOpener under on blob.DefaultMux.

func OpenBucket

func OpenBucket(dir string, opts *Options) (*blob.Bucket, error)

OpenBucket creates a *blob.Bucket backed by the filesystem and rooted at dir, which must exist. See the package documentation for an example.



package main

import (

func main() {
	// PRAGMA: This example is used on; PRAGMA comments adjust how it is shown and can be ignored.

	// The directory you pass to fileblob.OpenBucket must exist first.
	const myDir = "path/to/local/directory"
	if err := os.MkdirAll(myDir, 0777); err != nil {

	// Create a file-based bucket.
	bucket, err := fileblob.OpenBucket(myDir, nil)
	if err != nil {
	defer bucket.Close()

type Options

type Options struct {
	// URLSigner implements signing URLs (to allow access to a resource without
	// further authorization) and verifying that a given URL is unexpired and
	// contains a signature produced by the URLSigner.
	// URLSigner is only required for utilizing the SignedURL API.
	URLSigner URLSigner

Options sets options for constructing a *blob.Bucket backed by fileblob.

type URLOpener

type URLOpener struct {
	// Options specifies the default options to pass to OpenBucket.
	Options Options

URLOpener opens file bucket URLs like "file:///foo/bar/baz".

The URL's host is ignored unless it is ".", which is used to signal a relative path. For example, "file://./../.." uses "../.." as the path.

If os.PathSeparator != "/", any leading "/" from the path is dropped and remaining '/' characters are converted to os.PathSeparator.

The following query parameters are supported:

- base_url: the base URL to use to construct signed URLs; see URLSignerHMAC
- secret_key_path: path to read for the secret key used to construct signed URLs;
  see URLSignerHMAC

If either of these is provided, both must be.

- file:///a/directory
  -> Passes "/a/directory" to OpenBucket.
- file://localhost/a/directory
  -> Also passes "/a/directory".
- file://./../..
  -> The hostname is ".", signaling a relative path; passes "../..".
- file:///c:/foo/bar on Windows.
  -> Passes "c:\foo\bar".
- file://localhost/c:/foo/bar on Windows.
  -> Also passes "c:\foo\bar".
- file:///a/directory?base_url=/show&secret_key_path=secret.key
  -> Passes "/a/directory" to OpenBucket, and sets Options.URLSigner
     to a URLSignerHMAC initialized with base URL "/show" and secret key
     bytes read from the file "secret.key".

func (*URLOpener) OpenBucketURL

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens a blob.Bucket based on u.

type URLSigner

type URLSigner interface {
	// URLFromKey defines how the bucket's object key will be turned
	// into a signed URL. URLFromKey must be safe to call from multiple goroutines.
	URLFromKey(ctx context.Context, key string, opts *driver.SignedURLOptions) (*url.URL, error)

	// KeyFromURL must be able to validate a URL returned from URLFromKey.
	// KeyFromURL must only return the object if if the URL is
	// both unexpired and authentic. KeyFromURL must be safe to call from
	// multiple goroutines. Implementations of KeyFromURL should not modify
	// the URL argument.
	KeyFromURL(ctx context.Context, surl *url.URL) (string, error)

URLSigner defines an interface for creating and verifying a signed URL for objects in a fileblob bucket. Signed URLs are typically used for granting access to an otherwise-protected resource without requiring further authentication, and callers should take care to restrict the creation of signed URLs as is appropriate for their application.

type URLSignerHMAC

type URLSignerHMAC struct {
	// contains filtered or unexported fields

URLSignerHMAC signs URLs by adding the object key, expiration time, and a hash-based message authentication code (HMAC) into the query parameters. Values of URLSignerHMAC with the same secret key will accept URLs produced by others as valid.

func NewURLSignerHMAC

func NewURLSignerHMAC(baseURL *url.URL, secretKey []byte) *URLSignerHMAC

NewURLSignerHMAC creates a URLSignerHMAC. If the secret key is empty, then NewURLSignerHMAC panics.

func (*URLSignerHMAC) KeyFromURL

func (h *URLSignerHMAC) KeyFromURL(ctx context.Context, sURL *url.URL) (string, error)

KeyFromURL checks expiry and signature, and returns the object key only if the signed URL is both authentic and unexpired.

func (*URLSignerHMAC) URLFromKey

func (h *URLSignerHMAC) URLFromKey(ctx context.Context, key string, opts *driver.SignedURLOptions) (*url.URL, error)

URLFromKey creates a signed URL by copying the baseURL and appending the object key, expiry, and signature as a query params.

Package Files

Documentation was rendered with GOOS=linux and GOARCH=amd64.

Jump to identifier

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to identifier