backblaze

package module
Version: v0.0.0-...-35409b8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 24, 2021 License: MIT Imports: 15 Imported by: 25

README

go-backblaze

GoDoc Build Status

A golang client for Backblaze's B2 storage

Usage

Some simple examples to get you started. Errors are ommitted for brevity

Import the API package

import "gopkg.in/kothar/go-backblaze.v0"

Create an API client

b2, _ := backblaze.NewB2(backblaze.Credentials{
  AccountID:      accountID,
  ApplicationKey: applicationKey,
})

Create a bucket

bucket, _ := b2.CreateBucket("test_bucket", backblaze.AllPrivate)

Uploading a file

reader, _ := os.Open(path)
name := filepath.Base(path)
metadata := make(map[string]string)

file, _ := bucket.UploadFile(name, metadata, reader)

All API methods except B2.AuthorizeAccount and Bucket.UploadHashedFile will retry once if authorization fails, which allows the operation to proceed if the current authorization token has expired.

To disable this behaviour, set B2.NoRetry to true

b2 command line client

A test applicaiton has been implemented using this package, and can be found in the /b2 directory. It should provide you with more examples of how to use the API in your own applications.

To install the b2 command, use:

go get -u gopkg.in/kothar/go-backblaze.v0/b2

$ b2 --help
Usage:
  b2 [OPTIONS] <command>

Application Options:
      --account= The account ID to use [$B2_ACCOUNT_ID]
      --appKey=  The application key to use [$B2_APP_KEY]
  -b, --bucket=  The bucket to access [$B2_BUCKET]
  -d, --debug    Debug API requests
  -v, --verbose  Display verbose output

Help Options:
  -h, --help     Show this help message

Available commands:
  createbucket  Create a new bucket
  delete        Delete a file
  deletebucket  Delete a bucket
  get           Download a file
  list          List files in a bucket
  listbuckets   List buckets in an account
  put           Store a file

Documentation

Overview

Package backblaze B2 API for Golang

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type B2

type B2 struct {
	Credentials

	// If true, don't retry requests if authorization has expired
	NoRetry bool

	// If true, display debugging information about API calls
	Debug bool

	// Number of MaxIdleUploads to keep for reuse.
	// This must be set prior to creating a bucket struct
	MaxIdleUploads int
	// contains filtered or unexported fields
}

B2 implements a B2 API client. Do not modify state concurrently.

Example
NewB2(Credentials{
	AccountID:      "", // Obtained from your B2 account page.
	ApplicationKey: "", // Obtained from your B2 account page.
})
Output:

func NewB2

func NewB2(creds Credentials) (*B2, error)

NewB2 creates a new Client for accessing the B2 API. The AuthorizeAccount method will be called immediately.

func (*B2) AuthorizeAccount

func (c *B2) AuthorizeAccount() error

AuthorizeAccount is used to log in to the B2 API.

func (*B2) Bucket

func (b *B2) Bucket(bucketName string) (*Bucket, error)

Bucket looks up a bucket for the currently authorized client

func (*B2) CreateBucket

func (b *B2) CreateBucket(bucketName string, bucketType BucketType) (*Bucket, error)

CreateBucket creates a new B2 Bucket in the authorized account.

Buckets can be named. The name must be globally unique. No account can use a bucket with the same name. Buckets are assigned a unique bucketId which is used when uploading, downloading, or deleting files.

func (*B2) CreateBucketWithInfo

func (b *B2) CreateBucketWithInfo(bucketName string, bucketType BucketType, bucketInfo map[string]string, lifecycleRules []LifecycleRule) (*Bucket, error)

CreateBucketWithInfo extends CreateBucket to add bucket info and lifecycle rules to the creation request

func (*B2) DownloadFileByID

func (c *B2) DownloadFileByID(fileID string) (*File, io.ReadCloser, error)

DownloadFileByID downloads a file from B2 using its unique ID

func (*B2) DownloadFileRangeByID

func (c *B2) DownloadFileRangeByID(fileID string, fileRange *FileRange) (*File, io.ReadCloser, error)

DownloadFileRangeByID downloads part of a file from B2 using its unique ID and a requested byte range.

func (*B2) DownloadURL

func (c *B2) DownloadURL() (string, error)

DownloadURL returns the URL prefix needed to construct download links. Bucket.FileURL will costruct a full URL for given file names.

func (*B2) ListBuckets

func (b *B2) ListBuckets() ([]*Bucket, error)

ListBuckets lists buckets associated with an account, in alphabetical order by bucket ID.

func (*B2) ReadaheadFile

func (c *B2) ReadaheadFile(file *File) (io.ReadCloser, error)

ReadaheadFile attempts to load chunks of the file being downloaded ahead of time to improve transfer rates. This method attempts to optimise the chunk size based on the file size and a target of 15 workers

File ranges are downloaded using Content-Range requests. See DownloadFileRangeByName

func (*B2) ReadaheadFileOptions

func (c *B2) ReadaheadFileOptions(file *File, chunkSize, chunkAhead, numWorkers int) (io.ReadCloser, error)

ReadaheadFileOptions attempts to load chunks of the file being downloaded ahead of time to improve transfer rates. This method extends ReadaheadFile with options to configure the chunk size, number of chunks to read ahead and the number of workers to use.

File ranges are downloaded using Content-Range requests. See DownloadFileRangeByName

type B2Error

type B2Error struct {
	Code    string `json:"code"`
	Message string `json:"message"`
	Status  int    `json:"status"`
}

B2Error encapsulates an error message returned by the B2 API.

Failures to connect to the B2 servers, and networking problems in general can cause errors

func (B2Error) Error

func (e B2Error) Error() string

func (*B2Error) IsFatal

func (e *B2Error) IsFatal() bool

IsFatal returns true if this error represents an error which can't be recovered from by retrying

type Bucket

type Bucket struct {
	*BucketInfo
	// contains filtered or unexported fields
}

Bucket provides access to the files stored in a B2 Bucket

Example
var b2 B2
// ...

b2.CreateBucket("test_bucket", AllPrivate)
Output:

func (*Bucket) CopyFile

func (b *Bucket) CopyFile(fileID, fileName, destinationBucketId string, metadataDirective FileMetadataDirective) (*File, error)

CopyFile copies file

If destination bucket is empty, file will be copy to the current file's bucket

func (*Bucket) Delete

func (b *Bucket) Delete() error

Delete removes removes the bucket from the authorized account. Only buckets that contain no version of any files can be deleted.

func (*Bucket) DeleteFileVersion

func (b *Bucket) DeleteFileVersion(fileName, fileID string) (*FileStatus, error)

DeleteFileVersion deletes one version of a file from B2.

If the version you delete is the latest version, and there are older versions, then the most recent older version will become the current version, and be the one that you'll get when downloading by name. See the File Versions page for more details.

func (*Bucket) DownloadFileByName

func (b *Bucket) DownloadFileByName(fileName string) (*File, io.ReadCloser, error)

DownloadFileByName downloads one file by providing the name of the bucket and the name of the file.

func (*Bucket) DownloadFileRangeByName

func (b *Bucket) DownloadFileRangeByName(fileName string, fileRange *FileRange) (*File, io.ReadCloser, error)

DownloadFileRangeByName downloads part of a file by providing the name of the bucket, the name of the file, and a requested byte range

func (*Bucket) FileURL

func (b *Bucket) FileURL(fileName string) (string, error)

FileURL returns a URL which may be used to download the latest version of a file. This returned URL will only work for public buckets unless the correct authorization header is provided.

func (*Bucket) GetFileInfo

func (b *Bucket) GetFileInfo(fileID string) (*File, error)

GetFileInfo retrieves information about one file stored in B2.

func (*Bucket) GetUploadAuth

func (b *Bucket) GetUploadAuth() (*UploadAuth, error)

GetUploadAuth retrieves the URL to use for uploading files.

When you upload a file to B2, you must call b2_get_upload_url first to get the URL for uploading directly to the place where the file will be stored.

If the upload is successful, ReturnUploadAuth(*uploadAuth) should be called to place it back in the pool for reuse.

func (*Bucket) HideFile

func (b *Bucket) HideFile(fileName string) (*FileStatus, error)

HideFile hides a file so that downloading by name will not find the file, but previous versions of the file are still stored. See File Versions about what it means to hide a file.

func (*Bucket) ListFileNames

func (b *Bucket) ListFileNames(startFileName string, maxFileCount int) (*ListFilesResponse, error)

ListFileNames lists the names of all files in a bucket, starting at a given name.

See ListFileNamesWithPrefix

func (*Bucket) ListFileNamesWithPrefix

func (b *Bucket) ListFileNamesWithPrefix(startFileName string, maxFileCount int, prefix, delimiter string) (*ListFilesResponse, error)

ListFileNamesWithPrefix lists the names of all files in a bucket, starting at a given name.

This call returns at most 1000 file names per transaction, but it can be called repeatedly to scan through all of the file names in a bucket. Each time you call, it returns a "nextFileName" that can be used as the starting point for the next call.

If you set maxFileCount to more than 1000 (max 10,000) and more than 1000 are returned, the call will be billed as multiple transactions, as if you had made requests in a loop asking for 1000 at a time.

Files returned will be limited to those with the given prefix. The empty string matches all files.

If a delimiter is provided, files returned will be limited to those within the top folder, or any one subfolder. Folder names will also be returned. The delimiter character will be used to "break" file names into folders.

func (*Bucket) ListFileVersions

func (b *Bucket) ListFileVersions(startFileName, startFileID string, maxFileCount int) (*ListFileVersionsResponse, error)

ListFileVersions lists all of the versions of all of the files contained in one bucket, in alphabetical order by file name, and by reverse of date/time uploaded for versions of files with the same name.

func (*Bucket) ReadaheadFileByName

func (b *Bucket) ReadaheadFileByName(fileName string) (*File, io.ReadCloser, error)

ReadaheadFileByName attempts to load chunks of the file being downloaded ahead of time to improve transfer rates. File ranges are downloaded using Content-Range requests. See DownloadFileRangeByName

func (*Bucket) ReturnUploadAuth

func (b *Bucket) ReturnUploadAuth(uploadAuth *UploadAuth)

ReturnUploadAuth returns an upload URL to the available pool. This should not be called if the upload fails. Instead request another GetUploadAuth() and retry.

func (*Bucket) Update

func (b *Bucket) Update(bucketType BucketType) error

Update allows the bucket type to be changed

func (*Bucket) UpdateAll

func (b *Bucket) UpdateAll(bucketType BucketType, bucketInfo map[string]string, lifecycleRules []LifecycleRule, ifRevisionIs int) error

UpdateAll allows all bucket properties to be changed

bucketType (optional) -- One of: "allPublic", "allPrivate". "allPublic" means that anybody can download the files is the bucket; "allPrivate" means that you need an authorization token to download them. If not specified, setting will remain unchanged.

bucketInfo (optional) -- User-defined information to be stored with the bucket. If specified, the existing bucket info will be replaced with the new info. If not specified, setting will remain unchanged. Cache-Control policies can be set here on a global level for all the files in the bucket.

lifecycleRules (optional) -- The list of lifecycle rules for this bucket. If specified, the existing lifecycle rules will be replaced with this new list. If not specified, setting will remain unchanged.

ifRevisionIs (optional) -- When set (> 0), the update will only happen if the revision number stored in the B2 service matches the one passed in. This can be used to avoid having simultaneous updates make conflicting changes.

func (*Bucket) UploadFile

func (b *Bucket) UploadFile(name string, meta map[string]string, file io.Reader) (*File, error)

UploadFile calls UploadTypedFile with the b2/x-auto contentType

Example
var bucket Bucket
// ...

path := "/path/to/file"
reader, _ := os.Open(path)
name := filepath.Base(path)
metadata := make(map[string]string)

bucket.UploadFile(name, metadata, reader)
Output:

func (*Bucket) UploadHashedFile

func (b *Bucket) UploadHashedFile(
	name string, meta map[string]string, file io.Reader,
	sha1Hash string, contentLength int64) (*File, error)

UploadHashedFile calls UploadHashedTypedFile with the b2/x-auto file type

func (*Bucket) UploadHashedTypedFile

func (b *Bucket) UploadHashedTypedFile(
	name, contentType string, meta map[string]string, file io.Reader,
	sha1Hash string, contentLength int64) (*File, error)

UploadHashedTypedFile Uploads a file to B2, returning its unique file ID.

This method will not retry if the upload fails, as the reader may have consumed some bytes. If the error type is B2Error and IsFatal returns false, you may retry the upload and expect it to succeed eventually.

func (*Bucket) UploadTypedFile

func (b *Bucket) UploadTypedFile(name, contentType string, meta map[string]string, file io.Reader) (*File, error)

UploadTypedFile uploads a file to B2, returning its unique file ID. This method computes the hash of the file before passing it to UploadHashedFile

type BucketInfo

type BucketInfo struct {
	// The account that the bucket is in.
	AccountID string `json:"accountId"`

	// The unique ID of the bucket.
	ID string `json:"bucketId"`

	// User-defined information to be stored with the bucket.
	Info map[string]string `json:"bucketInfo"`

	// The name to give the new bucket.
	// Bucket names must be a minimum of 6 and a maximum of 50 characters long, and must be globally unique;
	// two different B2 accounts cannot have buckets with the name name. Bucket names can consist of: letters,
	// digits, and "-". Bucket names cannot start with "b2-"; these are reserved for internal Backblaze use.
	Name string `json:"bucketName"`

	// Either "allPublic", meaning that files in this bucket can be downloaded by anybody, or "allPrivate",
	// meaning that you need a bucket authorization token to download the files.
	BucketType BucketType `json:"bucketType"`

	// The initial list of lifecycle rules for this bucket.
	LifecycleRules []LifecycleRule `json:"lifecycleRules"`

	// A counter that is updated every time the bucket is modified.
	Revision int `json:"revision"`
}

BucketInfo describes a bucket

type BucketType

type BucketType string

BucketType defines the security setting for a bucket

const (
	AllPublic  BucketType = "allPublic"
	AllPrivate BucketType = "allPrivate"
	Snapshot   BucketType = "snapshot"
)

Buckets can be either public, private, or snapshot

type Credentials

type Credentials struct {
	AccountID      string
	ApplicationKey string
	KeyID          string
}

Credentials are the identification required by the Backblaze B2 API

The account ID is a 12-digit hex number that you can get from your account page on backblaze.com.

The application key is a 40-digit hex number that you can get from your account page on backblaze.com.

The key id is the application key id that you get when generating an application key. If using the master application key, leave this set to an empty string as your account id will be used instead.

type File

type File struct {
	ID              string            `json:"fileId"`
	Name            string            `json:"fileName"`
	AccountID       string            `json:"accountId"`
	BucketID        string            `json:"bucketId"`
	ContentLength   int64             `json:"contentLength"`
	ContentSha1     string            `json:"contentSha1"`
	ContentType     string            `json:"contentType"`
	FileInfo        map[string]string `json:"fileInfo"`
	Action          FileAction        `json:"action"`
	Size            int               `json:"size"` // Deprecated - same as ContentSha1
	UploadTimestamp int64             `json:"uploadTimestamp"`
}

File descibes a file stored in a B2 bucket

type FileAction

type FileAction string

FileAction indicates the current status of a file in a B2 bucket

const (
	Upload FileAction = "upload"
	Hide   FileAction = "hide"
)

Files can be either uploads (visible) or hidden.

Hiding a file makes it look like the file has been deleted, without removing any of the history. It adds a new version of the file that is a marker saying the file is no longer there.

type FileMetadataDirective

type FileMetadataDirective string
const (
	FileMetaDirectiveCopy    FileMetadataDirective = "COPY"
	FileMetaDirectiveReplace FileMetadataDirective = "REPLACE"
)

type FileRange

type FileRange struct {
	Start int64
	End   int64
}

FileRange describes a range of bytes in a file by its 0-based start and end position (inclusive)

type FileStatus

type FileStatus struct {
	File
}

FileStatus is now identical to File in repsonses from ListFileNames and ListFileVersions

type LifecycleRule

type LifecycleRule struct {
	DaysFromUploadingToHiding int    `json:"daysFromUploadingToHiding"`
	DaysFromHidingToDeleting  int    `json:"daysFromHidingToDeleting"`
	FileNamePrefix            string `json:"fileNamePrefix"`
}

LifecycleRule instructs the B2 service to automatically hide and/or delete old files. You can set up rules to do things like delete old versions of files 30 days after a newer version was uploaded.

type ListFileVersionsResponse

type ListFileVersionsResponse struct {
	Files        []FileStatus `json:"files"`
	NextFileName string       `json:"nextFileName"`
	NextFileID   string       `json:"nextFileId"`
}

ListFileVersionsResponse lists a page of file versions stored in a B2 bucket

type ListFilesResponse

type ListFilesResponse struct {
	Files        []FileStatus `json:"files"`
	NextFileName string       `json:"nextFileName"`
}

ListFilesResponse lists a page of files stored in a B2 bucket

type UploadAuth

type UploadAuth struct {
	AuthorizationToken string
	UploadURL          *url.URL
	Valid              bool
}

UploadAuth encapsulates the details needed to upload a file to B2 These are pooled and must be returned when complete.

Directories

Path Synopsis
A minimal client for accessing B2
A minimal client for accessing B2

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL