Documentation

Overview

Package azureblob provides a blob implementation that uses Azure Storage’s BlockBlob. Use OpenBucket to construct a *blob.Bucket.

NOTE: SignedURLs for PUT created with this package are not fully portable; they will not work unless the PUT request includes a "x-ms-blob-type" header set to "BlockBlob". See https://stackoverflow.com/questions/37824136/put-on-sas-blob-url-without-specifying-x-ms-blob-type-header.

URLs

For blob.OpenBucket, azureblob registers for the scheme "azblob". The default URL opener will use credentials from the environment variables AZURE_STORAGE_ACCOUNT, AZURE_STORAGE_KEY, and AZURE_STORAGE_SAS_TOKEN. AZURE_STORAGE_ACCOUNT is required, along with one of the other two. AZURE_STORAGE_DOMAIN can optionally be used to provide an Azure Environment blob storage domain to use. If no AZURE_STORAGE_DOMAIN is provided, the default Azure public domain "blob.core.windows.net" will be used. Check the Azure Developer Guide for your particular cloud environment to see the proper blob storage domain name to provide. To customize the URL opener, or for more details on the URL format, see URLOpener. See https://gocloud.dev/concepts/urls/ for background information.

Escaping

Go CDK supports all UTF-8 strings; to make this work with services lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for azureblob:

- Blob keys: ASCII characters 0-31, 92 ("\"), and 127 are escaped to
  "__0x<hex>__". Additionally, the "/" in "../" and a trailing "/" in a
  key (e.g., "foo/") are escaped in the same way.
- Metadata keys: Per https://docs.microsoft.com/en-us/azure/storage/blobs/storage-properties-metadata,
  Azure only allows C# identifiers as metadata keys. Therefore, characters
  other than "[a-z][A-z][0-9]_" are escaped using "__0x<hex>__". In addition,
  characters "[0-9]" are escaped when they start the string.
  URL encoding would not work since "%" is not valid.
- Metadata values: Escaped using URL encoding.

As

azureblob exposes the following types for As:

- Bucket: *azblob.ContainerURL
- Error: azblob.StorageError
- ListObject: azblob.BlobItemInternal for objects, azblob.BlobPrefix for "directories"
- ListOptions.BeforeList: *azblob.ListBlobsSegmentOptions
- Reader: azblob.DownloadResponse
- Reader.BeforeRead: *azblob.BlockBlobURL, *azblob.BlobAccessConditions
- Attributes: azblob.BlobGetPropertiesResponse
- CopyOptions.BeforeCopy: azblob.Metadata, *azblob.ModifiedAccessConditions, *azblob.BlobAccessConditions
- WriterOptions.BeforeWrite: *azblob.UploadStreamToBlockBlobOptions
- SignedURLOptions.BeforeSign: *azblob.BlobSASSignatureValues
Example (OpenBucketFromURL)
Output:

Index

Examples

Constants

View Source
const Scheme = "azblob"

    Scheme is the URL scheme gcsblob registers its URLOpener under on blob.DefaultMux.

    Variables

      DefaultIdentity is a Wire provider set that provides an Azure storage account name, key, and SharedKeyCredential from environment variables.

        SASTokenIdentity is a Wire provider set that provides an Azure storage account name, SASToken, and anonymous credential from environment variables.

        View Source
        var Set = wire.NewSet(
        	NewPipeline,
        	wire.Struct(new(Options), "Credential", "SASToken"),
        	wire.Struct(new(URLOpener), "AccountName", "Pipeline", "Options"),
        )

          Set holds Wire providers for this package.

          Functions

          func NewCredential

          func NewCredential(accountName AccountName, accountKey AccountKey) (*azblob.SharedKeyCredential, error)

            NewCredential creates a SharedKeyCredential.

            func NewPipeline

            func NewPipeline(credential azblob.Credential, opts azblob.PipelineOptions) pipeline.Pipeline

              NewPipeline creates a Pipeline for making HTTP requests to Azure.

              func OpenBucket

              func OpenBucket(ctx context.Context, pipeline pipeline.Pipeline, accountName AccountName, containerName string, opts *Options) (*blob.Bucket, error)

                OpenBucket returns a *blob.Bucket backed by Azure Storage Account. See the package documentation for an example and https://godoc.org/github.com/Azure/azure-storage-blob-go/azblob for more details.

                Example
                Output:
                
                
                Example (UsingAADCredentials)
                Output:
                
                
                Example (UsingSASToken)
                Output:
                
                

                Types

                type AccountKey

                type AccountKey string

                  AccountKey is an Azure storage account key (primary or secondary).

                  func DefaultAccountKey

                  func DefaultAccountKey() (AccountKey, error)

                    DefaultAccountKey loads the Azure storage account key (primary or secondary) from the AZURE_STORAGE_KEY environment variable.

                    type AccountName

                    type AccountName string

                      AccountName is an Azure storage account name.

                      func DefaultAccountName

                      func DefaultAccountName() (AccountName, error)

                        DefaultAccountName loads the Azure storage account name from the AZURE_STORAGE_ACCOUNT environment variable.

                        type Options

                        type Options struct {
                        	// Credential represents the authorizer for SignedURL.
                        	// Required to use SignedURL. If you're using MSI for authentication, this will
                        	// attempt to be loaded lazily the first time you call SignedURL.
                        	Credential azblob.StorageAccountCredential
                        
                        	// SASToken can be provided along with anonymous credentials to use
                        	// delegated privileges.
                        	// See https://docs.microsoft.com/en-us/azure/storage/common/storage-dotnet-shared-access-signature-part-1#shared-access-signature-parameters.
                        	SASToken SASToken
                        
                        	// StorageDomain can be provided to specify an Azure Cloud Environment
                        	// domain to target for the blob storage account (i.e. public, government, china).
                        	// The default value is "blob.core.windows.net". Possible values will look similar
                        	// to this but are different for each cloud (i.e. "blob.core.govcloudapi.net" for USGovernment).
                        	// Check the Azure developer guide for the cloud environment where your bucket resides.
                        	StorageDomain StorageDomain
                        
                        	// Protocol can be provided to specify protocol to access Azure Blob Storage.
                        	// Protocols that can be specified are "http" for local emulator and "https" for general.
                        	// If blank is specified, "https" will be used.
                        	Protocol Protocol
                        }

                          Options sets options for constructing a *blob.Bucket backed by Azure Block Blob.

                          type Protocol

                          type Protocol string

                            Protocol is an protocol to access Azure Blob Storage. It must be "http" or "https". It is read from the AZURE_STORAGE_PROTOCOL environment variable.

                            func DefaultProtocol

                            func DefaultProtocol() (Protocol, error)

                              DefaultProtocol loads the protocol to access Azure Blob Storage from the AZURE_STORAGE_PROTOCOL environment variable.

                              type SASToken

                              type SASToken string

                                SASToken is an Azure shared access signature. https://docs.microsoft.com/en-us/azure/storage/common/storage-dotnet-shared-access-signature-part-1

                                func DefaultSASToken

                                func DefaultSASToken() (SASToken, error)

                                  DefaultSASToken loads a Azure SAS token from the AZURE_STORAGE_SAS_TOKEN environment variable.

                                  type StorageDomain

                                  type StorageDomain string

                                    StorageDomain is an Azure Cloud Environment domain name to target (i.e. blob.core.windows.net, blob.core.govcloudapi.net, blob.core.chinacloudapi.cn). It is read from the AZURE_STORAGE_DOMAIN environment variable.

                                    func DefaultStorageDomain

                                    func DefaultStorageDomain() (StorageDomain, error)

                                      DefaultStorageDomain loads the desired Azure Cloud to target from the AZURE_STORAGE_DOMAIN environment variable.

                                      type URLOpener

                                      type URLOpener struct {
                                      	// AccountName must be specified.
                                      	AccountName AccountName
                                      
                                      	// Pipeline must be set to a non-nil value.
                                      	Pipeline pipeline.Pipeline
                                      
                                      	// Options specifies the options to pass to OpenBucket.
                                      	Options Options
                                      }

                                        URLOpener opens Azure URLs like "azblob://mybucket".

                                        The URL host is used as the bucket name.

                                        The following query options are supported:

                                        - domain: The domain name used to access the Azure Blob storage (e.g. blob.core.windows.net)
                                        

                                        func (*URLOpener) OpenBucketURL

                                        func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

                                          OpenBucketURL opens a blob.Bucket based on u.

                                          Source Files