configazureblob

README

config-azure-blob

Read and write a config file that lives in an Azure Blob container, through config

Part of the phpboyscout Go toolkit — small, framework-free Go modules extracted from go-tool-base. Documented with the parent module at config.go.phpboyscout.uk

---

config reads and writes configuration through a small six-method filesystem interface. This module supplies that interface backed by an Azure Blob Storage container, so a tool whose config.yaml lives in a blob reads it — and writes it back — through the same codec and stage-and-rename machinery a local file uses. A consumer who reads config from a blob takes this module; one who does not pays nothing for it.

This is a filesystem adapter: it serves one config file that happens to live in a blob. Treating blob names as config keys — the parameter-store shape — is a different thing, and Azure's member of that family is the sibling config-azure-appconfig backend.

Usage

You build and configure the *azblob.Client — that is where every endpoint and credential decision lives; the credential is yours, via azidentity or a connection string, and this module never imports it. You hand the client in with the container name; the blob's name is the file path you pass to WithFiles:

import (
	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
	"gitlab.com/phpboyscout/go/config"
	configazureblob "gitlab.com/phpboyscout/go/config-azure-blob"
)

cred, _ := azidentity.NewDefaultAzureCredential(nil)
client, _ := azblob.NewClient("https://acct.blob.core.windows.net/", cred, nil)

store, err := config.NewStore(ctx,
	config.WithFiles(configazureblob.Wrap(client, "config"), "app/config.yaml"))

reads the blob named app/config.yaml from container config. The name is passed to the object operations unchanged — the adapter does not root, clean or rewrite it, and takes no prefix. store.Apply(ctx, config.Set(...)) writes it back.

The container is bound once, at Wrap, and every operation is scoped to it.

An object store is not a disk

Azure Blob Storage is a flat namespace of objects reached over HTTPS. It has no directories, no atomic rename, no stat, and no native change feed, and the adapter meets it honestly rather than pretending it is a disk. Three behaviours are worth understanding up front:

• The rename is a copy-then-delete. There is no atomic rename in Blob Storage. A commit stages content beside the target and renames over it; here that rename is a synchronous server-side copy (CopyFromURL, which completes in the one call for the small blobs a config file is) followed by a delete of the staging blob. Each object write is atomic, so the target is still replaced atomically and a reader never observes a half-written blob. The one imperfection: a crash between the copy and the delete leaves the staging blob behind — a blob whose name ends in .yamldoc-tmp — which is garbage to clean up, never a corrupt target. A stray *.yamldoc-tmp blob is always safe to delete.

• Remove of an absent blob succeeds. Deleting a blob that is not there returns nil, matching the idempotent intent of discarding staged content. This diverges from os.Remove, which errors on a missing file. (Reading an absent blob still returns an fs.ErrNotExist-shaped error, which the Store relies on to treat a missing optional source as normal.)

• The container must already exist. MkdirAll is a no-op — an object store has no directories — and Wrap does not create the container. Provisioning it is the consumer's job, consistent with the endpoint and credential being yours; a write to a missing container fails with ContainerNotFound.

Stat synthesises an fs.FileInfo from the blob's properties: real size and last-modified time, a fixed nominal mode (0o644 — a blob has no POSIX permission), IsDir false, and Sys nil.

Conflict detection still holds: the core fingerprints a source's content by SHA-256 at load and refuses a write if it changed underneath, which is content-based and works unchanged over an object store — no ETag machinery is needed for correctness, and Blob Storage's strong read-after-write consistency means the re-read sees the latest bytes.

Watching

A blob has no operating-system path, so the wrapped container is not a config.RealPather and is watched by polling. Each poll re-reads the blob — a billed GET — so the 2-second local default is far too eager. The adapter implements config.PollIntervalHinter and advertises a 60-second default; a watching consumer who wants a different cadence sets one explicitly with config.WithPollInterval, which always wins.

Injecting the client

Wrap is the ordinary path: it adapts the real *azblob.Client to the narrow BlobStore interface, deriving a per-blob client for the two operations the account client cannot express (GetProperties and the copy). For testing, or to supply your own implementation, New takes a BlobStore directly — so the whole unit suite runs against an in-memory fake, with no Azure account:

type BlobStore interface {
	Download(ctx context.Context, name string) ([]byte, error)
	Upload(ctx context.Context, name string, data []byte) error
	GetProperties(ctx context.Context, name string) (BlobInfo, error)
	Copy(ctx context.Context, src, dst string) error
	Delete(ctx context.Context, name string) error
}

fsys := configazureblob.New(myFakeBlobStore)

BlobStore and New are exported, consistent with the sibling App Configuration backend, so a consumer can inject a custom implementation or a fake.

What it costs

The config graph plus the modular Azure Blob SDK (github.com/Azure/azure-sdk-for-go/sdk/storage/azblob and its azcore runtime) — a strikingly small graph, asserted by an allowlist test (depfootprint_test.go) so an unforeseen transitive addition, or the SDK growing one on a version bump, fails the build rather than arriving quietly. In particular azidentity is not here: building the credential is your job, so the adapter never takes on the credential library, and the allowlist guards that boundary explicitly.

Testing against a real store

Unlike App Configuration, Blob Storage has an emulator, so the integration suite under ./test/integration runs against real Azurite in a container started by testcontainers-go. It exercises the real download / upload / get-properties path and the synchronous copy-then-delete rename end to end. It is gated on INT_TEST_INTEGRATION=1 (set only in the Docker-in-Docker CI job) and skips cleanly without a Docker daemon, so the merge gate stays container-free — an in-memory fake driving a real config Store carries it. Run it locally with:

just test-integration

License

See LICENSE.

Documentation

Overview

Package configazureblob adapts an Azure Blob Storage container to the config.FS interface, so a tool whose configuration lives in a blob can hand the container to a config Store and read — and write — it through the same codec and stage-and-rename machinery a local file uses.

cred, _ := azidentity.NewDefaultAzureCredential(nil)          // consumer's dep
client, _ := azblob.NewClient("https://acct.blob.core.windows.net/", cred, nil)

store, err := config.NewStore(ctx,
	config.WithFiles(configazureblob.Wrap(client, "config"), "app/config.yaml"))

reads the blob named app/config.yaml from container config. The blob's name is the file path: it is passed to the object operations unchanged, and the container is bound once, at Wrap.

An object store is not a disk

Azure Blob Storage is reached over HTTPS with an Azure credential. It has no directories (only key prefixes), no atomic rename (only copy-then-delete), no stat (only GetProperties returning object metadata), and no native change notification. The adapter meets it honestly rather than pretending it is a disk:

• Rename is a server-side copy followed by a delete. Each object write is atomic, so the target is still replaced atomically and a reader never observes a half-written blob — but a crash between the copy and the delete leaves the staging blob behind. That is garbage to clean up, never corruption of the target.
• MkdirAll is a no-op — there are no directories to make. The container must already exist; provisioning it is the consumer's job.
• Remove of an absent blob succeeds. This diverges from os.Remove, matching the idempotent intent of discarding staged content.
• A blob has no operating-system path, so the wrapped container is not a config.RealPather and is watched by polling. Each poll is a billed blob read, so the adapter implements config.PollIntervalHinter and advertises a 60-second default (raise it with config.WithPollInterval).

The BlobStore seam

The whole unit suite runs against a fake: Wrap adapts a real *azblob.Client to the narrow BlobStore interface, and New takes any BlobStore, so a consumer can inject a fake or a custom implementation. Building the client — and therefore choosing the credential and endpoint — is the consumer's job, so this package never touches azidentity.

Index

• Variables
• func FSFromConnectionString(conn, container string) (config.FS, error)
• func FSFromCredential(cred azcore.TokenCredential, serviceURL, container string) (config.FS, error)
• func New(store BlobStore) config.FS
• func Wrap(client *azblob.Client, container string) config.FS
• type BlobInfo
• type BlobStore

Examples

• Wrap

Variables

var (
	// ErrNoCredential reports a nil credential supplied to [FSFromCredential].
	//
	// It covers the TYPED nil too: a nil *azidentity.DefaultAzureCredential
	// carried in an [azcore.TokenCredential] interface compares unequal to nil,
	// which is what a dropped constructor error leaves you holding.
	ErrNoCredential = errors.NewSentinel("configazureblob.no_credential",
		"no Azure credential supplied; build one with azidentity, or use the ambient rung")

	// ErrNoServiceURL reports a rung called with no storage-account URL.
	//
	// Unlike an AWS region there is nothing ambient to fall back on: an Azure
	// credential names a principal and carries no endpoint.
	ErrNoServiceURL = errors.NewSentinel("configazureblob.no_service_url",
		"no service URL supplied; pass https://<account>.blob.core.windows.net/")

	// ErrNoContainer reports a rung called with no container name.
	ErrNoContainer = errors.NewSentinel("configazureblob.no_container",
		"no container supplied; pass the container the configuration blob lives in")

	// ErrNoConnectionString reports an empty connection string supplied to
	// [FSFromConnectionString].
	ErrNoConnectionString = errors.NewSentinel("configazureblob.no_connection_string",
		"no connection string supplied; read it from the storage account's access keys")
)

Functions

func FSFromConnectionString

func FSFromConnectionString(conn, container string) (config.FS, error)

FSFromConnectionString builds the client from one of the storage account's access keys.

It sits alongside FSFromCredential rather than beneath it because Blob Storage genuinely has both shapes: a connection string carries the account URL AND the secret together, where a credential names a principal and needs the URL supplied separately.

The connection string is a SECRET — it embeds the account key. It is never logged here, and a malformed one is reported without echoing the input, because that is how a secret reaches a log.

func FSFromCredential

func FSFromCredential(
	cred azcore.TokenCredential, serviceURL, container string,
) (config.FS, error)

FSFromCredential builds the blob client from a credential the caller holds and returns a config.FS over the container.

azblob.NewClient does no I/O — it errors only on a malformed service URL — so this rung constructs eagerly and reaching the container stays deferred to the first read.

Nothing here needs closing

Unlike its Cloud Storage and Secret Manager counterparts, azblob.Client has NO Close: it is HTTP-backed with no connection to release, so this rung returns a plain config.FS rather than a type carrying Close. The obligation follows the SDK — where there is nothing to release, inventing a Close would be ceremony (spec 0012 L-7, R2).

Where the ambient rung is

Deliberately NOT here. Resolving the ambient Azure identity chain costs this module seven further dependencies, so it lives in the ambient subpackage over go/azureclient:

import blobambient "gitlab.com/phpboyscout/go/config-azure-blob/ambient"

fsys, err := blobambient.Default(ctx, serviceURL, "config")

func New

func New(store BlobStore) config.FS

New adapts a BlobStore to a read+write config.FS.

It is the injection seam: pass a fake in tests, or a Wrap-adapted client in production. The returned value satisfies config.FS and, deliberately, neither config.RealPather nor config.LinkReader — an object store has no operating-system path and no symlinks — so the core polls it when watched. It does implement config.PollIntervalHinter, advertising a 60-second poll default.

func Wrap

func Wrap(client *azblob.Client, container string) config.FS

Wrap adapts an Azure Blob container to a read+write config.FS. The client is already configured — its endpoint and credential are the consumer's (this package never touches azidentity) — and container is bound for the lifetime of the returned FS.

It is the convenience path over New: it adapts the account-scoped *azblob.Client to the narrow BlobStore interface, deriving a per-blob client for the two operations the account client cannot express (GetProperties and the copy). A consumer never sees BlobStore unless testing.

Example

ExampleWrap reads configuration from a blob in an Azure Blob Storage container. You build and configure the *azblob.Client — its endpoint and credential stay yours, so azidentity is your dependency, not this package's — and hand it in with the container name. The blob's name is the file path, so the blob app/config.yaml is read as that file and its content parsed by the codec.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
	"gitlab.com/phpboyscout/go/config"

	configazureblob "gitlab.com/phpboyscout/go/config-azure-blob"
)

func main() {
	ctx := context.Background()

	cred, err := azidentity.NewDefaultAzureCredential(nil)
	if err != nil {
		log.Fatal(err)
	}

	client, err := azblob.NewClient("https://acct.blob.core.windows.net/", cred, nil)
	if err != nil {
		log.Fatal(err)
	}

	store, err := config.NewStore(ctx,
		config.WithFiles(configazureblob.Wrap(client, "config"), "app/config.yaml"))
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(store.View().GetInt("server.port"))
}

Types

type BlobInfo

type BlobInfo struct {
	// Size is the blob's content length in bytes.
	Size int64
	// LastModified is the blob's last-modified time, reported as ModTime.
	LastModified time.Time
	// ETag is the opaque entity tag for the blob's current state, available to a
	// consumer who reads GetProperties directly. Conflict detection does not
	// depend on it — the core fingerprints content by SHA-256.
	ETag string
}

BlobInfo is the object metadata [adapter.Stat] synthesises an fs.FileInfo from.

type BlobStore

type BlobStore interface {
	// Download returns the blob's content, or an error satisfying
	// errors.Is(err, fs.ErrNotExist) when the blob is absent.
	Download(ctx context.Context, name string) ([]byte, error)

	// Upload replaces the blob's content in one atomic put.
	Upload(ctx context.Context, name string, data []byte) error

	// GetProperties returns the blob's metadata, or an error satisfying
	// errors.Is(err, fs.ErrNotExist) when the blob is absent.
	GetProperties(ctx context.Context, name string) (BlobInfo, error)

	// Copy replaces dst with a server-side copy of src. The copy is
	// synchronous: it completes within the call for the small blobs a config
	// file is.
	Copy(ctx context.Context, src, dst string) error

	// Delete removes the blob. An absent blob reports an error satisfying
	// errors.Is(err, fs.ErrNotExist); the adapter treats that as already-removed.
	Delete(ctx context.Context, name string) error
}

BlobStore is the slice of Azure Blob Storage this adapter uses, scoped to one container. A fake satisfying it drives the whole unit suite; the real *azblob.Client is adapted to it by Wrap.

Download and GetProperties report an absent blob as an error satisfying errors.Is(err, fs.ErrNotExist); that mapping is load-bearing, because the Store treats a missing optional source as normal and anything else as fatal. A missing container is a misconfiguration, not a normally-absent overlay, so it surfaces as its raw error rather than as fs.ErrNotExist.