configawssecrets

README

config-aws-secrets

Read secrets from AWS Secrets Manager through config

Part of the phpboyscout Go toolkit — small, framework-free Go modules extracted from go-tool-base. Documented with the parent module at config.go.phpboyscout.uk

---

config reads files. AWS Secrets Manager is a sibling backend module like this one, so a consumer who reads secrets from AWS takes it and one who does not pays nothing for it.

You build and configure the AWS client — that is where every credential, region and endpoint decision lives — and hand it in with a prefix that scopes and is stripped from the names:

import (
	"github.com/aws/aws-sdk-go-v2/config" // the AWS one
	"github.com/aws/aws-sdk-go-v2/service/secretsmanager"
	cfg "gitlab.com/phpboyscout/go/config"
	configawssecrets "gitlab.com/phpboyscout/go/config-aws-secrets"
)

awsCfg, _ := config.LoadDefaultConfig(ctx)
client := secretsmanager.NewFromConfig(awsCfg)

store, err := cfg.NewStore(ctx,
	cfg.WithFiles(fsys, "/etc/app.yaml"),                              // YAML defaults
	cfg.WithBackend(configawssecrets.FromClient(client, "app/")),      // secrets outrank them
)

A Secrets Manager layer takes part in precedence, per-key merge, provenance and hot-reload exactly as a file does. Names are paths, so the prefix is stripped and what remains nests:

app/db/password  = "s3cr3t"           store.View().GetString("db.password")  // "s3cr3t"
app/cache/url    = "redis://…"   →    store.View().GetString("cache.url")    // "redis://…"
app/region       = "eu-west-2"        store.View().GetString("region")       // "eu-west-2"

Reading a whole prefix costs one request, not one per secret: BatchGetSecretValue takes the same name filter ListSecrets does and returns the values with it. That is why scoping to a prefix is the natural way to use this adapter rather than something to avoid.

Credentials are yours

This adapter never resolves credentials, reads the environment, or picks a region or endpoint. It uses the client you give it, which is what lets every AWS authentication mechanism — instance roles, IRSA, SSO, static keys, AWS_PROFILE — work without the adapter knowing any of them. It is also why pointing the suite at LocalStack needs no support here: that is a BaseEndpoint on the client.

The IAM policy needs secretsmanager:BatchGetSecretValue and secretsmanager:GetSecretValue for the secrets under your prefix, plus kms:Decrypt on the key they are encrypted with if it is not the AWS-managed one.

A partial read is refused

BatchGetSecretValue does not fail when it cannot read one of the secrets it matched — it returns the successes and reports the rest separately, typically for a secret your IAM policy excludes or whose KMS key you cannot use.

This adapter refuses the load in that case, naming the secrets and their error codes:

// errors.Is(err, configawssecrets.ErrPartialRead)
// configawssecrets: some secrets could not be read under "app/": app/locked (AccessDeniedException)

Serving the layer anyway would mean a configuration silently missing keys it should have — and here a missing key is a password. An application that starts without one fails later, further from the cause, or falls back to a default that is worse than not starting at all. If a secret is genuinely not meant for this application, scope the prefix so it is not matched.

Binary secrets are skipped

A secret holds either a string or binary data. Binary secrets — a certificate, a keystore — are skipped: they contribute no key, and the rest of the prefix loads normally. A View has no useful way to serve them, and base64-encoding one would invent a representation you did not ask for. Unlike a partial read this is not an error, because one certificate sitting in a shared prefix should not stop an unrelated application from starting.

A secrets layer is sensitive, so writes to it are refused

Every value here is a secret, so this backend declares itself Sensitive, and it is read-only: a configuration library writing to a secrets manager is a surprising capability, so it waits for a specification that justifies it.

Those two facts combine into a guard worth understanding before you meet it. Because the layer is read-only, a write to a key it provides cannot land in Secrets Manager — so it would otherwise fall through to the next writable layer, typically a plain YAML file on disk. config refuses that write:

_, err := store.Apply(ctx, cfg.Set("db.password", "rotated"))
// errors.Is(err, cfg.ErrSensitiveLeak) — refused, NOT written to app.yaml

That is the core protecting you from writing a secret into a plaintext file. If a key needs to be writable, do not source it from Secrets Manager.

What it costs

Modules added	14 — 5 for the AWS SDK, 9 for the config graph
Requires	config v0.7.0+

Five modules for the SDK is the leanest of any backend adapter in this toolkit, because the AWS SDK for Go v2 ships per service: reading secrets does not drag in the rest of AWS. Note what is absent — aws-sdk-go-v2/config and credentials are how you build a client, so they are yours, not this module's. The whole figure is pinned by an allowlist test in both directions, so a version bump that widens or narrows the graph fails the build rather than arriving quietly.

Install

go get gitlab.com/phpboyscout/go/config-aws-secrets

Documentation

Full documentation lives with the parent module at config.go.phpboyscout.uk. The Go API reference is on pkg.go.dev.

Licence

MIT

Documentation

Overview

Package configawssecrets contributes secrets from AWS Secrets Manager as a first-class config layer: precedence, per-key merge, provenance and safe hot-reload, exactly like a file layer — but marked sensitive, so the core refuses to write a value sourced from here into a plainer layer beneath.

A Secrets Manager name is a path, so a prefix scopes the backend and the remaining segments nest into the layer's tree: app/db/password under prefix "app/" reads as db.password. The service returns a whole prefix in one bulk request, so scoping a backend to one costs a single call.

A secret's value is an opaque string, so it is a scalar by default. AWS tooling conventionally writes a JSON object into it, and WithValueCodec decodes that into a subtree.

The service is reached through the narrow SecretsAPI interface, which Wrap adapts from a configured *secretsmanager.Client. Injecting the client keeps every credential, region and endpoint decision with the consumer — the adapter never authenticates — and lets the whole unit suite run against a fake, needing no AWS.

See the config-aws-secrets spec for the decisions behind all of this.

Index

• Constants
• Variables
• func FromClient(client *secretsmanager.Client, prefix string, opts ...Option) config.Backend
• func FromClientSecret(client *secretsmanager.Client, name string, codec config.Codec, opts ...Option) config.Backend
• func FromConfig(cfg aws.Config, prefix string, opts ...Option) (config.Backend, error)
• func FromConfigSecret(cfg aws.Config, name string, codec config.Codec, opts ...Option) (config.Backend, error)
• func New(api SecretsAPI, prefix string, opts ...Option) config.Backend
• func NewSecret(api SecretsAPI, name string, codec config.Codec, opts ...Option) config.Backend
• type Failure
• type Option
  • func WithPollInterval(d time.Duration) Option
  • func WithValueCodec(codec config.Codec) Option
  • func WithVersionStage(stage string) Option
• type Secret
• type SecretsAPI
  • func Wrap(client *secretsmanager.Client) SecretsAPI

Constants

const DefaultPollInterval = time.Minute

DefaultPollInterval is how often the backend polls for change when the consumer does not say otherwise.

Sixty seconds. Secrets Manager bills per API call, and a prefix poll is one bulk request, so this is affordable — but the file watcher's two-second cadence would not be.

const SourceKind = config.SourceKind("aws-secrets")

SourceKind is what a layer from this backend reports as, so provenance can name Secrets Manager as the origin of a value.

Variables

var ErrNoRegion = errors.NewSentinel("configawssecrets.no_region",
	"no AWS region in the supplied config; set one, or set AWS_REGION for the ambient rung")

ErrNoRegion reports a configuration that names no AWS region.

A config with no region names a secret in an account nobody chose, so it is refused here rather than failing at the first request. There is no default: AWS documents none, and inventing one is a different act from adopting a provider's own — which is why go/awsclient refuses too.

var ErrPartialRead = errors.NewSentinel("config-aws-secrets.partial_read", "configawssecrets: some secrets could not be read")

ErrPartialRead is returned when the service could read some of the secrets under the prefix but not all of them.

A bulk read reports per-secret failures — an IAM policy that excludes one secret, or a KMS key the caller cannot use — alongside its successes, rather than failing. Serving the layer anyway would mean a configuration silently missing keys it should have, and here a missing key is a password: the application starts, then fails further from the cause, or falls back to a default worse than not starting. So the load is refused, naming what could not be read.

Functions

func FromClient

func FromClient(client *secretsmanager.Client, prefix string, opts ...Option) config.Backend

FromClient returns a backend reading every secret under prefix from a configured Secrets Manager client — the convenience path over New and Wrap.

client must already be configured with credentials and a region. This adapter never resolves credentials, reads the environment or picks an endpoint: every AWS authentication mechanism — instance roles, IRSA, SSO, static keys — works here precisely because the adapter knows about none of them. Pointing at LocalStack is likewise a client-level decision (BaseEndpoint), needing no support here.

func FromClientSecret

func FromClientSecret(
	client *secretsmanager.Client, name string, codec config.Codec, opts ...Option,
) config.Backend

FromClientSecret returns a backend reading ONE secret, whose value is a whole document, from a configured client — the convenience path over NewSecret and Wrap. The same client rules as FromClient apply.

func FromConfig

func FromConfig(cfg aws.Config, prefix string, opts ...Option) (config.Backend, error)

FromConfig builds the Secrets Manager client from an aws.Config the caller resolved — a specific profile, an assumed role, a client pointed at LocalStack — and returns a backend over the prefix.

secretsmanager.NewFromConfig does no I/O, returns no error and needs no close, so this rung constructs eagerly and reaching Secrets Manager stays deferred to Load, which has a context to bound it.

Where the ambient rung is

Deliberately NOT here. Resolving the ambient AWS chain costs this module ten further dependencies, and charging every consumer for a credential graph they may never use would spend the dependency-footprint guarantee this adapter states in its own test. It lives in the ambient subpackage instead:

import secretsambient "gitlab.com/phpboyscout/go/config-aws-secrets/ambient"

b, err := secretsambient.Default(ctx, "app/")

To share ONE resolved chain across several adapters — and across go/signing and go/encryption — resolve it with go/awsclient and pass the config here (spec 0012 L-4, L-5).

func FromConfigSecret

func FromConfigSecret(
	cfg aws.Config, name string, codec config.Codec, opts ...Option,
) (config.Backend, error)

FromConfigSecret is FromConfig for the single-secret shape: one secret whose payload is a whole document, decoded through the supplied codec.

func New

func New(api SecretsAPI, prefix string, opts ...Option) config.Backend

New returns a backend contributing every secret under prefix as one config layer: the prefix is stripped and the remaining name segments nest.

The service returns a whole prefix in a single bulk request, so there is no per-secret cost to avoid.

api is the injected client: a fake in tests, a Wrap-ped *secretsmanager.Client in production.

func NewSecret

func NewSecret(api SecretsAPI, name string, codec config.Codec, opts ...Option) config.Backend

NewSecret returns a backend contributing ONE secret, whose value is a whole document, as a config layer. It is the shape AWS's own tooling produces — an RDS-managed secret is a JSON object of username, password, host and port.

The codec is a parameter rather than an option because it is not optional here: a single secret's value is one opaque string, so without something to decode it there is no tree to contribute. Pass configjson.Codec{} for the usual JSON secret. In prefix mode the codec stays optional (WithValueCodec), because there the names supply the structure.

Types

type Failure

type Failure struct {
	Name    string
	Code    string
	Message string
}

Failure is one secret the service matched but could not return.

type Option

type Option func(*backend)

Option configures a backend.

func WithPollInterval

func WithPollInterval(d time.Duration) Option

WithPollInterval sets how often the backend polls for change. The default is DefaultPollInterval.

func WithValueCodec

func WithValueCodec(codec config.Codec) Option

WithValueCodec decodes each secret's value through codec: a value that decodes to a mapping becomes a subtree at its path, and a value the codec rejects — a bare password, say — stays a scalar string, so a prefix mixing plain secrets and JSON blobs reads correctly.

AWS's own tooling writes JSON objects into a secret's string (the console's key/value editor, and the RDS and Redshift managed secrets), which is what this exists for. codec is any config.Codec, so a JSON store is read with configjson.Codec{} and this module takes no codec dependency of its own.

func WithVersionStage

func WithVersionStage(stage string) Option

WithVersionStage reads a staging label other than AWSCURRENT — AWSPREVIOUS during a rotation, say, or a custom label.

It is not free, and the cost is worth knowing before you reach for it. The bulk value API has no staging parameter, so a staged prefix read names the secrets and then fetches each one: a prefix of n secrets costs one list plus n reads, where the default costs a single request. Every watch poll pays it too.

type Secret

type Secret struct {
	// Name is the full Secrets Manager name, prefix included.
	Name string

	// Value is the secret string. Empty when Binary is set.
	Value string

	// Binary reports that the secret held binary data rather than a string.
	// Such a secret contributes no key to the layer.
	Binary bool

	// Version is the VersionId of what was read — the marker the poll compares
	// to notice a rotation or an out-of-band write.
	Version string
}

Secret is one secret's current value.

type SecretsAPI

type SecretsAPI interface {
	// Get reads one secret at the given staging label; an empty stage means
	// AWSCURRENT. It returns a nil Secret and a nil error when the secret does
	// not exist, so absence is not an error.
	Get(ctx context.Context, name, stage string) (*Secret, error)

	// List reads every secret whose name begins with prefix, at AWSCURRENT, in
	// as few requests as the service allows.
	//
	// Failures are per-secret and returned separately rather than folded into
	// err: the service reports them that way, and the caller decides what a
	// partial read means. err is for a failure of the call itself.
	List(ctx context.Context, prefix string) (secrets []Secret, failures []Failure, err error)

	// Names lists the secret names under prefix without reading their values.
	//
	// This exists only for the staged read. The bulk value API has no staging
	// parameter — verified against the SDK — so reading anything other than
	// AWSCURRENT means naming the secrets first and fetching each one.
	Names(ctx context.Context, prefix string) ([]string, error)
}

SecretsAPI is the slice of Secrets Manager this adapter uses, behind an interface it owns so a fake drives the unit suite and the real client is adapted by Wrap.

func Wrap

func Wrap(client *secretsmanager.Client) SecretsAPI

Wrap adapts a real Secrets Manager client to the narrow SecretsAPI interface.