configvault

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 17, 2026 License: MIT Imports: 12 Imported by: 0

README

config-vault

Read secrets from HashiCorp Vault through config

Go Reference Pipeline Coverage phpboyscout Go toolkit

Part of the phpboyscout Go toolkit — small, framework-free Go modules extracted from go-tool-base. Documented with the parent module at config.go.phpboyscout.uk

config reads files. HashiCorp Vault is a sibling backend module like this one, so a consumer who reads secrets from Vault takes it and one who does not pays nothing for it.

You build and authenticate the Vault client — that is where every auth method, address, namespace and TLS decision lives — and hand it in with the KV v2 mount and the secret path:

import (
	vaultapi "github.com/hashicorp/vault/api"
	"gitlab.com/phpboyscout/go/config"
	configvault "gitlab.com/phpboyscout/go/config-vault"
)

client, _ := vaultapi.NewClient(vaultapi.DefaultConfig())
client.SetToken(token)

store, err := config.NewStore(ctx,
	config.WithFiles(fsys, "/etc/app.yaml"),                            // YAML defaults
	config.WithBackend(configvault.FromClient(client, "secret", "app")), // Vault outranks them
)

A Vault layer takes part in precedence, per-key merge, provenance and hot-reload exactly as a file does. The secret's fields become the layer, keeping the structure Vault stores them with:

secret/app = {                          store.View().GetString("db.host") // "db.internal"
  "db": { "host": "db.internal",   →    store.View().GetInt("db.port")    // 5432
          "port": 5432 } }

Unlike the byte-valued stores this family also adapts, Vault returns already-structured JSON — nested maps, slices, booleans and nulls all survive the round trip — so there is no value codec here and none is needed.

Two modes: one secret, or a tree

FromClient reads one secret — the common shape, one API call, one read grant in your Vault policy. When configuration genuinely spans a tree of secrets, FromClientPrefix walks a prefix recursively, stripping it and nesting each secret's fields under its remaining path segments:

secret/app        = { "name": "checkout" }         name: checkout
secret/app/db     = { "user": "admin" }       →    db:    { user: admin }
secret/app/cache  = { "url": "redis://…" }         cache: { url: "redis://…" }

config.WithBackend(configvault.FromClientPrefix(client, "secret", "app"))

Prefix mode is opt-in rather than the default because it costs one request per directory plus one per secret — there is no recursive list in Vault — and it needs the list capability on every directory it descends, which least-privilege policies often withhold. Each poll of the watch pays that same cost again.

A field and a child secret cannot share a name

[!WARNING] In Vault a path is both a secret and a directory. In prefix mode that means a secret's own fields and its child secrets merge into the same node — and can collide. This adapter refuses the load rather than guessing.

Given a field db on secret/app, and a secret at secret/app/db:

secret/app     = { "db": "postgres://…" }     ← a field named "db"
secret/app/db  = { "host": "db.internal" }    ← a child secret also named "db"

both claim db, and there is no correct silent answer — either resolution throws away a value you can see in Vault. So the load fails:

_, err := config.NewStore(ctx, config.WithBackend(configvault.FromClientPrefix(client, "secret", "app")))
// errors.Is(err, configvault.ErrFieldCollision)
// configvault: secret field collides with a child secret: "app" holds a field "db" and a child secret "app/db"

Fix it in Vault, whichever way suits: rename the field, or move the child secret. The error names the path and the segment so you know which two to look at.

This is the same choice the sibling adapters make for ambiguous structure — config-xml refuses an attribute colliding with a child element — because an ambiguous merge is a defect in the source, and hiding it is worse than failing at startup.

Single-secret mode cannot produce this, having only one flat map of fields. That is a further reason it is the default.

Authentication is yours, and so is renewal

This adapter never authenticates. It has no auth method, no address, no token handling and no credential anywhere in its API — it uses the client you give it. That is what lets every Vault auth method work without the adapter knowing any of them, and it keeps credentials out of a configuration library's surface area.

Every auth method ends the same way: an authenticated *api.Client goes into FromClient.

// AppRole — the common service-to-service path.
appRole, _ := approle.NewAppRoleAuth(roleID, &approle.SecretID{FromEnv: "APPROLE_SECRET_ID"})
client.Auth().Login(ctx, appRole)

// Kubernetes — the common in-cluster path.
k8s, _ := kubernetes.NewKubernetesAuth("your-role")
client.Auth().Login(ctx, k8s)

// …then, either way:
config.WithBackend(configvault.FromClient(client, "secret", "app"))

[!IMPORTANT] Vault tokens expire, and this adapter does not renew them. When a token lapses, reads through this backend start failing — reloads included. That is Vault rejecting the token, not a defect here. If your process outlives its token's TTL, renew it, with the SDK's LifetimeWatcher, Vault Agent, or whatever your platform provides:

watcher, _ := client.NewLifetimeWatcher(&vaultapi.LifetimeWatcherInput{Secret: login})
go watcher.Start()
defer watcher.Stop()

Runnable versions of all of these are in the package examples — they are compiled by this module's test suite, so they cannot drift from the API.

The auth helper packages (vault/api/auth/approle, .../kubernetes) are separate modules. Add whichever you use; this module does not depend on any of them.

A Vault layer is sensitive, so writes to it are refused

Every value in Vault is a secret, so this backend declares itself Sensitive, and it is read-only: a configuration library writing to a secrets manager is a surprising capability, so it waits for a specification that justifies it.

Those two facts combine into a guard worth understanding before you meet it. Because the layer is read-only, a write to a key it provides cannot land in Vault — so it would otherwise fall through to the next writable layer, typically a plain YAML file on disk. config refuses that write:

err := store.Set("db.password", "rotated")
// errors.Is(err, config.ErrSensitiveLeak) — refused, NOT written to app.yaml

That is the core protecting you from writing a secret into a plaintext file. If a key needs to be writable, do not source it from Vault.

Vault rounds integers above 2^53

A limitation of Vault, not of this adapter, but it is silent so it is worth knowing. Vault decodes a submitted JSON number through a float, so an integer larger than 2​^53 (9007199254740992) is rounded on write — before this adapter ever sees the value:

kv.Put(ctx, "app", map[string]any{"id": 9007199254740993})
store.View().Get("id")  // 9007199254740992  ← Vault stored the rounded value

If you keep large identifiers or nanosecond timestamps in Vault, store them as strings, which round-trip exactly. This adapter converts integers as int64 rather than through a float, so it adds no further loss — but it cannot recover precision Vault has already discarded.

Namespaces, and which engine

Vault Enterprise namespaces are set on the client (client.SetNamespace(…)), so this adapter is namespace-agnostic and takes no parameter for it.

The KV v2 engine only — Vault's default and recommended secrets engine. KV v1 is legacy, has no version metadata, and lists through a different path; support would be a second, quietly weaker code path, so it waits for someone to need it.

What it costs
Modules added 26 — 17 for the Vault SDK, 9 for the config graph
Requires config v0.7.0+

A backend adapter carries its system's client, and the Vault SDK is the largest thing here. That cost is pinned by an allowlist test, so a version bump that widens the graph fails the build rather than arriving quietly.

Install

go get gitlab.com/phpboyscout/go/config-vault

Documentation

Full documentation lives with the parent module at config.go.phpboyscout.uk. The Go API reference is on pkg.go.dev.

Licence

MIT

Documentation

Overview

Package configvault contributes secrets from a HashiCorp Vault KV v2 store as a first-class config layer: precedence, per-key merge, provenance and safe hot-reload, exactly like a file layer — but marked sensitive, so the core refuses to write a value sourced from here into a plainer layer beneath.

A Vault secret is a path holding a map of fields, and each field holds arbitrary JSON. In the default single-secret mode (New) one secret's fields become the layer; NewPrefix opts into a recursive walk over a whole tree of secrets. Values keep the structure Vault returns them with — nested maps and slices included — so no value codec is needed here, unlike the byte-valued stores this family also adapts.

Vault is reached through the narrow KV interface, which Wrap adapts from a configured *api.Client. Injecting the client keeps every credential, auth method, namespace and TLS decision with the consumer — the adapter never authenticates — and lets the whole unit suite run against a fake, needing no Vault. Tokens expire: renewing them is the consumer's job, and a lapsed token surfaces as a Load error rather than being swallowed.

See the config-vault spec for the decisions behind all of this.

Example (AppRoleAuth)

Example_appRoleAuth authenticates with AppRole, the common service-to-service method, before handing the client over.

Note the shape: every auth method ends the same way — an authenticated *api.Client goes into FromClient. The adapter knows about none of them, which is exactly why they all work. 

package main

import (
	"context"
	"fmt"
	"log"

	vaultapi "github.com/hashicorp/vault/api"
	auth "github.com/hashicorp/vault/api/auth/approle"
	"gitlab.com/phpboyscout/go/config"

	configvault "gitlab.com/phpboyscout/go/config-vault"
)

func main() {
	client, err := vaultapi.NewClient(vaultapi.DefaultConfig())
	if err != nil {
		log.Fatal(err)
	}

	appRole, err := auth.NewAppRoleAuth("your-role-id",
		&auth.SecretID{FromEnv: "APPROLE_SECRET_ID"})
	if err != nil {
		log.Fatal(err)
	}

	if _, err = client.Auth().Login(context.Background(), appRole); err != nil {
		log.Fatal(err)
	}

	store, err := config.NewStore(context.Background(),
		config.WithBackend(configvault.FromClient(client, "secret", "app")),
	)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(store.View().GetString("db.host"))
}
Example (KubernetesAuth)

Example_kubernetesAuth authenticates with the Kubernetes method, the common in-cluster path: the pod's service-account token is exchanged for a Vault one. 

package main

import (
	"context"
	"fmt"
	"log"

	vaultapi "github.com/hashicorp/vault/api"

	k8sauth "github.com/hashicorp/vault/api/auth/kubernetes"
	"gitlab.com/phpboyscout/go/config"

	configvault "gitlab.com/phpboyscout/go/config-vault"
)

func main() {
	client, err := vaultapi.NewClient(vaultapi.DefaultConfig())
	if err != nil {
		log.Fatal(err)
	}

	k8s, err := k8sauth.NewKubernetesAuth("your-role")
	if err != nil {
		log.Fatal(err)
	}

	if _, err = client.Auth().Login(context.Background(), k8s); err != nil {
		log.Fatal(err)
	}

	store, err := config.NewStore(context.Background(),
		config.WithBackend(configvault.FromClient(client, "secret", "app")),
	)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(store.View().GetString("db.host"))
}
Example (TokenRenewal)

Example_tokenRenewal keeps a client's token alive for the life of a long-running process.

This adapter does not renew tokens, and nothing else will do it for you. A Vault token has a TTL; when it lapses, reads through this backend start failing — reloads and the poll watch included — and the failure is Vault rejecting the token, not a defect in the adapter. If your process outlives its token's TTL, run something like this. 

package main

import (
	"context"
	"fmt"
	"log"

	vaultapi "github.com/hashicorp/vault/api"
	auth "github.com/hashicorp/vault/api/auth/approle"
	"gitlab.com/phpboyscout/go/config"

	configvault "gitlab.com/phpboyscout/go/config-vault"
)

func main() {
	client, err := vaultapi.NewClient(vaultapi.DefaultConfig())
	if err != nil {
		log.Fatal(err)
	}

	appRole, err := auth.NewAppRoleAuth("your-role-id",
		&auth.SecretID{FromEnv: "APPROLE_SECRET_ID"})
	if err != nil {
		log.Fatal(err)
	}

	login, err := client.Auth().Login(context.Background(), appRole)
	if err != nil {
		log.Fatal(err)
	}

	watcher, err := client.NewLifetimeWatcher(&vaultapi.LifetimeWatcherInput{Secret: login})
	if err != nil {
		log.Fatal(err)
	}

	go watcher.Start()
	defer watcher.Stop()

	// The same client, now kept alive, backs the config layer for as long as the
	// process runs. Returning rather than log.Fatal-ing from here on, so the
	// watcher's Stop actually runs.
	store, err := config.NewStore(context.Background(),
		config.WithBackend(configvault.FromClient(client, "secret", "app")),
	)
	if err != nil {
		fmt.Println("config:", err)

		return
	}

	fmt.Println(store.View().GetString("db.host"))
}

Index

Examples

Constants

View Source 
const DefaultPollInterval = time.Minute

DefaultPollInterval is how often a Vault backend polls for change when the consumer does not say otherwise.

Sixty seconds, deliberately slower than the file watcher's cadence: every poll is an authenticated read that lands in Vault's audit log, and a two-second cadence would flood a stream that exists to be readable. WithPollInterval overrides it.

View Source 
const SourceKind = config.SourceKind("vault")

SourceKind is what a layer from this backend reports as, so provenance can name Vault as the origin of a value.

Variables

View Source 
var (
	// ErrNoConfig reports a nil config supplied to a FromConfig rung.
	ErrNoConfig = errors.NewSentinel("configvault.no_config",
		"no Vault config supplied; pass one, or use Default to read the ambient environment")

	// ErrNoAddress reports a config that names no Vault address.
	//
	// It cannot arise from the Default rungs, which inherit Vault's own
	// documented 127.0.0.1:8200, but a caller assembling a *vaultapi.Config by
	// hand can leave it empty — and an empty address fails at the first request
	// rather than here.
	ErrNoAddress = errors.NewSentinel("configvault.no_address",
		"no Vault address configured; set Address on the config or VAULT_ADDR")
)
View Source 
var ErrFieldCollision = errors.NewSentinel("config-vault.field_collision", "configvault: secret field collides with a child secret")

ErrFieldCollision is returned when a secret's field and a child secret claim the same name, so the layer cannot represent both.

In Vault a path is simultaneously a secret and a directory, so in prefix mode a secret's own fields merge with its child secrets into one node. If secret "app" holds a field "db" while secret "app/db" also exists, both claim "db" and either resolution silently discards a value the operator can see in Vault. Refusing is the honest outcome — the same choice config-xml makes for an attribute colliding with a child element.

Resolve it in Vault, by renaming the field or moving the child secret. Single- secret mode cannot produce this, having only one flat map of fields.

Functions

func Default added in v0.4.0 

func Default(mount, secretPath string, opts ...Option) (config.Backend, error)

Default builds a single-secret backend from the ambient Vault environment — VAULT_ADDR, VAULT_TOKEN, VAULT_NAMESPACE and the rest — as vaultapi.DefaultConfig reads them.

This is the zero-conf rung. The defaults are Vault's own, not this adapter's: with nothing set, DefaultConfig resolves https://127.0.0.1:8200, which is what every Vault client on the machine does. Adopting a provider's documented default is a different act from inventing one, which is why config-etcd has no rung here at all (spec 0012 R1).

To share ONE Vault client across this adapter, go/signing and go/encryption, use gitlab.com/phpboyscout/go/vaultclient and hand the client it yields to FromClient. Vault is the provider where the client itself is the connection prerequisite, so no further seam is needed.

The token is read once, and this rung does not renew it

vaultapi.NewClient reads VAULT_TOKEN a single time and holds it for the life of the client; nothing re-reads the environment and nothing renews the lease. A backend built here therefore carries whatever token was set when it was built, for as long as the Store holds it.

That is fine for a command, which outlives its token rarely. It is NOT fine for a long-lived process reloading configuration: once the token's TTL expires every subsequent Load fails with a permission error and there is no path back, because the backend has no way to learn its credential lapsed.

This is a property of Vault's client rather than of this rung — FromClient has always behaved the same way — but the zero-conf rung is where it is easiest to reach without noticing, precisely because the caller never handles the token. If the process outlives the TTL, build the client yourself, keep a vaultapi.LifetimeWatcher renewing it, and pass it to FromClient.

func DefaultPrefix added in v0.4.0 

func DefaultPrefix(mount, prefix string, opts ...Option) (config.Backend, error)

DefaultPrefix is Default for the prefix shape.

func FromClient 

func FromClient(client *vaultapi.Client, mount, secretPath string, opts ...Option) config.Backend

FromClient returns a backend reading one secret from a configured Vault client — the convenience path over New and Wrap.

client must already be authenticated. This adapter never authenticates, never renews, and never reads a token from the environment: every auth method Vault supports works here precisely because the adapter knows about none of them. Vault tokens expire, and renewing them (with the SDK's LifetimeWatcher, Vault Agent, or whatever the platform provides) is the consumer's job — a lapsed token surfaces as a Load error.

mount is the KV v2 mount, conventionally "secret"; path is the secret within it, without the mount or the engine's internal "data"/"metadata" segments.

Example

ExampleFromClient reads configuration from a Vault KV v2 secret.

You build and authenticate the Vault client — address, auth method, namespace and TLS all stay yours — and hand it in with the mount and the secret path. The secret's fields become the layer, so a secret holding {"host": …} at secret/app reads back as db.host when it lives under a "db" field. 

package main

import (
	"context"
	"fmt"
	"log"

	vaultapi "github.com/hashicorp/vault/api"
	"gitlab.com/phpboyscout/go/config"

	configvault "gitlab.com/phpboyscout/go/config-vault"
)

func main() {
	client, err := vaultapi.NewClient(vaultapi.DefaultConfig())
	if err != nil {
		log.Fatal(err)
	}

	// The simplest auth there is: a token you already hold. VAULT_TOKEN in the
	// environment is picked up by DefaultConfig, so this line is only needed
	// when you source the token yourself.
	client.SetToken("s.your-token")

	store, err := config.NewStore(context.Background(),
		config.WithBackend(configvault.FromClient(client, "secret", "app")),
	)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(store.View().GetInt("db.port"))
}

func FromClientPrefix added in v0.2.0 

func FromClientPrefix(client *vaultapi.Client, mount, prefix string, opts ...Option) config.Backend

FromClientPrefix returns a backend reading every secret beneath prefix from a configured Vault client — the convenience path over NewPrefix and Wrap.

The same authentication rules apply as for FromClient: client must already be authenticated, and keeping it renewed is the consumer's job. Prefix mode additionally needs the "list" capability on every directory it descends, and costs one request per directory plus one per secret.

func FromConfig added in v0.4.0 

func FromConfig(cfg *vaultapi.Config, mount, secretPath string, opts ...Option) (config.Backend, error)

FromConfig builds the client from a config the caller assembled and returns a single-secret backend over it.

func FromConfigPrefix added in v0.4.0 

func FromConfigPrefix(cfg *vaultapi.Config, mount, prefix string, opts ...Option) (config.Backend, error)

FromConfigPrefix is FromConfig for the prefix shape: every secret under the prefix becomes one layer.

func New 

func New(kv KV, path string, opts ...Option) config.Backend

New returns a backend contributing one Vault secret as a config layer: the secret's fields become the layer's values.

This is the common shape — one secret holds one application's configuration — and it costs exactly one API call and one `read` capability in the consumer's Vault policy.

kv is the injected client: a fake in tests, a Wrap-ped *api.Client in production.

func NewPrefix added in v0.2.0 

func NewPrefix(kv KV, prefix string, opts ...Option) config.Backend

NewPrefix returns a backend contributing every Vault secret beneath prefix as one config layer: the prefix is stripped, and each secret's fields nest under its remaining path segments.

This is the opt-in mode. It costs one API call per directory plus one per secret, and it needs the "list" capability on every directory it descends — which least-privilege Vault policies often withhold. Prefer New unless the configuration genuinely spans a tree of secrets.

A field colliding with a child secret refuses the Load with ErrFieldCollision.

Types

type KV 

type KV interface {
	// Get reads the latest version of the secret at path. It returns a nil
	// Secret and a nil error when the secret is absent, soft-deleted or
	// destroyed — three states Vault reports differently but which mean the
	// same thing to a reader: there is nothing there.
	Get(ctx context.Context, path string) (*Secret, error)

	// List returns the immediate children of a directory. Child directories
	// carry a trailing "/", and a path that is both a secret and a directory
	// appears twice — once each way. An absent directory returns nil, nil.
	//
	// Only the prefix mode calls this, so a single-secret consumer may supply
	// a KV whose List is never reached.
	List(ctx context.Context, dir string) (children []string, err error)
}

KV is the slice of Vault this adapter uses, behind an interface it owns so a fake drives the unit suite and the real client is adapted by Wrap.

func Wrap 

func Wrap(client *vaultapi.Client, mount string) KV

Wrap adapts a real Vault client to the narrow KV interface.

The mount belongs here rather than in the secret paths because KV v2 splits its endpoints — reads go through the typed KVv2 helper while listing goes to the mount's "metadata" tree — and that split is an artefact of the engine that the adapter's own logic should not have to know about.

type Option 

type Option func(*backend)

Option configures a backend.

func WithPollInterval added in v0.2.0 

func WithPollInterval(d time.Duration) Option

WithPollInterval sets how often the backend polls Vault for change.

Vault has no change feed, so watching is polling (D10) and the interval is a direct trade between how fresh configuration is and how much audit-log traffic and load the process generates. The default is DefaultPollInterval.

type Secret 

type Secret struct {
	// Data holds the secret's fields. Vault preserves JSON structure, so a
	// value may be a scalar, a nested map or a slice. Nil or empty means the
	// secret is deleted or holds nothing.
	Data map[string]any

	// Version is the KV v2 version. It is the marker the poll watch compares
	// to notice a foreign change.
	Version int
}

Secret is one version of one Vault secret: its fields, and the KV v2 version that identifies this revision of them.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.