otelflags

README

otel-flags

The feature-switch layer shared by this repository's OpenTelemetry instrumentation modules.

It emits no spans and wraps no client. Applications do not normally import it — otel-mongo, otel-mongo/v2, otel-nats and otel-gorilla-ws do, and configuration reaches it through environment variables and constructor options. Import it directly only to match on ErrInvalidFlagValue.

For the operator-facing reference — every switch, every variable, worked examples and the relay wiring — see docs/feature-flags.md (繁體中文). Hands-on tutorial: docs/otel-nats-kill-switch.en-US.html.

The precedence ladder

Every switch resolves down four rungs, first source with an opinion winning:

relay  >  env  >  option (With*Enabled)  >  hardcoded default

The order is by how late each source is decided: compiled in, written when the wrapper is constructed, set when the process is deployed, changed while it runs. Each later stage overrides the earlier ones.

The option sits below its environment variable deliberately. A deployment must be able to disable one module without silencing the process and without a relay, even when the application's Go code asked for it. The case that forces it is otel-mongo's document propagation, which appends a permanent field to the operator's own documents — every other switch merely produces or withholds telemetry.

The whole ladder is one call. Client.Boolean returns the value passed to it on every path where the relay has no usable answer — no provider, not ready, key absent, evaluation error, type mismatch — so this package hands it the already-resolved local value and lets the SDK perform the fallback.

The switches

Switch	Relay key	Option	Environment variable	Default
master	otel-instrumentation-go-tracing	—	OTEL_INSTRUMENTATION_GO_TRACING_ENABLED	true
per-module tracing	otel-<module>-tracing	WithTracingEnabled	OTEL_<MODULE>_TRACING_ENABLED	false
Mongo propagation	otel-mongo-propagation	WithTracePropagationEnabled	OTEL_MONGO_PROPAGATION_ENABLED	false

They compose by conjunction: tracing = master && moduleTracing, and propagation = tracing && mongoPropagation.

The master defaults to enabled because it is a veto, not an enabler. Setting it truthy — in the environment or on the relay — changes nothing. The only value with an effect is false, which stops every module in the process, including connections whose Go code passed an option. Do not document it as an enable; it will read as a broken flag.

Nothing turns on because the master is true. The per-module default of false is what keeps a zero-configuration process silent.

Environment values are a strict tri-state

Lookup has three outcomes and only three:

Value	Outcome
unset	no opinion — resolution falls through to the option, then the default
1 true yes on / 0 false no off (trimmed, case-insensitive)	this source decides
anything else, including the empty string	ErrInvalidFlagValue — the constructor fails

Guessing is prohibited because under a ladder there is no safe direction to guess in: the master tier defaults to true and every other tier to false, so a value silently read as false would stop a whole fleet on one tier and change nothing on the others.

export VAR= is invalid for the same reason. Both readings are wrong somewhere — as false it lets an unexpanded ${SOMETHING} template variable express an opinion the deployment never had; as unset it silently reverses meaning for anyone who used it as an off switch. The rule has no exceptions: set it to a recognised value, or do not set it.

Relay control without Go code

Set three environment variables and nothing else:

Variable	Meaning
OTEL_INSTRUMENTATION_GO_FLAGS_ENDPOINT	GO Feature Flag relay proxy URL, with a scheme and a host (http://relay:1031). Unset ⇒ nothing is installed, no OpenFeature state is written, RelayPossible() is false
OTEL_INSTRUMENTATION_GO_FLAGS_API_KEY	optional; never logged
OTEL_INSTRUMENTATION_GO_FLAGS_POLL_INTERVAL	optional; a positive Go duration, default 60s. A value that cannot be read fails construction — see below. It sets the centre of the polling period: the effective interval is deviated by at most ±10%, drawn once per process
OTEL_SERVICE_NAME	optional; supplies serviceName and service.name targeting attributes, on this path only. Relay rules must key on the dot-free spelling — a dot is a nested-path separator in both query languages

The install fires only when the application has bound no provider to otel-instrumentation-go, and registers a named provider on that domain. DataCollectorDisabled: true and in-process evaluation are hardcoded, so the zero-code path cannot be misconfigured into the stall those two settings otherwise cause during a relay outage.

A provider the application installed in the default slot — its own feature flags — is not treated as a relay for these switches: it does not make RelayPossible() true, it never has an instrumentation key evaluated against it, and it does not stand this install down.

It happens during construction, in ValidateAndInstall, which every wrapper constructor calls and whose error it joins with its own. It does not wait for the provider's first fetch — the startup window below is the consequence — but it is off the evaluation path, so no instrumented operation can be parked behind a provider initialisation.

A value nobody can read fails construction

The two variables above are validated whether or not a relay is configured, and an unreadable one fails the constructor with an error wrapping ErrInvalidFlagValue:

Variable	Rejected	Why not warn and fall back
…_POLL_INTERVAL	60, soon, 0s, -5s	a bare integer read as milliseconds turns 60 into 60ms, and a fleet polling the relay sixty times a second is worse than a failed startup
…_ENDPOINT	relay:1031, relay, http://	relay:1031 parses cleanly — as scheme relay with opaque 1031 — and yields a provider that can never reach anything

Blank means "not configured" for both. Unlike a boolean, a duration and a URL have no second reading for export VAR= to be misread as, which is why Lookup rejects it and these do not. The API key is never validated: any string can be a legitimate key, and it never appears in an error or a log line.

Installing your own provider

if err := otelflags.SetNamedProvider(provider); err != nil {
    slog.Warn("feature flag provider registration failed", "error", err)
}

SetNamedProvider binds any OpenFeature provider to FlagDomain, waits for it to initialise — so there is no startup window — and records that this process installed one. That record is what makes detection exact: openfeature.NamedProviderMetadata falls back to the default provider's metadata when the domain is unbound, so a heuristic alone cannot distinguish an application that bound the same provider to both slots. Sharing one provider instance between your own flags and these switches requires this function, for exactly that reason.

Raw openfeature.SetNamedProviderAndWait(otelflags.FlagDomain, p) remains supported and is still detected.

Install before constructing any wrapper — RelayPossible is resolved at construction, and a wrapper built earlier resolves statically for the rest of its life. SetNamedProvider warns when it can see that this happened; a raw binding cannot be detected, so nothing warns there.

Alarming on a dead relay

There is no health API here. Read the state the SDK already maintains:

state := openfeature.NewClient(otelflags.FlagDomain).State()

Do not gate startup on it. A process that refuses to start until the relay answers has made a telemetry control plane an availability dependency, which is the outcome every other decision in this module exists to avoid.

What a running process reports instead is a log line, and only when something changes. Value reads the evaluation's error code and reports a transition, never an evaluation:

Code	Level	Meaning
FLAG_NOT_FOUND, PROVIDER_NOT_READY	debug	the relay has no opinion — an ordinary state, and the only signal available to someone who mistyped a key name
TARGETING_KEY_MISSING, TYPE_MISMATCH, PARSE_ERROR, INVALID_CONTEXT, PROVIDER_FATAL, GENERAL	warn	something is broken and the relay cannot change this switch
the code clears	info	the relay decides this switch again

Things worth knowing

• A flag change is not immediate. End-to-end latency is the provider's poll interval, 60 s by default, deviated by at most ±10% so that a fleet does not poll the relay on a shared period. The deviation is drawn once per process and is the only delay this module adds; it follows the same rule as the relay proxy's own enablePollingJitter one hop further up. The provider's first fetch, during initialisation, is deliberately not delayed — see docs/feature-flags.md.
• Nothing is evaluated unless a provider is bound to FlagDomain. The SDK's ForEvaluation falls back to the DEFAULT provider for an unbound domain, so evaluating regardless would resolve instrumentation keys against the application's own flag backend — a network call per instrumented operation if that backend is remote, and a wrong answer outright if it defines a key by the same name. Value and MasterEnabled short-circuit to the local value instead.
• An evaluation against a provider you installed is bounded at 250 ms, after which the local value decides. The auto-installed provider evaluates in process and skips the deadline entirely. The caller's context is deliberately not threaded through: cancelling a Mongo operation must not change what an instrumentation switch resolves to.
• An instrumented operation makes two evaluations (three on a Mongo write), and pays for them whatever the flag's value — only a process where no relay is possible skips the pipeline. Order of magnitude on developer hardware: single-digit microseconds each; this repository ships no benchmark, so measure on your own workload. Nothing is cached; a cache would fit inside Resolver without changing Value's signature.
• This module never touches the default provider, the global evaluation context, hooks or shutdown — the same rule the instrumentation packages follow for TracerProvider.
• Nothing shuts the auto-installed provider down. One poller goroutine per process, ending with the process. An application needing lifecycle control installs its own provider.

Documentation

Overview

Package otelflags resolves the feature switches that govern this repository's OpenTelemetry instrumentation modules.

The precedence ladder

Every switch is resolved down four rungs, first source with an opinion winning:

relay  >  env  >  option (With*Enabled)  >  hardcoded default

The ordering is by how late in the pipeline each source is decided — compiled in, written when the wrapper is constructed, set when the process is deployed, changed while it runs — so each later stage overrides the earlier ones. That is why the per-connection option sits BELOW its environment variable: a deployment must be able to disable one module without silencing the process and without a relay, even when the application's Go code asked for it. The case that forces the order is otel-mongo's document propagation, which appends a permanent field to the operator's own documents.

The whole ladder is one call. Client.Boolean returns the value passed to it on every path where the relay has no usable answer — no provider installed, not ready, key absent, evaluation error, type mismatch — so this package hands it the already-resolved local value and lets the SDK perform the fallback. Relay silence and relay failure are deliberately indistinguishable: both mean "the next rung down decides".

What lives here and what does not

Every name this file defines is PROCESS-scoped: the master switch, the three provider variables, the service-name attribute, the OpenFeature domain. Module flag keys, module environment variable names and module defaults belong to the module that owns them and reach this package only as the arguments of Value. Adding an instrumentation module must not require a change here.

The two entry points

ValidateAndInstall runs at construction: it validates this package's own environment, fails the constructor on anything it cannot read, and performs the one-time provider install. Value runs per operation and only evaluates. Keeping the install off the evaluation path is what stops an instrumented operation from parking on a provider initialisation somebody else started.

The single-provider guarantee

This package exists as one published module rather than four vendored copies because four packages sharing no state cannot guarantee a single provider: two of them can observe "nothing installed" concurrently and both register one. Go resolves one module path to one version per build, so there is one instance of the installMu/installDone latch below, which every path that binds FlagDomain goes through — the auto-install, its retries, and an application's own SetNamedProvider — and therefore exactly one install.

What this package will not touch

It never calls SetProvider, SetEvaluationContext, AddHooks or Shutdown — the same rule the instrumentation packages follow for TracerProvider. The one piece of OpenFeature state it may write is a NAMED provider bound to FlagDomain, and only when the environment asks for one and the application installed none. Nothing it does can change how the application's own feature flags resolve.

Nothing here is cached. Value evaluates on every call, so a relay change is observed on the next operation; the end-to-end delay is the provider's poll interval, which this package lengthens by at most a tenth — see jitterInterval — and nothing else.

Index

• Constants
• Variables
• func Lookup(name string) (value bool, set bool, err error)
• func MasterEnabled(local bool) bool
• func MasterLocal() (bool, error)
• func RelayPossible() bool
• func ResolveLocal(option *bool, envName string, def bool) (bool, error)
• func SetNamedProvider(provider openfeature.FeatureProvider) error
• func ValidateAndInstall() error
• func Version() string
• type Resolver
  • func NewResolver() *Resolver
  • func (r *Resolver) Value(key string, local bool) bool

Constants

const (
	// EnvGlobalTracing is the process-wide master switch.
	//
	// It defaults to enabled, which makes it a veto rather than an enabler: the
	// only value with an effect is a falsy one, and setting it truthy changes
	// nothing. What it buys is a single variable that stops every module in the
	// process, including connections whose Go code passed an option.
	EnvGlobalTracing = "OTEL_INSTRUMENTATION_GO_TRACING_ENABLED"

	// EnvFlagsEndpoint is the GO Feature Flag relay proxy URL. Setting it is an
	// operator's request for relay control; leaving it unset means no provider is
	// ever constructed, no OpenFeature state is written, and RelayPossible
	// reports false.
	EnvFlagsEndpoint = "OTEL_INSTRUMENTATION_GO_FLAGS_ENDPOINT"

	// EnvFlagsAPIKey authenticates against a relay proxy that requires it. Its
	// value is never logged.
	EnvFlagsAPIKey = "OTEL_INSTRUMENTATION_GO_FLAGS_API_KEY"

	// EnvFlagsPollInterval overrides how often the provider polls the relay.
	// Go duration strings only: a bare integer is rejected rather than read as
	// milliseconds, because misreading a polling interval that way is
	// catastrophic rather than merely wrong. The value configured here is
	// jittered by at most plus or minus a tenth for the life of the process —
	// see jitterInterval — so it sets the centre of the polling period, not an
	// exact one.
	EnvFlagsPollInterval = "OTEL_INSTRUMENTATION_GO_FLAGS_POLL_INTERVAL"

	// EnvServiceName is the OpenTelemetry-specified service name. It is the only
	// source of targeting attributes this package uses, and only on the
	// auto-install path.
	EnvServiceName = "OTEL_SERVICE_NAME"
)

const FlagDomain = "otel-instrumentation-go"

FlagDomain is the single OpenFeature domain every module resolves through.

One domain rather than one per module is forced by the provider: the in-process evaluator's Init is not idempotent, so registering one instance under N domains starts N polling goroutines of which N−1 can never be stopped, and N separate instances would poll the relay N times over identical configuration.

Exported because module-package tests install their in-memory provider on it.

const FlagKeyGlobalTracing = "otel-instrumentation-go-tracing"

FlagKeyGlobalTracing is the master switch's relay key.

Its evaluation default is the master switch's local value, which defaults to true, so setting this key to true on a relay has no effect at all. The only useful value is false, which stops every module in every process the relay serves. Documentation must describe it that way; presented as an enable it will read as a broken flag.

Variables

var ErrInvalidFlagValue = errors.New("otel-flags: invalid configuration value")

ErrInvalidFlagValue reports an environment variable set to something this package cannot interpret: a switch that is neither truthy nor falsy, a poll interval that is not a positive Go duration, an endpoint that is not a URL.

One sentinel serves every variable and every module. That is possible only because this package is published rather than internal/, and it is why the per-module configuration-conflict sentinels an earlier design needed are gone. A caller that needs to know WHICH variable failed reads the message: it always names the variable and the observed value, and never the API key.

Functions

func Lookup

func Lookup(name string) (value bool, set bool, err error)

Lookup reads one environment variable as a tri-state.

Three outcomes, and only three:

• unset → (false, false, nil): this source has no opinion, and resolution falls through to the next rung down.
• recognised → (value, true, nil): this source decides.
• anything else → a non-nil error wrapping ErrInvalidFlagValue.

The third case is an error rather than a warning-and-a-guess because under a precedence ladder there is no safe direction to guess in. The master tier defaults to true and every other tier defaults to false, so a value silently read as false would stop a whole fleet on one tier and change nothing on the others — the same input meaning two different things, with a log line as the only evidence.

The empty string is invalid for the same reason. `export VAR=` reads as "set, to nothing", and both available readings are wrong somewhere: as false it lets an unexpanded ${SOMETHING} template variable express an opinion the deployment never had, and as unset it silently reverses meaning for anyone who used it as an off switch. Failing makes the ambiguity visible at the only moment anyone can act on it. The rule has no exceptions: set it to a recognised value, or do not set it.

The error names the variable and the observed value, so the fix needs no documentation lookup.

func MasterEnabled

func MasterEnabled(local bool) bool

MasterEnabled resolves the master switch for one operation, given the local value MasterLocal returned at construction.

It is resolved per operation like every other relay-backed switch. Resolving it once at construction would mean a relay veto reached only connections created afterwards, which is the opposite of what a veto is for.

func MasterLocal

func MasterLocal() (bool, error)

MasterLocal resolves the master switch from everything the relay cannot change: the environment variable, else the default of true.

There is no option parameter, and there must not be one. The master switch is process-scoped and an option is per-connection, so an option supplying it would give each connection its own "process-wide" switch — and would leave no single setting able to stop a process whose Go code hardcodes an opinion.

func RelayPossible

func RelayPossible() bool

RelayPossible reports whether a relay could ever have an opinion in this process: an endpoint is configured, or a provider is bound to FlagDomain.

When it is false, Client.Boolean can only ever return the value passed to it, so the relay is not merely silent — it is structurally incapable of speaking. Callers use that to keep the pre-dynamic zero-cost path: resolve from env > option > default alone, allocate the instrumented implementation only if that answer is on, and never touch the OpenFeature SDK.

A provider the application installed for its OWN flags does not count; see providerBound.

Callers MUST resolve it once per construction and MUST NOT memoize it process-wide. A package-level sync.Once would be cheaper and would guarantee that every wrapper agrees, but it would freeze the answer at whichever wrapper was built first — which in a test binary is whichever test ran first, making every subsequent relay test unreachable without a reset hook this design does not have.

The consequence for applications is an ordering rule: install your own provider BEFORE constructing any wrapper. One built earlier resolves statically for the rest of its life. SetNamedProvider warns when it can see that this happened; a raw openfeature.SetNamedProviderAndWait cannot be detected and is an accepted blind spot.

The rule only bites when EnvFlagsEndpoint is unset. With it set this is true from the process's first instruction, and ordering stops mattering.

func ResolveLocal

func ResolveLocal(option *bool, envName string, def bool) (bool, error)

ResolveLocal resolves the three rungs below the relay — env > option > default — into the single value Value takes as its evaluation default.

The environment variable is applied LAST because it outranks the option. That ordering is the operator's per-module control: a deployment can disable one module without silencing the process and without a relay, even when the application's Go code asked for it. The case that forces it is otel-mongo's document propagation, which appends a permanent field to the operator's own documents.

A Lookup error is returned even when an option was supplied. The option does not excuse an unreadable variable that outranks it, and a caller cannot know from the option alone what the deployment meant.

envName is a parameter rather than a constant here so this package still names no module. Modules own their variable names, their flag keys and their defaults; this package owns the ladder.

func SetNamedProvider

func SetNamedProvider(provider openfeature.FeatureProvider) error

SetNamedProvider binds provider to FlagDomain and records that this process deliberately gave the instrumentation switches a relay.

The name mirrors the SDK verb for what actually happens. This is set-or- replace, not a one-time idempotent install, and the difference matters now that sharing one provider between the application and this library is a supported story. It is deliberately not called SetProvider: that name is taken by openfeature.SetProvider, which writes the DEFAULT slot — the opposite one.

It is the recommended way for an application to install its own provider. The raw openfeature.SetNamedProviderAndWait(FlagDomain, p) still works and is still detected, but only through the heuristic in providerBound, which cannot tell an explicit binding from a fallback when the same provider is bound to both the default slot and this domain. Going through here removes that one blind spot: the record is exact. An application that wants ONE provider instance serving both its own flags and these switches must use this function — with the heuristic alone, the two slots read equal and the domain reads as unbound.

It installs with SetNamedProviderAndWait rather than the asynchronous form, so when it returns the provider has finished initialising and no startup window remains. Call it BEFORE constructing any wrapper — see RelayPossible.

It writes nothing but the named binding on FlagDomain. The default provider, the global evaluation context, hooks and shutdown all remain the application's. It takes installMu, which is what makes it safe to call concurrently with the first instrumented operation: without it, this and the environment auto-install were two unsynchronised read-then-bind sequences on the same domain — the very race installMu exists to prevent, reachable through the one entry point that skipped it. It also latches installDone, so an application that installs its own provider is never followed by an auto-install, whichever of the two the process reaches first.

Holding installMu across the wait does block a wrapper being constructed concurrently on another goroutine, for as long as the provider takes to initialise. That is the correct outcome and not merely an acceptable one: the wrapper would otherwise resolve its first operations against a provider this call is in the middle of replacing.

func ValidateAndInstall

func ValidateAndInstall() error

ValidateAndInstall validates this package's process-scoped environment and performs the one-time provider install. Every wrapper constructor calls it and joins its error with the module's own.

It reports every value it cannot read, together, as one error wrapping ErrInvalidFlagValue — a deployment carrying two typos must not have to fix one to discover the other. Nothing is installed when validation fails: a typo in the endpoint is exactly the case where guessing at what was meant is worst.

Both variables are validated whether or not a relay is configured. Making the poll interval's validity conditional on the endpoint being set would put back the unpredictability this removes — the same typo failing one deployment and passing another. The API key is deliberately not validated: any string can be a legitimate key.

The install itself does not block; see installProviderFromEnv. What moving it here buys is that it no longer runs inside an evaluation, where it held installMu — the same mutex SetNamedProvider holds across a blocking provider initialisation — and could park an instrumented operation for the length of somebody else's HTTP timeout.

Calling it a second time re-validates (cheap, and the environment can change between two constructions) and installs nothing.

The name says what it guarantees. It does NOT wait for the provider to initialise: when it returns, the relay may still be minutes from its first fetch, and every switch resolves locally until then. An application that wants that window closed installs its own provider with SetNamedProvider.

func Version

func Version() string

Version reports the module version. It exists so the constant has a reader outside the release guard, and so a caller can record which build of the flag layer a process is running.

Types

type Resolver

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver resolves one module's flag keys through the OpenFeature client.

It caches nothing. Value evaluates on every call, so there is no TTL, no snapshot timestamp, no clock to inject, and no cross-flag consistency window beyond the microseconds between two consecutive calls.

The measured cost is roughly 2 µs and 7 allocations per call against 82 ns for an atomic snapshot read. That is the SDK's evaluation pipeline — hook chains, evaluation-context merging, the provider registry lock — not the flag lookup, and an in-memory provider does not make it cheaper. An instrumented operation makes two of these calls (three on a Mongo write), so the cost is real and is recorded rather than assumed. Caching remains a permitted optimisation: it fits entirely inside this type without changing Value's signature or any call site.

func NewResolver

func NewResolver() *Resolver

NewResolver returns a Resolver for one module.

There is no domain parameter: the domain is process-scoped, so making it a parameter would only create a string that has to agree across every module with nothing checking it. There is no key list either — keys are passed to Value, so nothing positional can be mis-wired.

No OpenFeature client is created here. One is created lazily on the first Value call that finds a provider bound, so a process with no relay never touches the OpenFeature SDK at all. The provider install is not here either: it belongs to ValidateAndInstall, which a wrapper's constructor calls.

func (*Resolver) Value

func (r *Resolver) Value(key string, local bool) bool

Value returns the effective value of key, given the local value resolved from the option, the environment variable and the hardcoded default.

This single call is the whole precedence ladder. local is passed as the evaluation default, so the relay's value wins when it has one and local stands on every other path.

The key is a parameter rather than an index into a per-resolver list, and that is a correctness property rather than a taste one: an index couples two modules' flags by position with nothing checking it, so swapping two lines in a WithFlagKeys call used to compile, pass, and silently make otel-mongo's propagation flag control its tracing.

Nothing is evaluated unless a provider is bound to FlagDomain. That guard belongs here rather than in each wrapper, and it is not an optimisation: the SDK's ForEvaluation falls back to the DEFAULT provider when a domain is unbound, so evaluating regardless would resolve instrumentation keys against whatever the application installed for its own feature flags — a network call per instrumented operation if that provider evaluates remotely, and a wrong answer outright if it happens to define a key by the same name. Every wrapper in this repository hand-rolls an equivalent short-circuit before calling here; this makes the module that owns the ladder enforce it too, for the wrapper that forgets and for the callers of the exported MasterEnabled.