envsecret

README

EnvSecret Library

A Go library for late secret resolution during application startup. Instead of storing actual secrets in environment variables, store backend references (URIs) and resolve them when the application starts.

Features

• Late Secret Resolution: Secrets are fetched when the application starts, not when environment variables are set
• Multiple Backends: AWS SSM Parameter Store, AWS Secrets Manager, and AWS S3
• Environment Integration: Automatically populates environment variables with resolved secrets
• Security Protection: Prevents modification of sensitive system environment variables

Quick Start

Create a Populater via NewEnvSecret, passing AWS clients with WithSSMClient, WithSMClient, and/or WithS3Client. Then call PopulateEnv during application startup to resolve all secret references in environment variables.

Secret Reference Formats

Backend	Format	Example
SSM Parameter Store	ssm://parameter-name	ssm://prod/database/password
Secrets Manager	sm://secret-name	sm://prod/jwt/secret
S3	s3://bucket/key	s3://config-bucket/app.json

Environment Setup

Before running your application:

export DATABASE_PASSWORD="sm://prod/database/password"
export API_KEY="ssm://prod/api/key"
export CONFIG_FILE="s3://config-bucket/app.json"
export APP_NAME="my-application"  # Regular value

After PopulateEnv:

DATABASE_PASSWORD="actual-database-password"
API_KEY="actual-api-key-value"
CONFIG_FILE="json-config-content"
APP_NAME="my-application"  # Unchanged

API Reference

Core Interface

The Populater interface defines three methods:

• Populate(ctx, reference) — Resolves a single secret reference to its actual value. Non-secret strings are returned unchanged.
• PopulateEnv(ctx) — Scans all environment variables and resolves any secret references.
• PopulateEnvFiltered(ctx, filter) — Like PopulateEnv, but only processes variables where the filter function returns true.

Construction

NewEnvSecret(opts ...Option) creates an EnvSecret (which implements Populater) with the provided options:

• WithSSMClient(client) — Configures the AWS SSM Parameter Store client for ssm:// references.
• WithSMClient(client) — Configures the AWS Secrets Manager client for sm:// references.
• WithS3Client(client) — Configures the AWS S3 client for s3:// references.

Client Interfaces

Each backend requires a single-method client interface, making them easy to mock:

• SSMClient — requires GetParameter
• SMClient — requires GetSecretValue
• S3Client — requires GetObject

Error Handling

The library provides detailed error messages that include:

• Which environment variable failed
• The original secret reference that was being resolved
• The specific AWS service and resource name
• The underlying AWS error

When multiple variables fail, errors are aggregated using errors.Join, supporting errors.Is and errors.As unwrapping.

Required IAM Permissions

SSM Parameter Store

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": ["ssm:GetParameter"],
            "Resource": "arn:aws:ssm:*:*:parameter/your-parameter-path/*"
        },
        {
            "Effect": "Allow",
            "Action": ["kms:Decrypt"],
            "Resource": "arn:aws:kms:*:*:key/your-kms-key-id",
            "Condition": {
                "StringEquals": {
                    "kms:ViaService": "ssm.*.amazonaws.com"
                }
            }
        }
    ]
}

Secrets Manager

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": ["secretsmanager:GetSecretValue"],
            "Resource": "arn:aws:secretsmanager:*:*:secret:your-secret-path/*"
        },
        {
            "Effect": "Allow",
            "Action": ["kms:Decrypt"],
            "Resource": "arn:aws:kms:*:*:key/your-kms-key-id",
            "Condition": {
                "StringEquals": {
                    "kms:ViaService": "secretsmanager.*.amazonaws.com"
                }
            }
        }
    ]
}

S3

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": ["s3:GetObject"],
        "Resource": "arn:aws:s3:::your-bucket/*"
    }]
}

Security Features

The library automatically protects 94 sensitive system environment variables from being modified, including:

• System: PATH, LD_PRELOAD, SHELL, HOME, USER
• AWS: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN
• Lambda: AWS_LAMBDA_FUNCTION_NAME, LAMBDA_TASK_ROOT, _HANDLER
• ECS: ECS_CONTAINER_METADATA_URI, AWS_CONTAINER_CREDENTIALS_*
• Go: GOROOT, GOPATH, GOPROXY, CGO_ENABLED

See sensitiveEnvVars in sensitive.go for the full list.

Binary secrets from Secrets Manager are validated for UTF-8 encoding, since environment variables must be valid strings.

Concurrency and Safety

This library is NOT safe for concurrent use.

• All methods should be called from a single goroutine
• Typically used during application initialization before starting concurrent operations
• If concurrent access is required, external synchronization must be provided
• Environment variable modification affects the entire process

Input Validation

• S3 Size Limit: S3 objects are limited to 10MB to prevent memory exhaustion
• Input Sanitization: All inputs are trimmed and validated before processing
• Empty/whitespace references: Returned as-is without error
• S3 path format: Validated for correct bucket/key structure
• Client configuration: Clear errors when AWS clients are not configured for required backends

Container Usage

Docker

docker run -e DATABASE_PASSWORD="sm://prod/database/password" your-app

The application handles secret resolution during startup.

Documentation

Overview

Package envsecret provides late secret resolution during application startup. Instead of storing actual secrets in environment variables, store backend references (URIs) and resolve them when the application starts.

CONCURRENCY: This package is NOT safe for concurrent use. All methods should be called from a single goroutine, typically during application initialization. If concurrent access is required, external synchronization must be provided.

MEMORY: Resolved secrets are stored in memory and environment variables. Ensure proper memory management and consider the security implications of secrets in memory.

Index

• type EnvSecret
  • func NewEnvSecret(opts ...Option) *EnvSecret
  • func (e *EnvSecret) Populate(ctx context.Context, reference string) (string, error)
  • func (e *EnvSecret) PopulateEnv(ctx context.Context) error
  • func (e *EnvSecret) PopulateEnvFiltered(ctx context.Context, filter func(string) bool) error
• type Option
  • func WithS3Client(s3Client S3Client) Option
  • func WithSMClient(smClient SMClient) Option
  • func WithSSMClient(ssmClient SSMClient) Option
• type Populater
• type S3Client
• type SMClient
• type SSMClient

Types

type EnvSecret

type EnvSecret struct {
	// contains filtered or unexported fields
}

EnvSecret implements the Populater interface for AWS-based secret backends.

CONCURRENCY: EnvSecret is NOT safe for concurrent use. All methods should be called from a single goroutine. If concurrent access is required, external synchronization must be provided.

LIFECYCLE: Create one instance per application and use it during startup to populate all required secrets before starting concurrent operations.

func NewEnvSecret

func NewEnvSecret(opts ...Option) *EnvSecret

NewEnvSecret creates a new EnvSecret instance with the provided options. At least one AWS client must be configured for the corresponding secret backends you plan to use.

Example:

cfg, err := config.LoadDefaultConfig(context.Background())
if err != nil {
    log.Fatal(err)
}

populater := envsecret.NewEnvSecret(
    envsecret.WithSSMClient(ssm.NewFromConfig(cfg)),
    envsecret.WithSMClient(secretsmanager.NewFromConfig(cfg)),
    envsecret.WithS3Client(s3.NewFromConfig(cfg)),
)

func (*EnvSecret) Populate

func (e *EnvSecret) Populate(ctx context.Context, reference string) (string, error)

Populate resolves a secret reference to its actual value. This is the core method that handles secret resolution from various backends.

CONCURRENCY: This method is NOT safe for concurrent use with the same EnvSecret instance.

INPUT VALIDATION: Empty or whitespace-only references are returned as-is. This allows for graceful handling of optional configuration values.

Example:

// Resolve a secret
secret, err := populater.Populate(ctx, "sm://prod/database/password")
if err != nil {
    return fmt.Errorf("failed to get database password: %w", err)
}

// Non-secret references are returned unchanged
appName, _ := populater.Populate(ctx, "my-application")
// appName == "my-application"

// Empty values are handled gracefully
optional, _ := populater.Populate(ctx, "")
// optional == ""

func (*EnvSecret) PopulateEnv

func (e *EnvSecret) PopulateEnv(ctx context.Context) error

PopulateEnv scans all environment variables and populates secret references.

CONCURRENCY: This method is NOT safe for concurrent use. It modifies process-wide environment variables and should only be called during application initialization.

SECURITY: This method automatically protects sensitive system environment variables from modification, regardless of their content.

Example:

if err := populater.PopulateEnv(ctx); err != nil {
    log.Fatalf("Failed to populate secrets: %v", err)
}

func (*EnvSecret) PopulateEnvFiltered

func (e *EnvSecret) PopulateEnvFiltered(ctx context.Context, filter func(string) bool) error

PopulateEnvFiltered populates environment variables matching the filter.

CONCURRENCY: This method is NOT safe for concurrent use. It modifies process-wide environment variables and should only be called during application initialization.

INPUT VALIDATION: Returns an error if filter function is nil.

SECURITY: This method automatically protects sensitive system environment variables from modification, regardless of the filter function result.

type Option

type Option func(*EnvSecret)

Option configures an EnvSecret instance during creation.

func WithS3Client

func WithS3Client(s3Client S3Client) Option

WithS3Client configures the AWS S3 client. This client will be used for resolving s3:// references.

Example:

cfg, _ := config.LoadDefaultConfig(context.Background())
populater := envsecret.NewEnvSecret(
    envsecret.WithS3Client(s3.NewFromConfig(cfg)),
)

func WithSMClient

func WithSMClient(smClient SMClient) Option

WithSMClient configures the AWS Secrets Manager client. This client will be used for resolving sm:// references.

Example:

cfg, _ := config.LoadDefaultConfig(context.Background())
populater := envsecret.NewEnvSecret(
    envsecret.WithSMClient(secretsmanager.NewFromConfig(cfg)),
)

func WithSSMClient

func WithSSMClient(ssmClient SSMClient) Option

WithSSMClient configures the AWS SSM Parameter Store client. This client will be used for resolving ssm:// references.

Example:

cfg, _ := config.LoadDefaultConfig(context.Background())
populater := envsecret.NewEnvSecret(
    envsecret.WithSSMClient(ssm.NewFromConfig(cfg)),
)

type Populater

type Populater interface {
	// Populate resolves a secret reference to its actual value.
	//
	// The reference format is: <backend>://<identifier>
	// Supported backends:
	//   - ssm://parameter-name (AWS Systems Manager Parameter Store)
	//   - sm://secret-id (AWS Secrets Manager)
	//   - s3://bucket-name/key (AWS S3)
	//
	// If the reference doesn't match any known backend prefix,
	// it returns the original value unchanged.
	//
	// This method is designed to be called during application startup
	// to resolve all secret references to their actual values.
	//
	// Example:
	//   secret, err := populater.Populate(ctx, "sm://prod/database/password")
	//   if err != nil {
	//       return err
	//   }
	//   // secret now contains the actual password from AWS Secrets Manager
	//
	//   regular := populater.Populate(ctx, "not-a-secret-reference")
	//   // regular == "not-a-secret-reference" (unchanged)
	Populate(ctx context.Context, reference string) (string, error)

	// PopulateEnv scans all environment variables and populates any secret references
	// with their actual values. This is the primary method for application startup
	// secret resolution.
	//
	// Environment variables with supported prefixes will be resolved:
	//   MY_SECRET=sm://prod/database/password  -> MY_SECRET=actual_password_value
	//   API_KEY=ssm://prod/api/key            -> API_KEY=actual_api_key_value
	//   CONFIG=s3://config-bucket/app.json    -> CONFIG=json_content
	//   REGULAR_VAR=normal_value              -> REGULAR_VAR=normal_value (unchanged)
	//
	// SECURITY: This method will NOT modify sensitive system environment variables
	// such as PATH, LD_PRELOAD, SHELL, HOME, etc. for security reasons.
	//
	// Example:
	//   // Set environment variables with secret references
	//   os.Setenv("DATABASE_PASSWORD", "sm://prod/database/password")
	//   os.Setenv("API_KEY", "ssm://prod/api/key")
	//   os.Setenv("APP_NAME", "my-application")
	//
	//   // Resolve all secret references
	//   if err := populater.PopulateEnv(ctx); err != nil {
	//       log.Fatal(err)
	//   }
	//
	//   // Environment variables now contain actual secret values
	//   dbPassword := os.Getenv("DATABASE_PASSWORD") // actual password
	//   apiKey := os.Getenv("API_KEY")               // actual API key
	//   appName := os.Getenv("APP_NAME")             // "my-application" (unchanged)
	PopulateEnv(ctx context.Context) error

	// PopulateEnvFiltered scans environment variables matching the filter function
	// and populates any secret references with their actual values.
	//
	// The filter function receives the environment variable name and should return
	// true if the variable should be processed for secret resolution.
	//
	// SECURITY: This method will NOT modify sensitive system environment variables
	// regardless of the filter function result.
	//
	// Example:
	//   // Only populate secrets with "SECRET_" prefix
	//   err := populater.PopulateEnvFiltered(ctx, func(key string) bool {
	//       return strings.HasPrefix(key, "SECRET_") || key == "DATABASE_PASSWORD"
	//   })
	//
	//   // Example with multiple criteria
	//   err = populater.PopulateEnvFiltered(ctx, func(key string) bool {
	//       return strings.HasSuffix(key, "_SECRET") ||
	//              strings.Contains(key, "PASSWORD") ||
	//              key == "API_KEY"
	//   })
	PopulateEnvFiltered(ctx context.Context, filter func(string) bool) error
}

Populater interface defines the contract for populating secrets from various backends. The main use case is for secrets to be fetched as late as possible when starting an application by only providing the backend as a prefix and the secret identifier.

CONCURRENCY: Implementations are NOT required to be safe for concurrent use.

Example usage:

cfg, _ := config.LoadDefaultConfig(context.Background())
populater := envsecret.NewEnvSecret(
    envsecret.WithSSMClient(ssm.NewFromConfig(cfg)),
    envsecret.WithSMClient(secretsmanager.NewFromConfig(cfg)),
)

// Populate all environment variables at startup
if err := populater.PopulateEnv(context.Background()); err != nil {
    log.Fatal(err)
}

type S3Client

type S3Client interface {
	// GetObject retrieves an object from AWS S3.
	// Required IAM permissions: s3:GetObject
	GetObject(ctx context.Context, params *s3.GetObjectInput, optFns ...func(*s3.Options)) (*s3.GetObjectOutput, error)
}

S3Client defines the interface for AWS S3 operations. This interface allows for easy testing and mocking of AWS S3 calls.

type SMClient

type SMClient interface {
	// GetSecretValue retrieves a secret from AWS Secrets Manager.
	// Required IAM permissions:
	//   - secretsmanager:GetSecretValue
	//   - kms:Decrypt (required only if you use a customer-managed AWS KMS key to encrypt the secret)
	GetSecretValue(ctx context.Context, params *secretsmanager.GetSecretValueInput, optFns ...func(*secretsmanager.Options)) (*secretsmanager.GetSecretValueOutput, error)
}

SMClient defines the interface for AWS Secrets Manager operations. This interface allows for easy testing and mocking of AWS Secrets Manager calls.

type SSMClient

type SSMClient interface {
	// GetParameter retrieves a parameter from AWS SSM Parameter Store.
	// Required IAM permissions: ssm:GetParameter, kms:Decrypt (for SecureString parameters)
	GetParameter(ctx context.Context, params *ssm.GetParameterInput, optFns ...func(*ssm.Options)) (*ssm.GetParameterOutput, error)
}

SSMClient defines the interface for AWS Systems Manager Parameter Store operations. This interface allows for easy testing and mocking of AWS SSM calls.