README
¶
Secret Source
Secret Source is a module/library to load secrets from various sources in a uniform way. It was implemented in response to CIS Kubernetes Benchmark controls:
- 5.4.1 - Prefer using secrets as files over secrets as environment variables
- 5.4.2 - Consider external secret storage
5.4.1 proposes an insecure/dangerous approach to secrets management that is justified as a solution to certain developers including environment variables in error/debug logs. As the recommendation of 5.4.1 is not actually recommended to be followed, but we cannot ignore the possibility of developer laziness to simply log environment variables, the only way to go is implementing 5.4.2.
Applications and services must fetch their own secrets. We must not rely on the infrastructure (e.g. Kubernetes) or the service mesh to fetch app/service secrets from external sources. This leads to the most secure way of handling secrets in a containerised environment. I also wrote about this in my blog here.
Usage
The first step is to obtain an instance of the secret handler that provides you the secret from the specified source.
handler, err := secretsource.SecretSource(secretSource)
secretSource is a string that specifies the name of the handler to use and the init argument of the handler. For example, if you provided plain,my-secret as the value of secretSource above, SecretSource would initialize the plain-text secret handler with the value my-secret and return an initialised instance of the handler. As you can see, you define a secret as a string that has two parts separated by comma. The first half is the name of the handler. What the second half represents depends on the handler implementation. For example, in case of the plain handler, the second half is the actual value of the secret. In case of the asm handler the second half is the ARN of the secret to be fetched from AWS Secrets Manager.
Please note that the default secret handler is plain, thus if you provide a secretSource without a handler name, it will be handled as a static plain-text secret. For example:
handler, err := secretsource.SecretSource('my-secret')
// Below always returns `my-secret` in `value`
value, err := handler.Get()
Plain-text secrets
The plain-text secret handler returns the secret value passed to it.
// Get a handle to the `plain-text` secret source handler and pass the
// plain-text secret "my-test-secret" as the init argument.
handler, err := secretsource.SecretSource("plain,my-test-secret")
//
// The above is equivalent to when there is no handler designation:
//
// handler, err := secretsource.SecretSource("my-test-secret")
//
// Get the value of the secret from the handler.
secretValue, err := handler.Get()
// secretValue above contains `my-test-secret`.
Fetch Secret from AWS Secrets Manager
The ASM handler returns fetches the secret with the given ARN or Secret Name from AWS Secrets Manager (ASM). The AWS region is extracted automatically when an ARN is provided. Otherwise, if you provide a Secret Name, you must specify the AWS region via the AWS_REGION environment variable.
The AWS SDK used by the library handles the authentication to AWS Secrets Manager thus AWS credentials can be provided in any ways supported by the AWS.
If you provide both the SECRET_USER_UID and SECRET_USER_GID environment variables, secretsource will attempt to seteuid and setegid to the IDs provided when communicating with AWS SM. Then, it will seteuid and setegid back to the real user ID and real group ID. Please note that if, for whatever reason, secretsource could not drop privileges after AWS SM operations, it will panic.
The ASM handler does not only fetches the current value of the secret, but also takes care of refreshing expired secrets locally on subsequest Get() calls.
secretSource := "asm,arn:aws:secretsmanager:eu-west-1:603493154151:secret:kubewatch/handler/test-u66pVt"
// Get an instance of the `asm` secret source handler and pass the secret's
// ARN as the init argument.
handler, err := secretsource.SecretSource(secretSource)
// Get the value of the secret.
secretValue, err := handler.Get()
//
// ...
// Finally, safely dispose of the secret once no longer needed.
handler.Close()
It is critical for AWS SM secrets to be created as binary secrets as the ASM handler expect the secret from secretsmanager.GetSecretValueOutput.SecretBinary instead of secretsmanager.GetSecretValueOutput.SecretString. This means credentials set from the AWS management console will not be fetched by this module.
Or, fetching a secret by its name:
secretSource := "asm,name_of_your_secret_here"
// Get an instance of the `asm` secret source handler and pass the secret's
// name as the init argument.
handler, err := secretsource.SecretSource(secretSource)
// Get the value of the secret.
secretValue, err := handler.Get()
//
// ...
//
// Finally, safely dispose of the secret once no longer needed.
handler.Close()
Cost Optimisation
If you initialised a new instance of the secret handler using secretsource.SecretSource every time your application needed a specific secret that would be costly if the secret was coming from an external source, such as AWS Secrets Manager. Not only it's expensive because of the pricing, but also because the service had to make network requests to obtain the secret, which would have a performance impact.
To optimise for both cost and performance, the handlers were implemented to cache the secrets indefinitely. An instance of a handler holds the secret fetched initially and only attempts to fetch the secret again if the secret has expired (if refresh is supported by the handler). As of this, for each secret you should keep and use the corresponding secret handler instance until your application terminates.
Development
Create a Custom Secret Handler
Secret source handlers implement the SourceHandler interface defined in sourcehandler.go. The most basic example of a source handler is the plain-text source handler implemented in source_plain.go:
package secretsource
type PlainSecret struct {
value []byte
}
func (s *PlainSecret) Init(value []byte) error {
s.value = value
return nil
}
func (s *PlainSecret) Close() {
if len(s.value) == 0 {
return
}
var pos int = 0
for pos < len(s.value) {
s.value[pos] = 0
pos++
}
}
func (s *PlainSecret) Get() ([]byte, error) {
return s.value
}
func (s *PlainSecret) Refresh() error {
return nil
}
As can be seen in the PlainSecret struct, the secret handler has a value property to store the secret and the value of it is initialised by the Init method. The Init method is responsible for setting up everything so that whenever the Get and Refresh methods are called, those can perform their duties.
The Close method clears the secret and should be called as soon as the secret is no longer needed. The Close method must overwrite each byte of the secret with zero to remove the secret from memory.
A more complex example you can look at is the AWS Secrets Manager handler.
Register your Custom Secret Handler
Custom secret handlers should be registered in the SecretSource method implemented in source.go.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DropPrivileges ¶ added in v0.0.4
func DropPrivileges() error
func GetPrivileges ¶ added in v0.0.4
func GetPrivileges() error
Types ¶
type AwsSecretManagerSecret ¶
type AwsSecretManagerSecret struct {
sync.Mutex
Arn string
Region string
Expires time.Time
// contains filtered or unexported fields
}
func (*AwsSecretManagerSecret) Close ¶
func (s *AwsSecretManagerSecret) Close()
func (*AwsSecretManagerSecret) Get ¶
func (s *AwsSecretManagerSecret) Get() ([]byte, error)
func (*AwsSecretManagerSecret) Init ¶
func (s *AwsSecretManagerSecret) Init(value []byte) error
func (*AwsSecretManagerSecret) Refresh ¶
func (s *AwsSecretManagerSecret) Refresh() error
type PlainSecret ¶
type PlainSecret struct {
// contains filtered or unexported fields
}
func (*PlainSecret) Close ¶
func (s *PlainSecret) Close()
func (*PlainSecret) Get ¶
func (s *PlainSecret) Get() ([]byte, error)
func (*PlainSecret) Init ¶
func (s *PlainSecret) Init(value []byte) error
func (*PlainSecret) Refresh ¶
func (s *PlainSecret) Refresh() error
type SourceHandler ¶
type SourceHandler interface {
Init(value []byte) error
Close()
Get() ([]byte, error)
Refresh() error
}
func SecretSource ¶
func SecretSource(secretValue string) (SourceHandler, error)
Source Files