kms

README

kms package

kms is a package that provides key management system features for go-kms-wrapping Wrappers.

The following domain terms are key to understanding the system and how to use it:

• wrapper: all keys within the system are a Wrapper from go-kms-wrapping.

• root external wrapper: the external wrapper that will serve as the root of trust for the kms system. Typically you'd get this root wrapper via go-kms-wrapper from a KMS provider. See the go-kms-wrapper docs for further details.

• scope: a scope defines a rotational boundary for a set of keys. The system tracks only the scope identifier and which is used to find keys with a specific scope.

  IMPORTANT: You should define a FK from kms_root_key.scope_id with cascading deletes and updates to the PK of the table within your domain that contains scopes. This FK will prevent orphaned kms keys.

  For example, you could assign organizations and projects scope IDs and then associate a set of keys with each org and project within your domain.

• root key: the KEKs (key-encryption-key) of the system.

• data key: the DEKs (data-encryption-key) of the system and must have a parent root key and a purpose.

• data key version: versions of DEKs (data-encryption-key) which are used to encrypt/decrypt data.

  IMPORTANT: You should define a FK with a restricted delete from any application table that stores encrypted data to kms_data_key_version(private_id). This FK will prevent kms keys from being deleted that are currently being used to encrypt/decrypt data.

  For example, you have a table named oidc which contains the app's encrypted oidc client_secret. The oidc table should have a key_version_id column with a restricted FK to kms_data_key_version(private_id) which prevents in use DEKs from being deleted.

• purpose: Each data key (DEK) must have a one purpose. For example, you could define a purpose of client-secrets for a DEK that will be used encrypt/decrypt wrapper operations on client-secrets

---

Database Schema

You'll find the database schema within the migrations directory. Currently postgres and sqlite are supported. The implementation does make some use of triggers to ensure some of its data integrity.

The migrations are intended to be incorporated into your existing go-migrate migrations. Feel free to change the migration file names, as long as they are applied in the same order as defined here. FYI, the migrations include kms_version table which is used to ensure that the schema and module are compatible.

High-level ERD                                          
                                                          
                                                          
             ┌───────────────────────────────┐            
             │                               ○            
             ┼                               ┼            
┌────────────────────────┐      ┌────────────────────────┐
│      kms_root_key      │      │      kms_data_key      │
├────────────────────────┤      ├────────────────────────┤
│private_id              │      │private_id              │
│scope_id                │      │root_key_id             │
│                        │      │purpose                 │
└────────────────────────┘      │                        │
             ┼                  └────────────────────────┘
             │                               ┼            
             │                               │            
             │                               │            
             │                               │            
             ┼                               ┼            
            ╱│╲                             ╱│╲           
┌────────────────────────┐      ┌────────────────────────┐
│  kms_root_key_version  │      │  kms_data_key_version  │
├────────────────────────┤      ├────────────────────────┤
│private_id              │      │private_id              │
│root_key_id             │      │data_key_id             │
│key                     │      │root_key_id             │
│version                 │      │key                     │
│                        │      │version                 │
└────────────────────────┘      └────────────────────────┘
             ┼                               ┼            
             │                               ○            
             └───────────────────────────────┘               
          

Documentation

Overview

kms is a package that provides key management system features for go-kms-wrapping Wrappers.

The following domain terms are key to understanding the system and how to use it:

• wrapper: all keys within the system are a Wrapper from go-kms-wrapping.

• root external wrapper: the external wrapper that will serve as the root of trust for the kms system. Typically you'd get this root wrapper via go-kms-wrapper from a KMS provider. See the go-kms-wrapper docs for further details.

• scope: a scope defines a rotational boundary for a set of keys. The system tracks only the scope identifier and which is used to find keys with a specific scope.

  **IMPORTANT**: You should define a FK from kms_root_key.scope_id with cascading deletes and updates to the PK of the table within your domain that tracks scopes. This FK will prevent orphaned kms keys.

  For example, you could assign organizations and projects scope IDs and then associate a set of keys with each org and project within your domain.

• root key: the KEKs (keys to encrypt keys) of the system.

• data key: the DEKs (keys to encrypt data) of the system and must have a parent root key and a purpose.

• purpose: Each data key (DEK) must have a one purpose. For example, you could define a purpose of client-secrets for a DEK that will be used encrypt/decrypt wrapper operations on `client-secrets`

Database Schema

You'll find the database schema within the migrations directory. Currently postgres and sqlite are supported. The implementation does make some use of triggers to ensure some of its data integrity.

The migrations are intended to be incorporated into your existing go-migrate migrations. Feel free to change the migration file names, as long as they are applied in the same order as defined here. FYI, the migrations include `kms_version` table which is used to ensure that the schema and module are compatible.

High-level ERD

             ┌───────────────────────────────┐
             │                               ○
             ┼                               ┼
┌────────────────────────┐      ┌────────────────────────┐
│      kms_root_key      │      │      kms_data_key      │
├────────────────────────┤      ├────────────────────────┤
│private_id              │      │private_id              │
│scope_id                │      │root_key_id             │
│                        │      │purpose                 │
└────────────────────────┘      │                        │
             ┼                  └────────────────────────┘
             │                               ┼
             │                               │
             │                               │
             │                               │
             ┼                               ┼
            ╱│╲                             ╱│╲
┌────────────────────────┐      ┌────────────────────────┐
│  kms_root_key_version  │      │  kms_data_key_version  │
├────────────────────────┤      ├────────────────────────┤
│private_id              │      │private_id              │
│root_key_id             │      │data_key_id             │
│key                     │      │root_key_id             │
│version                 │      │key                     │
│                        │      │version                 │
└────────────────────────┘      └────────────────────────┘
             ┼                               ┼
             │                               ○
             └───────────────────────────────┘

Index

• Constants
• Variables
• func TestDb(t *testing.T) (*dbw.DB, string)
• func TestDeleteKeysForPurpose(t *testing.T, conn *dbw.DB, purpose KeyPurpose)
• func TestKmsDeleteAllKeys(t *testing.T, conn *dbw.DB)
• type Key
• type KeyPurpose
• type KeyType
• type KeyVersion
• type Kms
  • func New(r dbw.Reader, w dbw.Writer, purposes []KeyPurpose, opt ...Option) (*Kms, error)
  • func (k *Kms) AddExternalWrapper(ctx context.Context, purpose KeyPurpose, wrapper wrapping.Wrapper) error
  • func (k *Kms) CreateKeys(ctx context.Context, scopeId string, purposes []KeyPurpose, opt ...Option) error
  • func (k *Kms) GetExternalRootWrapper() (wrapping.Wrapper, error)
  • func (k *Kms) GetExternalWrapper(_ context.Context, purpose KeyPurpose) (wrapping.Wrapper, error)
  • func (k *Kms) GetWrapper(ctx context.Context, scopeId string, purpose KeyPurpose, opt ...Option) (wrapping.Wrapper, error)
  • func (k *Kms) ListDataKeyVersionReferencers(ctx context.Context, opt ...Option) ([]string, error)
  • func (k *Kms) ListKeys(ctx context.Context, scopeId string) ([]Key, error)
  • func (k *Kms) Purposes() []KeyPurpose
  • func (k *Kms) ReconcileKeys(ctx context.Context, scopeIds []string, purposes []KeyPurpose, opt ...Option) error
  • func (k *Kms) RevokeKey(ctx context.Context, keyVersionId string) error
  • func (k *Kms) RevokeKeyVersion(ctx context.Context, keyVersionId string) error
  • func (k *Kms) RewrapKeys(ctx context.Context, scopeId string, opt ...Option) error
  • func (k *Kms) RotateKeys(ctx context.Context, scopeId string, opt ...Option) error
  • func (k *Kms) ValidateSchema(ctx context.Context) (string, error)
• type Option
  • func WithKeyId(keyVersionId string) Option
  • func WithKeyVersionId(keyVersionId string) Option
  • func WithRandomReader(randomReader io.Reader) Option
  • func WithReader(r dbw.Reader) Option
  • func WithReaderWriter(r dbw.Reader, w dbw.Writer) Option
  • func WithRewrap(enableRewrap bool) Option
  • func WithScopeIds(id ...string) Option
  • func WithTableNamePrefix(prefix string) Option
  • func WithTx(tx *dbw.RW) Option

Constants

const (

	// DefaultTableNamePrefix defines the default prefix that will be add to
	// every table name when an optional WithTableNamePrefix is not used when
	// calling New(...). For example the root table name might be "data_key" and
	// then the default is added to form "kms_data_key"
	DefaultTableNamePrefix = "kms"
)

Variables

var (
	// ErrInvalidVersion represents a runtime error when the database version
	// doesn't match the require version of the module.
	ErrInvalidVersion = errors.New("invalid version")

	// ErrInvalidParameter represents an invalid parameter error condition.
	ErrInvalidParameter = errors.New("invalid parameter")

	// ErrMultipleRecords represents multiple records were affected when only
	// one was expected
	ErrMultipleRecords = errors.New("multiple records")

	// ErrRecordNotFound represents that no record was found
	ErrRecordNotFound = errors.New("record not found")

	// ErrKeyNotFound represents that no key was found
	ErrKeyNotFound = errors.New("key not found")

	// ErrInternal represents an internal error/issue
	ErrInternal = errors.New("internal issue")
)

Functions

func TestDb

func TestDb(t *testing.T) (*dbw.DB, string)

TestDb will return a test db and a url for that db

func TestDeleteKeysForPurpose

func TestDeleteKeysForPurpose(t *testing.T, conn *dbw.DB, purpose KeyPurpose)

TestDeleteKeysForPurpose allows you to delete all the keys for a KeyPurpose for testing.

func TestKmsDeleteAllKeys

func TestKmsDeleteAllKeys(t *testing.T, conn *dbw.DB)

TestKmsDeleteAllKeys allows you to delete ALL the keys for testing.

Types

type Key

type Key struct {
	// Id is the key's id
	Id string `json:"id"`

	// Scope is the scope of the key
	Scope string `json:"scope"`

	// Type is the key's KeyType.
	Type KeyType `json:"type"`

	// CreateTime is the time this key was created in the db
	CreateTime time.Time `json:"create_time"`

	// Purpose is the key's purpose
	Purpose KeyPurpose `json:"key_purpose"`

	// Versions is a list of key versions for this key
	Versions []KeyVersion `json:"versions"`
}

Key is the permanent construct representing ephemeral key versions

type KeyPurpose

type KeyPurpose string

KeyPurpose allows an application to specify the reason they need a key; this is used to select which DEK to return

const (
	// KeyPurposeUnknown is the default, and indicates that a correct purpose
	// wasn't specified
	KeyPurposeUnknown KeyPurpose = ""

	// KeyPurposeRootKey defines a root key purpose
	KeyPurposeRootKey = "rootKey"
)

type KeyType

type KeyType string

const (
	// KeyTypeDek defines a KEK (key encryption key)
	KeyTypeKek KeyType = "kek"

	// KeyTypeDek defines a DEK (data encryption key)
	KeyTypeDek = "dek"
)

type KeyVersion

type KeyVersion struct {
	// Id is the key version's id
	Id string `json:"id"`

	// Version is the key version's version
	Version uint `json:"version"`

	// CreateTime is the key version's create time.
	CreateTime time.Time `json:"create_time"`
}

KeyVersion is a key's version (the construct containing the key material)

type Kms

type Kms struct {
	// contains filtered or unexported fields
}

Kms is a way to access wrappers for a given scope and purpose. Since keys can never change, only be added or (eventually) removed, it opportunistically caches, going to the database as needed.

func New

func New(r dbw.Reader, w dbw.Writer, purposes []KeyPurpose, opt ...Option) (*Kms, error)

New takes in a reader, writer and a list of key purposes it will support. Every kms will support a KeyPurposeRootKey by default and it doesn't need to be passed in as one of the supported purposes.

Supported options: WithTableNamePrefix.

func (*Kms) AddExternalWrapper

func (k *Kms) AddExternalWrapper(ctx context.Context, purpose KeyPurpose, wrapper wrapping.Wrapper) error

AddExternalWrapper allows setting the external keys which are defined outside of the kms (e.g. in a configuration file).

func (*Kms) CreateKeys

func (k *Kms) CreateKeys(ctx context.Context, scopeId string, purposes []KeyPurpose, opt ...Option) error

CreateKeys creates the root key and DEKs for the given scope id. By default, CreateKeys manages its own transaction (begin/rollback/commit).

It's valid to provide no KeyPurposes (nil or empty), which means that the scope will only end up with a root key (and one rk version) with no DEKs.

CreateKeys also supports the WithTx(...) and WithReaderWriter(...) options which allows the caller to pass an inflight transaction to be used for all database operations. If WithTx(...) or WithReaderWriter(...) are used, then the caller is responsible for managing the transaction. The purpose of the WithTx(...) and WithReaderWriter(...) options are to allow the caller to create the scope and all of its keys in the same transaction.

The WithRandomReader(...) option is supported as well. If no optional random reader is provided, then the reader from crypto/rand will be used as a default.

func (*Kms) GetExternalRootWrapper

func (k *Kms) GetExternalRootWrapper() (wrapping.Wrapper, error)

GetExternalRootWrapper returns the external wrapper for KeyPurposeRootKey is is just a convenience function for GetExternalWrapper(...) and returns ErrKeyNotFound when a root key is not found.

func (*Kms) GetExternalWrapper

func (k *Kms) GetExternalWrapper(_ context.Context, purpose KeyPurpose) (wrapping.Wrapper, error)

GetExternalWrapper returns the external wrapper for a given purpose and returns ErrKeyNotFound when a key for the given purpose is not found.

func (*Kms) GetWrapper

func (k *Kms) GetWrapper(ctx context.Context, scopeId string, purpose KeyPurpose, opt ...Option) (wrapping.Wrapper, error)

GetWrapper returns a wrapper for the given scope and purpose. The WithReader(...) option is supported for getting a wrapper.

If an optional WithKeyVersionId(...) or WithKeyId(...) is passed, it will ensure that the returning wrapper has that key version ID in the multiwrapper. This is not necessary for encryption but should be supplied for decryption. WithKeyVersionId(...) and WithKeyId(...) are equivalent options, describing the key version ID requested in the wrapper.

Note: getting a wrapper for KeyPurposeRootKey is supported, but a root wrapper is a KEK and should never be used for data encryption.

func (*Kms) ListDataKeyVersionReferencers

func (k *Kms) ListDataKeyVersionReferencers(ctx context.Context, opt ...Option) ([]string, error)

ListDataKeyVersionReferencers will lists the names of all tables referencing the private_id column of the data key version table. This can be useful when re-encrypting data that references a specific data key version by private_id. Supported options:

• WithTx
• WithReaderWriter

func (*Kms) ListKeys

func (k *Kms) ListKeys(ctx context.Context, scopeId string) ([]Key, error)

ListKeys returns the current list of kms keys.

func (*Kms) Purposes

func (k *Kms) Purposes() []KeyPurpose

Purposes returns a copy of the key purposes for the kms

func (*Kms) ReconcileKeys

func (k *Kms) ReconcileKeys(ctx context.Context, scopeIds []string, purposes []KeyPurpose, opt ...Option) error

ReconcileKeys will reconcile the keys in the kms against known possible issues. This function reconciles the global scope unless the WithScopeIds(...) option is provided

The WithRandomReader(...) option is supported as well. If an optional random reader is not provided (is nill), then the reader from crypto/rand will be used as a default.

func (*Kms) RevokeKey

func (k *Kms) RevokeKey(ctx context.Context, keyVersionId string) error

RevokeKey will revoke (remove) a key version. Deprecated: use RevokeKeyVersion instead.

func (*Kms) RevokeKeyVersion

func (k *Kms) RevokeKeyVersion(ctx context.Context, keyVersionId string) error

RevokeKeyVersion will revoke (remove) a key version. Be sure to rotate and rewrap KEK versions before revoking them. If it's a DEK, then you need to re-encrypt all data that was encrypted with the key version before revoking it. You must have foreign key restrictions between DEK key version IDs (kms_data_key_version.private_id) and columns in your tables which store the wrapper key ID used for encrypt/decrypt operations, otherwise you could lose access to your encrypted data when you revoke a DEK version that's still being used.

func (*Kms) RewrapKeys

func (k *Kms) RewrapKeys(ctx context.Context, scopeId string, opt ...Option) error

RewrapKeys will re-encrypt all versions of a scope's KEK with the external root key wrapper and re-encrypt all versions of a scopes DEKs with the latest KEK version. If you wish to rewrap and rotate keys, then use the RotateKeys function.

WithTx(...) and WithReaderWriter(...) options are supported which allow the caller to pass an inflight transaction to be used for all database operations. If WithTx(...) or WithReaderWriter(...) are used, then the caller is responsible for managing the transaction. If neither WithTx or WithReaderWriter are specified, then RotateKeys will rotate the scope's keys within its own transaction, which will be managed by RewrapKeys.

The WithRandomReader(...) option is supported. If no optional random reader is provided, then the reader from crypto/rand will be used as a default.

Options supported: WithRandomReader, WithTx, WithReaderWriter

func (*Kms) RotateKeys

func (k *Kms) RotateKeys(ctx context.Context, scopeId string, opt ...Option) error

RotateKeys will create new versions for the KEK and all DEKs in the scope identified by the current KeyPurpose(s) of the KMS.

If an optional WithRewrap(...) is requested, then all existing KEK versions in the scope will be re-encrypted with the external root wrapper and all DEK versions in the scope will be re-encrypted with the new KEK version.

WithTx(...) and WithReaderWriter(...) options are supported which allow the caller to pass an inflight transaction to be used for all database operations. If WithTx(...) or WithReaderWriter(...) are used, then the caller is responsible for managing the transaction. If neither WithTx or WithReaderWriter are specified, then RotateKeys will rotate the scope's keys within its own transaction, which will be managed by RotateKeys.

The WithRandomReader(...) option is supported. If no optional random reader is provided, then the reader from crypto/rand will be used as a default.

Options supported: WithRandomReader, WithTx, WithRewrap, WithReaderWriter

func (*Kms) ValidateSchema

func (k *Kms) ValidateSchema(ctx context.Context) (string, error)

ValidateSchema will validate the database schema against the module's required migrations.Version

type Option

type Option func(*options)

Option - how Options are passed as arguments

func WithKeyId

func WithKeyId(keyVersionId string) Option

WithKeyId allows specifying a key version ID that should be found in a scope's multiwrapper; if it is not found, keys will be refreshed. Deprecated: use WithKeyVersionId instead.

func WithKeyVersionId

func WithKeyVersionId(keyVersionId string) Option

WithKeyVersionId allows specifying a key version ID that should be found in a scope's multiwrapper; if it is not found, keys will be refreshed

func WithRandomReader

func WithRandomReader(randomReader io.Reader) Option

WithRandomReadear(...) option allows an optional random reader to be provided. By default the reader from crypto/rand will be used.

func WithReader

func WithReader(r dbw.Reader) Option

WithReader provides an optional reader

func WithReaderWriter

func WithReaderWriter(r dbw.Reader, w dbw.Writer) Option

WithReaderWriter allows the caller to pass an inflight transaction to be used for all database operations. If WithReaderWriter(...) is used, then the caller is responsible for managing the transaction. The purpose of the WithReaderWriter(...) option is to allow the caller to create the scope and all of its keys in the same transaction.

func WithRewrap

func WithRewrap(enableRewrap bool) Option

WithRewrap allows for optionally specifying that the keys should be rewrapped.

func WithScopeIds

func WithScopeIds(id ...string) Option

WithScopeIds allows an optional set of scope ids to be specified

func WithTableNamePrefix

func WithTableNamePrefix(prefix string) Option

WithTableNamePrefix specifying a prefix that should be used for schema table names. If not specified, the DefaultTableNamePrefix will be used. This optional allows us to support custom prefixes as well as multi KMSs within a single schema.

func WithTx

func WithTx(tx *dbw.RW) Option

WithTx allows the caller to pass an inflight transaction to be used for all database operations. If WithTx(...) is used, then the caller is responsible for managing the transaction. The purpose of the WithTx(...) option is to allow the caller to create the scope and all of its keys in the same transaction.