crypto

package
Version: v0.0.0-...-0694325 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 23, 2021 License: AGPL-3.0 Imports: 34 Imported by: 0

Documentation

Overview

Package crypto implements AWS S3 related cryptographic building blocks for implementing Server-Side-Encryption (SSE-S3) and Server-Side-Encryption with customer provided keys (SSE-C).

All objects are encrypted with an unique and randomly generated 'ObjectKey'. The ObjectKey itself is never stored in plaintext. Instead it is only stored in a sealed from. The sealed 'ObjectKey' is created by encrypting the 'ObjectKey' with an unique key-encryption-key. Given the correct key-encryption-key the sealed 'ObjectKey' can be unsealed and the object can be decrypted.

## SSE-C

SSE-C computes the key-encryption-key from the client-provided key, an initialization vector (IV) and the bucket/object path.

1. Encrypt:

Input: ClientKey, bucket, object, metadata, object_data
-              IV := Random({0,1}²⁵⁶)
-       ObjectKey := SHA256(ClientKey || Random({0,1}²⁵⁶))
-       KeyEncKey := HMAC-SHA256(ClientKey, IV || 'SSE-C' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object)
-       SealedKey := DAREv2_Enc(KeyEncKey, ObjectKey)
- enc_object_data := DAREv2_Enc(ObjectKey, object_data)
-        metadata <- IV
-        metadata <- SealedKey
Output: enc_object_data, metadata

2. Decrypt:

Input: ClientKey, bucket, object, metadata, enc_object_data
-          IV <- metadata
-   SealedKey <- metadata
-   KeyEncKey := HMAC-SHA256(ClientKey, IV || 'SSE-C' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object)
-   ObjectKey := DAREv2_Dec(KeyEncKey, SealedKey)
- object_data := DAREv2_Dec(ObjectKey, enc_object_data)
Output: object_data

## SSE-S3

SSE-S3 can use either a master key or a KMS as root-of-trust. The en/decryption slightly depens upon which root-of-trust is used.

### SSE-S3 and single master key

The master key is used to derive unique object- and key-encryption-keys. SSE-S3 with a single master key works as SSE-C where the master key is used as the client-provided key.

1. Encrypt:

Input: MasterKey, bucket, object, metadata, object_data
-              IV := Random({0,1}²⁵⁶)
-       ObjectKey := SHA256(MasterKey || Random({0,1}²⁵⁶))
-       KeyEncKey := HMAC-SHA256(MasterKey, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object)
-       SealedKey := DAREv2_Enc(KeyEncKey, ObjectKey)
- enc_object_data := DAREv2_Enc(ObjectKey, object_data)
-        metadata <- IV
-        metadata <- SealedKey
Output: enc_object_data, metadata

2. Decrypt:

Input: MasterKey, bucket, object, metadata, enc_object_data
-          IV <- metadata
-   SealedKey <- metadata
-   KeyEncKey := HMAC-SHA256(MasterKey, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object)
-   ObjectKey := DAREv2_Dec(KeyEncKey, SealedKey)
- object_data := DAREv2_Dec(ObjectKey, enc_object_data)
Output: object_data

### SSE-S3 and KMS

SSE-S3 requires that the KMS provides two functions:

1.       Generate(KeyID) -> (Key, EncKey)
2. Unseal(KeyID, EncKey) -> Key

1. Encrypt:

Input: KeyID, bucket, object, metadata, object_data
-     Key, EncKey := Generate(KeyID)
-              IV := Random({0,1}²⁵⁶)
-       ObjectKey := SHA256(Key, Random({0,1}²⁵⁶))
-       KeyEncKey := HMAC-SHA256(Key, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object)
-       SealedKey := DAREv2_Enc(KeyEncKey, ObjectKey)
- enc_object_data := DAREv2_Enc(ObjectKey, object_data)
-        metadata <- IV
-        metadata <- KeyID
-        metadata <- EncKey
-        metadata <- SealedKey
Output: enc_object_data, metadata

2. Decrypt:

Input: bucket, object, metadata, enc_object_data
-      KeyID  <- metadata
-      EncKey <- metadata
-          IV <- metadata
-   SealedKey <- metadata
-         Key := Unseal(KeyID, EncKey)
-   KeyEncKey := HMAC-SHA256(Key, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object)
-   ObjectKey := DAREv2_Dec(KeyEncKey, SealedKey)
- object_data := DAREv2_Dec(ObjectKey, enc_object_data)
Output: object_data

Index

Constants

View Source
const (
	// MetaMultipart indicates that the object has been uploaded
	// in multiple parts - via the S3 multipart API.
	MetaMultipart = "X-Minio-Internal-Encrypted-Multipart"

	// MetaIV is the random initialization vector (IV) used for
	// the MinIO-internal key derivation.
	MetaIV = "X-Minio-Internal-Server-Side-Encryption-Iv"

	// MetaAlgorithm is the algorithm used to derive internal keys
	// and encrypt the objects.
	MetaAlgorithm = "X-Minio-Internal-Server-Side-Encryption-Seal-Algorithm"

	// MetaSealedKeySSEC is the sealed object encryption key in case of SSE-C.
	MetaSealedKeySSEC = "X-Minio-Internal-Server-Side-Encryption-Sealed-Key"
	// MetaSealedKeyS3 is the sealed object encryption key in case of SSE-S3
	MetaSealedKeyS3 = "X-Minio-Internal-Server-Side-Encryption-S3-Sealed-Key"
	// MetaSealedKeyKMS is the sealed object encryption key in case of SSE-KMS
	MetaSealedKeyKMS = "X-Minio-Internal-Server-Side-Encryption-Kms-Sealed-Key"

	// MetaKeyID is the KMS master key ID used to generate/encrypt the data
	// encryption key (DEK).
	MetaKeyID = "X-Minio-Internal-Server-Side-Encryption-S3-Kms-Key-Id"
	// MetaDataEncryptionKey is the sealed data encryption key (DEK) received from
	// the KMS.
	MetaDataEncryptionKey = "X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key"

	// MetaContext is the KMS context provided by a client when encrypting an
	// object with SSE-KMS. A client may not send a context in which case the
	// MetaContext will not be present.
	// MetaContext only contains the bucket/object name if the client explicitly
	// added it. However, when decrypting an object the bucket/object name must
	// be part of the object. Therefore, the bucket/object name must be added
	// to the context, if not present, whenever a decryption is performed.
	MetaContext = "X-Minio-Internal-Server-Side-Encryption-Context"
)
View Source
const (
	// SealAlgorithm is the encryption/sealing algorithm used to derive & seal
	// the key-encryption-key and to en/decrypt the object data.
	SealAlgorithm = "DAREv2-HMAC-SHA256"

	// InsecureSealAlgorithm is the legacy encryption/sealing algorithm used
	// to derive & seal the key-encryption-key and to en/decrypt the object data.
	// This algorithm should not be used for new objects because its key derivation
	// is not optimal. See: https://github.com/minio/minio/pull/6121
	InsecureSealAlgorithm = "DARE-SHA256"
)
View Source
const (
	// EnvKMSAutoEncryption is the environment variable used to en/disable
	// SSE-S3 auto-encryption. SSE-S3 auto-encryption, if enabled,
	// requires a valid KMS configuration and turns any non-SSE-C
	// request into an SSE-S3 request.
	// If present EnvAutoEncryption must be either "on" or "off".
	EnvKMSAutoEncryption = "MINIO_KMS_AUTO_ENCRYPTION"
)

Variables

View Source
var (
	// ErrInvalidEncryptionMethod indicates that the specified SSE encryption method
	// is not supported.
	ErrInvalidEncryptionMethod = Errorf("The encryption method is not supported")

	// ErrInvalidCustomerAlgorithm indicates that the specified SSE-C algorithm
	// is not supported.
	ErrInvalidCustomerAlgorithm = Errorf("The SSE-C algorithm is not supported")

	// ErrMissingCustomerKey indicates that the HTTP headers contains no SSE-C client key.
	ErrMissingCustomerKey = Errorf("The SSE-C request is missing the customer key")

	// ErrMissingCustomerKeyMD5 indicates that the HTTP headers contains no SSE-C client key
	// MD5 checksum.
	ErrMissingCustomerKeyMD5 = Errorf("The SSE-C request is missing the customer key MD5")

	// ErrInvalidCustomerKey indicates that the SSE-C client key is not valid - e.g. not a
	// base64-encoded string or not 256 bits long.
	ErrInvalidCustomerKey = Errorf("The SSE-C client key is invalid")

	// ErrSecretKeyMismatch indicates that the provided secret key (SSE-C client key / SSE-S3 KMS key)
	// does not match the secret key used during encrypting the object.
	ErrSecretKeyMismatch = Errorf("The secret key does not match the secret key used during upload")

	// ErrCustomerKeyMD5Mismatch indicates that the SSE-C key MD5 does not match the
	// computed MD5 sum. This means that the client provided either the wrong key for
	// a certain MD5 checksum or the wrong MD5 for a certain key.
	ErrCustomerKeyMD5Mismatch = Errorf("The provided SSE-C key MD5 does not match the computed MD5 of the SSE-C key")
	// ErrIncompatibleEncryptionMethod indicates that both SSE-C headers and SSE-S3 headers were specified, and are incompatible
	// The client needs to remove the SSE-S3 header or the SSE-C headers
	ErrIncompatibleEncryptionMethod = Errorf("Server side encryption specified with both SSE-C and SSE-S3 headers")
)
View Source
var ErrKESKeyExists = NewKESError(http.StatusBadRequest, "key does already exist")

ErrKESKeyExists is the error returned a KES server when a master key does exist.

View Source
var (
	// S3 represents AWS SSE-S3. It provides functionality to handle
	// SSE-S3 requests.
	S3 = sses3{}
)
View Source
var (
	// S3KMS represents AWS SSE-KMS. It provides functionality to
	// handle SSE-KMS requests.
	S3KMS = ssekms{}
)
View Source
var (
	// SSEC represents AWS SSE-C. It provides functionality to handle
	// SSE-C requests.
	SSEC = ssec{}
)
View Source
var (
	// SSECopy represents AWS SSE-C for copy requests. It provides
	// functionality to handle SSE-C copy requests.
	SSECopy = ssecCopy{}
)

Functions

func CreateMultipartMetadata

func CreateMultipartMetadata(metadata map[string]string) map[string]string

CreateMultipartMetadata adds the multipart flag entry to metadata and returns modifed metadata. It allocates a new metadata map if metadata is nil.

func DecryptSinglePart

func DecryptSinglePart(w io.Writer, offset, length int64, key ObjectKey) io.WriteCloser

DecryptSinglePart decrypts an io.Writer which must an object uploaded with the single-part PUT API. The offset and length specify the requested range.

func EncryptMultiPart

func EncryptMultiPart(r io.Reader, partID int, key ObjectKey) io.Reader

EncryptMultiPart encrypts an io.Reader which must be the body of multi-part PUT request. It derives an unique encryption key from the partID and the object key.

func EncryptSinglePart

func EncryptSinglePart(r io.Reader, key ObjectKey) io.Reader

EncryptSinglePart encrypts an io.Reader which must be the the body of a single-part PUT request.

func Errorf

func Errorf(format string, a ...interface{}) error

Errorf - formats according to a format specifier and returns the string as a value that satisfies error of type crypto.Error

func EscapeStringJSON

func EscapeStringJSON(dst *bytes.Buffer, s string)

EscapeStringJSON will escape a string for JSON and write it to dst.

func GenerateIV

func GenerateIV(random io.Reader) (iv [32]byte)

GenerateIV generates a new random 256 bit IV from the provided source of randomness. If random is nil the default PRNG of the system (crypto/rand) is used.

func IsETagSealed

func IsETagSealed(etag []byte) bool

IsETagSealed returns true if the etag seems to be encrypted.

func IsMultiPart

func IsMultiPart(metadata map[string]string) bool

IsMultiPart returns true if the object metadata indicates that it was uploaded using some form of server-side-encryption and the S3 multipart API.

func IsSourceEncrypted

func IsSourceEncrypted(metadata map[string]string) bool

IsSourceEncrypted returns true if the source is encrypted

func LinearJitterBackoff

func LinearJitterBackoff(min, max time.Duration, attemptNum int) time.Duration

LinearJitterBackoff provides the time.Duration for a caller to perform linear backoff based on the attempt number and with jitter to prevent a thundering herd.

min and max here are *not* absolute values. The number to be multiplied by the attempt number will be chosen at random from between them, thus they are bounding the jitter.

For instance: * To get strictly linear backoff of one second increasing each retry, set both to one second (1s, 2s, 3s, 4s, ...) * To get a small amount of jitter centered around one second increasing each retry, set to around one second, such as a min of 800ms and max of 1200ms (892ms, 2102ms, 2945ms, 4312ms, ...) * To get extreme jitter, set to a very wide spread, such as a min of 100ms and a max of 20s (15382ms, 292ms, 51321ms, 35234ms, ...)

func LookupAutoEncryption

func LookupAutoEncryption() bool

LookupAutoEncryption returns true if and only if the MINIO_KMS_AUTO_ENCRYPTION env. variable is set to "on".

func NewKESError

func NewKESError(code int, text string) error

NewKESError returns a new KES API error with the given HTTP status code and error message.

Two errors with the same status code and error message are equal:

e1 == e2 // true.

func ParseKESEndpoints

func ParseKESEndpoints(endpointStr string) ([]string, error)

ParseKESEndpoints parses the given endpoint string and returns a list of valid endpoint URLs. The order of the returned endpoints is randomized.

func RemoveInternalEntries

func RemoveInternalEntries(metadata map[string]string)

RemoveInternalEntries removes all crypto-specific internal metadata entries from the metadata map.

func RemoveSSEHeaders

func RemoveSSEHeaders(metadata map[string]string)

RemoveSSEHeaders removes all crypto-specific SSE header entries from the metadata map.

func RemoveSensitiveEntries

func RemoveSensitiveEntries(metadata map[string]string)

RemoveSensitiveEntries removes confidential encryption information - e.g. the SSE-C key - from the metadata map. It has the same semantics as RemoveSensitiveHeaders.

func RemoveSensitiveHeaders

func RemoveSensitiveHeaders(h http.Header)

RemoveSensitiveHeaders removes confidential encryption information - e.g. the SSE-C key - from the HTTP headers. It has the same semantics as RemoveSensitiveEntires.

Types

type Context

type Context = kms.Context

Context is a list of key-value pairs cryptographically associated with a certain object.

type Error

type Error struct {
	// contains filtered or unexported fields
}

Error is the generic type for any error happening during decrypting an object. It indicates that the object itself or its metadata was modified accidentally or maliciously.

func (Error) Error

func (e Error) Error() string

Error 'error' compatible method.

func (Error) Unwrap

func (e Error) Unwrap() error

Unwrap the internal error.

type KMS

type KMS = kms.KMS

KMS represents an active and authenticted connection to a Key-Management-Service. It supports generating data key generation and unsealing of KMS-generated data keys.

func NewKes

func NewKes(cfg KesConfig) (KMS, error)

NewKes returns a new kes KMS client. The returned KMS uses the X.509 certificate to authenticate itself to the kes server available at address.

The defaultKeyID is the key ID returned when calling KMS.KeyID().

type KesConfig

type KesConfig struct {
	Enabled bool

	// The KES server endpoints.
	Endpoint []string

	// The path to the TLS private key used
	// by MinIO to authenticate to the kes
	// server during the TLS handshake (mTLS).
	KeyFile string

	// The path to the TLS certificate used
	// by MinIO to authenticate to the kes
	// server during the TLS handshake (mTLS).
	//
	// The kes server will also allow or deny
	// access based on this certificate.
	// In particular, the kes server will
	// lookup the policy that corresponds to
	// the identity in this certificate.
	CertFile string

	// Path to a file or directory containing
	// the CA certificate(s) that issued / will
	// issue certificates for the kes server.
	//
	// This is required if the TLS certificate
	// of the kes server has not been issued
	// (e.g. b/c it's self-signed) by a CA that
	// MinIO trusts.
	CAPath string

	// The default key ID returned by KMS.KeyID().
	DefaultKeyID string

	// The HTTP transport configuration for
	// the KES client.
	Transport *http.Transport
}

KesConfig contains the configuration required to initialize and connect to a kes server.

func (KesConfig) Verify

func (k KesConfig) Verify() (err error)

Verify verifies if the kes configuration is correct

type ObjectKey

type ObjectKey [32]byte

ObjectKey is a 256 bit secret key used to encrypt the object. It must never be stored in plaintext.

func GenerateKey

func GenerateKey(extKey []byte, random io.Reader) (key ObjectKey)

GenerateKey generates a unique ObjectKey from a 256 bit external key and a source of randomness. If random is nil the default PRNG of the system (crypto/rand) is used.

func (ObjectKey) DerivePartKey

func (key ObjectKey) DerivePartKey(id uint32) (partKey [32]byte)

DerivePartKey derives an unique 256 bit key from an ObjectKey and the part index.

func (ObjectKey) Seal

func (key ObjectKey) Seal(extKey []byte, iv [32]byte, domain, bucket, object string) SealedKey

Seal encrypts the ObjectKey using the 256 bit external key and IV. The sealed key is also cryptographically bound to the object's path (bucket/object) and the domain (SSE-C or SSE-S3).

func (ObjectKey) SealETag

func (key ObjectKey) SealETag(etag []byte) []byte

SealETag seals the etag using the object key. It does not encrypt empty ETags because such ETags indicate that the S3 client hasn't sent an ETag = MD5(object) and the backend can pick an ETag value.

func (*ObjectKey) Unseal

func (key *ObjectKey) Unseal(extKey []byte, sealedKey SealedKey, domain, bucket, object string) error

Unseal decrypts a sealed key using the 256 bit external key. Since the sealed key may be cryptographically bound to the object's path the same bucket/object as during sealing must be provided. On success the ObjectKey contains the decrypted sealed key.

func (ObjectKey) UnsealETag

func (key ObjectKey) UnsealETag(etag []byte) ([]byte, error)

UnsealETag unseals the etag using the provided object key. It does not try to decrypt the ETag if len(etag) == 16 because such ETags indicate that the S3 client hasn't sent an ETag = MD5(object) and the backend has picked an ETag value.

type SealedKey

type SealedKey struct {
	Key       [64]byte // The encrypted and authenticted object-key.
	IV        [32]byte // The random IV used to encrypt the object-key.
	Algorithm string   // The sealing algorithm used to encrypt the object key.
}

SealedKey represents a sealed object key. It can be stored at an untrusted location.

type Type

type Type interface {
	fmt.Stringer

	IsRequested(http.Header) bool

	IsEncrypted(map[string]string) bool
}

Type represents an AWS SSE type:

• SSE-C
• SSE-S3
• SSE-KMS

func IsEncrypted

func IsEncrypted(metadata map[string]string) (Type, bool)

IsEncrypted returns true if the object metadata indicates that it was uploaded using some form of server-side-encryption.

IsEncrypted only checks whether the metadata contains at least one entry indicating SSE-C or SSE-S3.

func IsRequested

func IsRequested(h http.Header) (Type, bool)

IsRequested returns true and the SSE Type if the HTTP headers indicate that some form server-side encryption is requested.

If no SSE headers are present then IsRequested returns false and no Type.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL