crypto

package
v0.0.0-...-e9b145e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 5, 2021 License: Apache-2.0 Imports: 22 Imported by: 1

Documentation

Overview

Package crypto implements AWS S3 related cryptographic building blocks for implementing Server-Side-Encryption (SSE-S3) and Server-Side-Encryption with customer provided keys (SSE-C).

All objects are encrypted with an unique and randomly generated 'ObjectKey'. The ObjectKey itself is never stored in plaintext. Instead it is only stored in a sealed from. The sealed 'ObjectKey' is created by encrypting the 'ObjectKey' with an unique key-encryption-key. Given the correct key-encryption-key the sealed 'ObjectKey' can be unsealed and the object can be decrypted.

## SSE-C

SSE-C computes the key-encryption-key from the client-provided key, an initialization vector (IV) and the bucket/object path.

  1. Encrypt: Input: ClientKey, bucket, object, metadata, object_data - IV := Random({0,1}²⁵⁶) - ObjectKey := SHA256(ClientKey || Random({0,1}²⁵⁶)) - KeyEncKey := HMAC-SHA256(ClientKey, IV || 'SSE-C' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object) - SealedKey := DAREv2_Enc(KeyEncKey, ObjectKey) - enc_object_data := DAREv2_Enc(ObjectKey, object_data) - metadata <- IV - metadata <- SealedKey Output: enc_object_data, metadata
  1. Decrypt: Input: ClientKey, bucket, object, metadata, enc_object_data - IV <- metadata - SealedKey <- metadata - KeyEncKey := HMAC-SHA256(ClientKey, IV || 'SSE-C' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object) - ObjectKey := DAREv2_Dec(KeyEncKey, SealedKey) - object_data := DAREv2_Dec(ObjectKey, enc_object_data) Output: object_data

## SSE-S3

SSE-S3 can use either a master key or a KMS as root-of-trust. The en/decryption slightly depens upon which root-of-trust is used.

### SSE-S3 and single master key

The master key is used to derive unique object- and key-encryption-keys. SSE-S3 with a single master key works as SSE-C where the master key is used as the client-provided key.

  1. Encrypt: Input: MasterKey, bucket, object, metadata, object_data - IV := Random({0,1}²⁵⁶) - ObjectKey := SHA256(MasterKey || Random({0,1}²⁵⁶)) - KeyEncKey := HMAC-SHA256(MasterKey, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object) - SealedKey := DAREv2_Enc(KeyEncKey, ObjectKey) - enc_object_data := DAREv2_Enc(ObjectKey, object_data) - metadata <- IV - metadata <- SealedKey Output: enc_object_data, metadata
  1. Decrypt: Input: MasterKey, bucket, object, metadata, enc_object_data - IV <- metadata - SealedKey <- metadata - KeyEncKey := HMAC-SHA256(MasterKey, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object) - ObjectKey := DAREv2_Dec(KeyEncKey, SealedKey) - object_data := DAREv2_Dec(ObjectKey, enc_object_data) Output: object_data

### SSE-S3 and KMS

SSE-S3 requires that the KMS provides two functions:

  1. Generate(KeyID) -> (Key, EncKey)
  2. Unseal(KeyID, EncKey) -> Key
  1. Encrypt: Input: KeyID, bucket, object, metadata, object_data - Key, EncKey := Generate(KeyID) - IV := Random({0,1}²⁵⁶) - ObjectKey := SHA256(Key, Random({0,1}²⁵⁶)) - KeyEncKey := HMAC-SHA256(Key, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object) - SealedKey := DAREv2_Enc(KeyEncKey, ObjectKey) - enc_object_data := DAREv2_Enc(ObjectKey, object_data) - metadata <- IV - metadata <- KeyID - metadata <- EncKey - metadata <- SealedKey Output: enc_object_data, metadata
  1. Decrypt: Input: bucket, object, metadata, enc_object_data - KeyID <- metadata - EncKey <- metadata - IV <- metadata - SealedKey <- metadata - Key := Unseal(KeyID, EncKey) - KeyEncKey := HMAC-SHA256(Key, IV || 'SSE-S3' || 'DAREv2-HMAC-SHA256' || bucket || '/' || object) - ObjectKey := DAREv2_Dec(KeyEncKey, SealedKey) - object_data := DAREv2_Dec(ObjectKey, enc_object_data) Output: object_data

Index

Constants

View Source 
const (
	// SSEKmsID is the HTTP header key referencing the SSE-KMS
	// key ID.
	SSEKmsID = SSEHeader + "-Aws-Kms-Key-Id"

	// SSEKmsContext is the HTTP header key referencing the
	// SSE-KMS encryption context.
	SSEKmsContext = SSEHeader + "-Context"
)
View Source 
const (
	// SSECAlgorithm is the HTTP header key referencing
	// the SSE-C algorithm.
	SSECAlgorithm = SSEHeader + "-Customer-Algorithm"

	// SSECKey is the HTTP header key referencing the
	// SSE-C client-provided key..
	SSECKey = SSEHeader + "-Customer-Key"

	// SSECKeyMD5 is the HTTP header key referencing
	// the MD5 sum of the client-provided key.
	SSECKeyMD5 = SSEHeader + "-Customer-Key-Md5"
)
View Source 
const (
	// SSECopyAlgorithm is the HTTP header key referencing
	// the SSE-C algorithm for SSE-C copy requests.
	SSECopyAlgorithm = "X-Amz-Copy-Source-Server-Side-Encryption-Customer-Algorithm"

	// SSECopyKey is the HTTP header key referencing the SSE-C
	// client-provided key for SSE-C copy requests.
	SSECopyKey = "X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key"

	// SSECopyKeyMD5 is the HTTP header key referencing the
	// MD5 sum of the client key for SSE-C copy requests.
	SSECopyKeyMD5 = "X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key-Md5"
)
View Source 
const (
	// SSEAlgorithmAES256 is the only supported value for the SSE-S3 or SSE-C algorithm header.
	// For SSE-S3 see: https://docs.aws.amazon.com/AmazonS3/latest/dev/SSEUsingRESTAPI.html
	// For SSE-C  see: https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html
	SSEAlgorithmAES256 = "AES256"

	// SSEAlgorithmKMS is the value of 'X-Amz-Server-Side-Encryption' for SSE-KMS.
	// See: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html
	SSEAlgorithmKMS = "aws:kms"
)
View Source 
const (
	// SSEMultipart is the metadata key indicating that the object
	// was uploaded using the S3 multipart API and stored using
	// some from of server-side-encryption.
	SSEMultipart = "X-Minio-Internal-Encrypted-Multipart"

	// SSEIV is the metadata key referencing the random initialization
	// vector (IV) used for SSE-S3 and SSE-C key derivation.
	SSEIV = "X-Minio-Internal-Server-Side-Encryption-Iv"

	// SSESealAlgorithm is the metadata key referencing the algorithm
	// used by SSE-C and SSE-S3 to encrypt the object.
	SSESealAlgorithm = "X-Minio-Internal-Server-Side-Encryption-Seal-Algorithm"

	// SSECSealedKey is the metadata key referencing the sealed object-key for SSE-C.
	SSECSealedKey = "X-Minio-Internal-Server-Side-Encryption-Sealed-Key"

	// S3SealedKey is the metadata key referencing the sealed object-key for SSE-S3.
	S3SealedKey = "X-Minio-Internal-Server-Side-Encryption-S3-Sealed-Key"

	// S3KMSKeyID is the metadata key referencing the KMS key-id used to
	// generate/decrypt the S3-KMS-Sealed-Key. It is only used for SSE-S3 + KMS.
	S3KMSKeyID = "X-Minio-Internal-Server-Side-Encryption-S3-Kms-Key-Id"

	// S3KMSSealedKey is the metadata key referencing the encrypted key generated
	// by KMS. It is only used for SSE-S3 + KMS.
	S3KMSSealedKey = "X-Minio-Internal-Server-Side-Encryption-S3-Kms-Sealed-Key"
)
View Source 
const (
	// SealAlgorithm is the encryption/sealing algorithm used to derive & seal
	// the key-encryption-key and to en/decrypt the object data.
	SealAlgorithm = "DAREv2-HMAC-SHA256"

	// InsecureSealAlgorithm is the legacy encryption/sealing algorithm used
	// to derive & seal the key-encryption-key and to en/decrypt the object data.
	// This algorithm should not be used for new objects because its key derivation
	// is not optimal. See: https://github.com/minio/minio/pull/6121
	InsecureSealAlgorithm = "DARE-SHA256"
)
View Source 
const SSEHeader = "X-Amz-Server-Side-Encryption"

SSEHeader is the general AWS SSE HTTP header key.

Variables

View Source 
var (
	// ErrInvalidEncryptionMethod indicates that the specified SSE encryption method
	// is not supported.
	ErrInvalidEncryptionMethod = errors.New("The encryption method is not supported")

	// ErrInvalidCustomerAlgorithm indicates that the specified SSE-C algorithm
	// is not supported.
	ErrInvalidCustomerAlgorithm = errors.New("The SSE-C algorithm is not supported")

	// ErrMissingCustomerKey indicates that the HTTP headers contains no SSE-C client key.
	ErrMissingCustomerKey = errors.New("The SSE-C request is missing the customer key")

	// ErrMissingCustomerKeyMD5 indicates that the HTTP headers contains no SSE-C client key
	// MD5 checksum.
	ErrMissingCustomerKeyMD5 = errors.New("The SSE-C request is missing the customer key MD5")

	// ErrInvalidCustomerKey indicates that the SSE-C client key is not valid - e.g. not a
	// base64-encoded string or not 256 bits long.
	ErrInvalidCustomerKey = errors.New("The SSE-C client key is invalid")

	// ErrSecretKeyMismatch indicates that the provided secret key (SSE-C client key / SSE-S3 KMS key)
	// does not match the secret key used during encrypting the object.
	ErrSecretKeyMismatch = errors.New("The secret key does not match the secret key used during upload")

	// ErrCustomerKeyMD5Mismatch indicates that the SSE-C key MD5 does not match the
	// computed MD5 sum. This means that the client provided either the wrong key for
	// a certain MD5 checksum or the wrong MD5 for a certain key.
	ErrCustomerKeyMD5Mismatch = errors.New("The provided SSE-C key MD5 does not match the computed MD5 of the SSE-C key")
	// ErrIncompatibleEncryptionMethod indicates that both SSE-C headers and SSE-S3 headers were specified, and are incompatible
	// The client needs to remove the SSE-S3 header or the SSE-C headers
	ErrIncompatibleEncryptionMethod = errors.New("Server side encryption specified with both SSE-C and SSE-S3 headers")
)
View Source 
var (
	// SSEC represents AWS SSE-C. It provides functionality to handle
	// SSE-C requests.
	SSEC = ssec{}

	// SSECopy represents AWS SSE-C for copy requests. It provides
	// functionality to handle SSE-C copy requests.
	SSECopy = ssecCopy{}
)
View Source 
var (
	//ErrKMSAuthLogin is raised when there is a failure authenticating to KMS
	ErrKMSAuthLogin = errors.New("Vault service did not return auth info")
)
View Source 
var S3 = s3{}

S3 represents AWS SSE-S3. It provides functionality to handle SSE-S3 requests.

View Source 
var S3KMS = s3KMS{}

S3KMS represents AWS SSE-KMS. It provides functionality to handle SSE-KMS requests.

Functions

func CreateMultipartMetadata 

func CreateMultipartMetadata(metadata map[string]string) map[string]string

CreateMultipartMetadata adds the multipart flag entry to metadata and returns modifed metadata. It allocates a new metadata map if metadata is nil.

func DecryptSinglePart 

func DecryptSinglePart(w io.Writer, offset, length int64, key ObjectKey) io.WriteCloser

DecryptSinglePart decrypts an io.Writer which must an object uploaded with the single-part PUT API. The offset and length specify the requested range.

func EncryptMultiPart 

func EncryptMultiPart(r io.Reader, partID int, key ObjectKey) io.Reader

EncryptMultiPart encrypts an io.Reader which must be the body of multi-part PUT request. It derives an unique encryption key from the partID and the object key.

func EncryptSinglePart 

func EncryptSinglePart(r io.Reader, key ObjectKey) io.Reader

EncryptSinglePart encrypts an io.Reader which must be the the body of a single-part PUT request.

func GenerateIV 

func GenerateIV(random io.Reader) (iv [32]byte)

GenerateIV generates a new random 256 bit IV from the provided source of randomness. If random is nil the default PRNG of the system (crypto/rand) is used.

func IsETagSealed 

func IsETagSealed(etag []byte) bool

IsETagSealed returns true if the etag seems to be encrypted.

func IsEncrypted 

func IsEncrypted(metadata map[string]string) bool

IsEncrypted returns true if the object metadata indicates that it was uploaded using some form of server-side-encryption.

IsEncrypted only checks whether the metadata contains at least one entry indicating SSE-C or SSE-S3.

func IsMultiPart 

func IsMultiPart(metadata map[string]string) bool

IsMultiPart returns true if the object metadata indicates that it was uploaded using some form of server-side-encryption and the S3 multipart API.

func RemoveInternalEntries 

func RemoveInternalEntries(metadata map[string]string)

RemoveInternalEntries removes all crypto-specific internal metadata entries from the metadata map.

func RemoveSensitiveEntries 

func RemoveSensitiveEntries(metadata map[string]string)

RemoveSensitiveEntries removes confidential encryption information - e.g. the SSE-C key - from the metadata map. It has the same semantics as RemoveSensitiveHeaders.

func RemoveSensitiveHeaders 

func RemoveSensitiveHeaders(h http.Header)

RemoveSensitiveHeaders removes confidential encryption information - e.g. the SSE-C key - from the HTTP headers. It has the same semantics as RemoveSensitiveEntires.

Types

type Context 

type Context map[string]string

Context is a list of key-value pairs cryptographically associated with a certain object.

func (Context) WriteTo 

func (c Context) WriteTo(w io.Writer) (n int64, err error)

WriteTo writes the context in a canonical from to w. It returns the number of bytes and the first error encounter during writing to w, if any.

WriteTo sorts the context keys and writes the sorted key-value pairs as canonical JSON object to w.

type Error 

type Error struct {
	// contains filtered or unexported fields
}

Error is the generic type for any error happening during decrypting an object. It indicates that the object itself or its metadata was modified accidentally or maliciously.

func (Error) Error 

func (e Error) Error() string

type KMS 

type KMS interface {
	// GenerateKey generates a new random data key using
	// the master key referenced by the keyID. It returns
	// the plaintext key and the sealed plaintext key
	// on success.
	//
	// The context is cryptographically bound to the
	// generated key. The same context must be provided
	// again to unseal the generated key.
	GenerateKey(keyID string, context Context) (key [32]byte, sealedKey []byte, err error)

	// UnsealKey unseals the sealedKey using the master key
	// referenced by the keyID. The provided context must
	// match the context used to generate the sealed key.
	UnsealKey(keyID string, sealedKey []byte, context Context) (key [32]byte, err error)
}

KMS represents an active and authenticted connection to a Key-Management-Service. It supports generating data key generation and unsealing of KMS-generated data keys.

func NewKMS 

func NewKMS(key [32]byte) KMS

NewKMS returns a basic KMS implementation from a single 256 bit master key.

The KMS accepts any keyID but binds the keyID and context cryptographically to the generated keys.

func NewVault 

func NewVault(kmsConf KMSConfig) (KMS, error)

NewVault initializes Hashicorp Vault KMS by authenticating to Vault with the credentials in KMSConfig, and gets a client token for future api calls.

type KMSConfig 

type KMSConfig struct {
	Vault VaultConfig `json:"vault"`
}

KMSConfig has the KMS config for hashicorp vault

func NewVaultConfig 

func NewVaultConfig() (KMSConfig, error)

NewVaultConfig sets KMSConfig from environment variables and performs validations.

type ObjectKey 

type ObjectKey [32]byte

ObjectKey is a 256 bit secret key used to encrypt the object. It must never be stored in plaintext.

func GenerateKey 

func GenerateKey(extKey [32]byte, random io.Reader) (key ObjectKey)

GenerateKey generates a unique ObjectKey from a 256 bit external key and a source of randomness. If random is nil the default PRNG of the system (crypto/rand) is used.

func (ObjectKey) DerivePartKey 

func (key ObjectKey) DerivePartKey(id uint32) (partKey [32]byte)

DerivePartKey derives an unique 256 bit key from an ObjectKey and the part index.

func (ObjectKey) Seal 

func (key ObjectKey) Seal(extKey, iv [32]byte, domain, bucket, object string) SealedKey

Seal encrypts the ObjectKey using the 256 bit external key and IV. The sealed key is also cryptographically bound to the object's path (bucket/object) and the domain (SSE-C or SSE-S3).

func (ObjectKey) SealETag 

func (key ObjectKey) SealETag(etag []byte) []byte

SealETag seals the etag using the object key. It does not encrypt empty ETags because such ETags indicate that the S3 client hasn't sent an ETag = MD5(object) and the backend can pick an ETag value.

func (*ObjectKey) Unseal 

func (key *ObjectKey) Unseal(extKey [32]byte, sealedKey SealedKey, domain, bucket, object string) error

Unseal decrypts a sealed key using the 256 bit external key. Since the sealed key may be cryptographically bound to the object's path the same bucket/object as during sealing must be provided. On success the ObjectKey contains the decrypted sealed key.

func (ObjectKey) UnsealETag 

func (key ObjectKey) UnsealETag(etag []byte) ([]byte, error)

UnsealETag unseals the etag using the provided object key. It does not try to decrypt the ETag if len(etag) == 16 because such ETags indicate that the S3 client hasn't sent an ETag = MD5(object) and the backend has picked an ETag value.

type SealedKey 

type SealedKey struct {
	Key       [64]byte // The encrypted and authenticted object-key.
	IV        [32]byte // The random IV used to encrypt the object-key.
	Algorithm string   // The sealing algorithm used to encrypt the object key.
}

SealedKey represents a sealed object key. It can be stored at an untrusted location.

type VaultAppRole 

type VaultAppRole struct {
	ID     string `json:"id"`
	Secret string `json:"secret"`
}

VaultAppRole represents vault approle credentials

type VaultAuth 

type VaultAuth struct {
	Type    string       `json:"type"`
	AppRole VaultAppRole `json:"approle"`
}

VaultAuth represents vault auth type to use. For now, AppRole is the only supported auth type.

type VaultConfig 

type VaultConfig struct {
	Endpoint string    `json:"endpoint"`
	Auth     VaultAuth `json:"auth"`
	Key      VaultKey  `json:"key-id"`
}

VaultConfig holds config required to start vault service

type VaultKey 

type VaultKey struct {
	Name    string `json:"name"`
	Version int    `json:"version"`
}

VaultKey represents vault encryption key-id name & version

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.