Documentation

Overview

    Package s3crypto provides encryption to S3 using KMS and AES GCM.

    Keyproviders are interfaces that handle masterkeys. Masterkeys are used to encrypt and decrypt the randomly generated cipher keys. The SDK currently uses KMS to do this. A user does not need to provide a master key since all that information is hidden in KMS.

    Modes are interfaces that handle content encryption and decryption. It is an abstraction layer that instantiates the ciphers. If content is being encrypted we generate the key and iv of the cipher. For decryption, we use the metadata stored either on the object or an instruction file object to decrypt the contents.

    Ciphers are interfaces that handle encryption and decryption of data. This may be key wrap ciphers or content ciphers.

    Creating an S3 cryptography client

    cmkID := "<some key ID>"
    sess := session.New()
    // Create the KeyProvider
    handler := s3crypto.NewKMSKeyGenerator(kms.New(sess), cmkID)
    
    // Create an encryption and decryption client
    // We need to pass the session here so S3 can use it. In addition, any decryption that
    // occurs will use the KMS client.
    svc := s3crypto.NewEncryptionClient(sess, s3crypto.AESGCMContentCipherBuilder(handler))
    svc := s3crypto.NewDecryptionClient(sess)
    

    Configuration of the S3 cryptography client

    cfg := s3crypto.EncryptionConfig{
    	// Save instruction files to separate objects
    	SaveStrategy: NewS3SaveStrategy(session.New(), ""),
    	// Change instruction file suffix to .example
    	InstructionFileSuffix: ".example",
    	// Set temp folder path
    	TempFolderPath: "/path/to/tmp/folder/",
    	// Any content less than the minimum file size will use memory
    	// instead of writing the contents to a temp file.
    	MinFileSize: int64(1024 * 1024 * 1024),
    }
    

    The default SaveStrategy is to the object's header.

    The InstructionFileSuffix defaults to .instruction. Careful here though, if you do this, be sure you know what that suffix is in grabbing data. All requests will look for fooKey.example instead of fooKey.instruction. This suffix only affects gets and not puts. Put uses the keyprovider's suffix.

    Registration of new wrap or cek algorithms are also supported by the SDK. Let's say we want to support `AES Wrap` and `AES CTR`. Let's assume we have already defined the functionality.

    svc := s3crypto.NewDecryptionClient(sess)
    svc.WrapRegistry["AESWrap"] = NewAESWrap
    svc.CEKRegistry["AES/CTR/NoPadding"] = NewAESCTR
    

    We have now registered these new algorithms to the decryption client. When the client calls `GetObject` and sees the wrap as `AESWrap` then it'll use that wrap algorithm. This is also true for `AES/CTR/NoPadding`.

    For encryption adding a custom content cipher builder and key handler will allow for encryption of custom defined ciphers.

    // Our wrap algorithm, AESWrap
    handler := NewAESWrap(key, iv)
    // Our content cipher builder, AESCTRContentCipherBuilder
    svc := s3crypto.NewEncryptionClient(sess, NewAESCTRContentCipherBuilder(handler))
    

    Index

    Constants

    View Source
    const AESCBC = "AES/CBC"

      AESCBC is the string constant that signifies the AES CBC algorithm cipher.

      View Source
      const AESGCMNoPadding = "AES/GCM/NoPadding"

        AESGCMNoPadding is the constant value that is used to specify the CEK algorithm consiting of AES GCM with no padding.

        View Source
        const DefaultInstructionKeySuffix = ".instruction"

          DefaultInstructionKeySuffix is appended to the end of the instruction file key when grabbing or saving to S3

          View Source
          const DefaultMinFileSize = 1024 * 512 * 5

            DefaultMinFileSize is used to check whether we want to write to a temp file or store the data in memory.

            View Source
            const (
            	// KMSWrap is a constant used during decryption to build a KMS key handler.
            	KMSWrap = "kms"
            )

            Variables

            View Source
            var AESCBCPadder = Padder(aescbcPadding)

              AESCBCPadder is used to pad AES encrypted and decrypted data. Altough it uses the pkcs5Padder, it isn't following the RFC for PKCS5. The only reason why it is called pkcs5Padder is due to the Name returning PKCS5Padding.

              View Source
              var NoPadder = Padder(noPadder{})

                NoPadder does not pad anything

                Functions

                This section is empty.

                Types

                type CEKEntry

                type CEKEntry func(CipherData) (ContentCipher, error)

                  CEKEntry is a builder thatn returns a proper content decrypter and error

                  type Cipher

                  type Cipher interface {
                  	Encrypter
                  	Decrypter
                  }

                    Cipher interface allows for either encryption and decryption of an object

                    type CipherData

                    type CipherData struct {
                    	Key                 []byte
                    	IV                  []byte
                    	WrapAlgorithm       string
                    	CEKAlgorithm        string
                    	TagLength           string
                    	MaterialDescription MaterialDescription
                    	// EncryptedKey should be populated when calling GenerateCipherData
                    	EncryptedKey []byte
                    
                    	Padder Padder
                    }

                      CipherData is used for content encryption. It is used for storing the metadata of the encrypted content.

                      type CipherDataDecrypter

                      type CipherDataDecrypter interface {
                      	DecryptKey([]byte) ([]byte, error)
                      }

                        CipherDataDecrypter is a handler to decrypt keys from the envelope.

                        type CipherDataGenerator

                        type CipherDataGenerator interface {
                        	GenerateCipherData(int, int) (CipherData, error)
                        }

                          CipherDataGenerator handles generating proper key and IVs of proper size for the content cipher. CipherDataGenerator will also encrypt the key and store it in the CipherData.

                          func NewKMSKeyGenerator

                          func NewKMSKeyGenerator(kmsClient kmsiface.KMSAPI, cmkID string) CipherDataGenerator

                            NewKMSKeyGenerator builds a new KMS key provider using the customer key ID and material description.

                            Example:

                            sess := session.New(&aws.Config{})
                            cmkID := "arn to key"
                            matdesc := s3crypto.MaterialDescription{}
                            handler := s3crypto.NewKMSKeyGenerator(kms.New(sess), cmkID)
                            

                            func NewKMSKeyGeneratorWithMatDesc

                            func NewKMSKeyGeneratorWithMatDesc(kmsClient kmsiface.KMSAPI, cmkID string, matdesc MaterialDescription) CipherDataGenerator

                              NewKMSKeyGeneratorWithMatDesc builds a new KMS key provider using the customer key ID and material description.

                              Example:

                              sess := session.New(&aws.Config{})
                              cmkID := "arn to key"
                              matdesc := s3crypto.MaterialDescription{}
                              handler := s3crypto.NewKMSKeyGeneratorWithMatDesc(kms.New(sess), cmkID, matdesc)
                              

                              type ContentCipher

                              type ContentCipher interface {
                              	EncryptContents(io.Reader) (io.Reader, error)
                              	DecryptContents(io.ReadCloser) (io.ReadCloser, error)
                              	GetCipherData() CipherData
                              }

                                ContentCipher deals with encrypting and decrypting content

                                type ContentCipherBuilder

                                type ContentCipherBuilder interface {
                                	ContentCipher() (ContentCipher, error)
                                }

                                  ContentCipherBuilder is a builder interface that builds ciphers for each request.

                                  func AESCBCContentCipherBuilder

                                  func AESCBCContentCipherBuilder(generator CipherDataGenerator, padder Padder) ContentCipherBuilder

                                    AESCBCContentCipherBuilder returns a new encryption only mode structure with a specific cipher for the master key

                                    func AESGCMContentCipherBuilder

                                    func AESGCMContentCipherBuilder(generator CipherDataGenerator) ContentCipherBuilder

                                      AESGCMContentCipherBuilder returns a new encryption only mode structure with a specific cipher for the master key

                                      type CryptoReadCloser

                                      type CryptoReadCloser struct {
                                      	Body      io.ReadCloser
                                      	Decrypter io.Reader
                                      	// contains filtered or unexported fields
                                      }

                                        CryptoReadCloser handles closing of the body and allowing reads from the decrypted content.

                                        func (*CryptoReadCloser) Close

                                        func (rc *CryptoReadCloser) Close() error

                                          Close lets the CryptoReadCloser satisfy io.ReadCloser interface

                                          func (*CryptoReadCloser) Read

                                          func (rc *CryptoReadCloser) Read(b []byte) (int, error)

                                            Read lets the CryptoReadCloser satisfy io.ReadCloser interface

                                            type Decrypter

                                            type Decrypter interface {
                                            	Decrypt(io.Reader) io.Reader
                                            }

                                              Decrypter interface with only the decrypt method

                                              type DecryptionClient

                                              type DecryptionClient struct {
                                              	S3Client s3iface.S3API
                                              	// LoadStrategy is used to load the metadata either from the metadata of the object
                                              	// or from a separate file in s3.
                                              	//
                                              	// Defaults to our default load strategy.
                                              	LoadStrategy LoadStrategy
                                              
                                              	WrapRegistry   map[string]WrapEntry
                                              	CEKRegistry    map[string]CEKEntry
                                              	PadderRegistry map[string]Padder
                                              }

                                                DecryptionClient is an S3 crypto client. The decryption client will handle all get object requests from Amazon S3. Supported key wrapping algorithms:

                                                *AWS KMS
                                                

                                                Supported content ciphers:

                                                * AES/GCM
                                                * AES/CBC
                                                

                                                func NewDecryptionClient

                                                func NewDecryptionClient(prov client.ConfigProvider, options ...func(*DecryptionClient)) *DecryptionClient

                                                  NewDecryptionClient instantiates a new S3 crypto client

                                                  Example:

                                                  sess := session.New()
                                                  svc := s3crypto.NewDecryptionClient(sess, func(svc *s3crypto.DecryptionClient{
                                                  	// Custom client options here
                                                  }))
                                                  

                                                  func (*DecryptionClient) GetObject

                                                  func (c *DecryptionClient) GetObject(input *s3.GetObjectInput) (*s3.GetObjectOutput, error)

                                                    GetObject is a wrapper for GetObjectRequest

                                                    func (*DecryptionClient) GetObjectRequest

                                                    func (c *DecryptionClient) GetObjectRequest(input *s3.GetObjectInput) (*request.Request, *s3.GetObjectOutput)

                                                      GetObjectRequest will make a request to s3 and retrieve the object. In this process decryption will be done. The SDK only supports V2 reads of KMS and GCM.

                                                      Example:

                                                      sess := session.New()
                                                      svc := s3crypto.NewDecryptionClient(sess)
                                                      req, out := svc.GetObjectRequest(&s3.GetObjectInput {
                                                        Key: aws.String("testKey"),
                                                        Bucket: aws.String("testBucket"),
                                                      })
                                                      err := req.Send()
                                                      

                                                      func (*DecryptionClient) GetObjectWithContext

                                                      func (c *DecryptionClient) GetObjectWithContext(ctx aws.Context, input *s3.GetObjectInput, opts ...request.Option) (*s3.GetObjectOutput, error)

                                                        GetObjectWithContext is a wrapper for GetObjectRequest with the additional context, and request options support.

                                                        GetObjectWithContext is the same as GetObject with the additional support for Context input parameters. The Context must not be nil. A nil Context will cause a panic. Use the Context to add deadlining, timeouts, etc. In the future this may create sub-contexts for individual underlying requests.

                                                        type Encrypter

                                                        type Encrypter interface {
                                                        	Encrypt(io.Reader) io.Reader
                                                        }

                                                          Encrypter interface with only the encrypt method

                                                          type EncryptionClient

                                                          type EncryptionClient struct {
                                                          	S3Client             s3iface.S3API
                                                          	ContentCipherBuilder ContentCipherBuilder
                                                          	// SaveStrategy will dictate where the envelope is saved.
                                                          	//
                                                          	// Defaults to the object's metadata
                                                          	SaveStrategy SaveStrategy
                                                          	// TempFolderPath is used to store temp files when calling PutObject.
                                                          	// Temporary files are needed to compute the X-Amz-Content-Sha256 header.
                                                          	TempFolderPath string
                                                          	// MinFileSize is the minimum size for the content to write to a
                                                          	// temporary file instead of using memory.
                                                          	MinFileSize int64
                                                          }

                                                            EncryptionClient is an S3 crypto client. By default the SDK will use Authentication mode which will use KMS for key wrapping and AES GCM for content encryption. AES GCM will load all data into memory. However, the rest of the content algorithms do not load the entire contents into memory.

                                                            func NewEncryptionClient

                                                            func NewEncryptionClient(prov client.ConfigProvider, builder ContentCipherBuilder, options ...func(*EncryptionClient)) *EncryptionClient

                                                              NewEncryptionClient instantiates a new S3 crypto client

                                                              Example:

                                                              cmkID := "arn:aws:kms:region:000000000000:key/00000000-0000-0000-0000-000000000000"
                                                              sess := session.New()
                                                              handler := s3crypto.NewKMSKeyGenerator(kms.New(sess), cmkID)
                                                              svc := s3crypto.New(sess, s3crypto.AESGCMContentCipherBuilder(handler))
                                                              

                                                              func (*EncryptionClient) PutObject

                                                              func (c *EncryptionClient) PutObject(input *s3.PutObjectInput) (*s3.PutObjectOutput, error)

                                                                PutObject is a wrapper for PutObjectRequest

                                                                func (*EncryptionClient) PutObjectRequest

                                                                func (c *EncryptionClient) PutObjectRequest(input *s3.PutObjectInput) (*request.Request, *s3.PutObjectOutput)

                                                                  PutObjectRequest creates a temp file to encrypt the contents into. It then streams that data to S3.

                                                                  Example:

                                                                  svc := s3crypto.New(session.New(), s3crypto.AESGCMContentCipherBuilder(handler))
                                                                  req, out := svc.PutObjectRequest(&s3.PutObjectInput {
                                                                    Key: aws.String("testKey"),
                                                                    Bucket: aws.String("testBucket"),
                                                                    Body: strings.NewReader("test data"),
                                                                  })
                                                                  err := req.Send()
                                                                  

                                                                  func (*EncryptionClient) PutObjectWithContext

                                                                  func (c *EncryptionClient) PutObjectWithContext(ctx aws.Context, input *s3.PutObjectInput, opts ...request.Option) (*s3.PutObjectOutput, error)

                                                                    PutObjectWithContext is a wrapper for PutObjectRequest with the additional context, and request options support.

                                                                    PutObjectWithContext is the same as PutObject with the additional support for Context input parameters. The Context must not be nil. A nil Context will cause a panic. Use the Context to add deadlining, timeouts, etc. In the future this may create sub-contexts for individual underlying requests.

                                                                    type Envelope

                                                                    type Envelope struct {
                                                                    	// IV is the randomly generated IV base64 encoded.
                                                                    	IV string `json:"x-amz-iv"`
                                                                    	// CipherKey is the randomly generated cipher key.
                                                                    	CipherKey string `json:"x-amz-key-v2"`
                                                                    	// MaterialDesc is a description to distinguish from other envelopes.
                                                                    	MatDesc               string `json:"x-amz-matdesc"`
                                                                    	WrapAlg               string `json:"x-amz-wrap-alg"`
                                                                    	CEKAlg                string `json:"x-amz-cek-alg"`
                                                                    	TagLen                string `json:"x-amz-tag-len"`
                                                                    	UnencryptedMD5        string `json:"x-amz-unencrypted-content-md5"`
                                                                    	UnencryptedContentLen string `json:"x-amz-unencrypted-content-length"`
                                                                    }

                                                                      Envelope encryption starts off by generating a random symmetric key using AES GCM. The SDK generates a random IV based off the encryption cipher chosen. The master key that was provided, whether by the user or KMS, will be used to encrypt the randomly generated symmetric key and base64 encode the iv. This will allow for decryption of that same data later.

                                                                      type HeaderV2LoadStrategy

                                                                      type HeaderV2LoadStrategy struct{}

                                                                        HeaderV2LoadStrategy will load the envelope from the metadata

                                                                        func (HeaderV2LoadStrategy) Load

                                                                        func (load HeaderV2LoadStrategy) Load(req *request.Request) (Envelope, error)

                                                                          Load from a given object's header

                                                                          type HeaderV2SaveStrategy

                                                                          type HeaderV2SaveStrategy struct{}

                                                                            HeaderV2SaveStrategy will save the metadata of the crypto contents to the header of the object.

                                                                            func (HeaderV2SaveStrategy) Save

                                                                            func (strat HeaderV2SaveStrategy) Save(env Envelope, req *request.Request) error

                                                                              Save will save the envelope to the request's header.

                                                                              type LoadStrategy

                                                                              type LoadStrategy interface {
                                                                              	Load(*request.Request) (Envelope, error)
                                                                              }

                                                                                LoadStrategy ...

                                                                                type MaterialDescription

                                                                                type MaterialDescription map[string]*string

                                                                                  MaterialDescription is used to identify how and what master key has been used.

                                                                                  type Padder

                                                                                  type Padder interface {
                                                                                  	// Pad will pad the byte array.
                                                                                  	// The second parameter is NOT how many
                                                                                  	// bytes to pad by, but how many bytes
                                                                                  	// have been read prior to the padding.
                                                                                  	// This allows for streamable padding.
                                                                                  	Pad([]byte, int) ([]byte, error)
                                                                                  	// Unpad will unpad the byte bytes. Unpad
                                                                                  	// methods must be constant time.
                                                                                  	Unpad([]byte) ([]byte, error)
                                                                                  	// Name returns the name of the padder.
                                                                                  	// This is used when decrypting on
                                                                                  	// instantiating new padders.
                                                                                  	Name() string
                                                                                  }

                                                                                    Padder handles padding of crypto data

                                                                                    func NewPKCS7Padder

                                                                                    func NewPKCS7Padder(blockSize int) Padder

                                                                                      NewPKCS7Padder follows the RFC 2315: https://www.ietf.org/rfc/rfc2315.txt PKCS7 padding is subject to side-channel attacks and timing attacks. For the most secure data, use an authenticated crypto algorithm.

                                                                                      type S3LoadStrategy

                                                                                      type S3LoadStrategy struct {
                                                                                      	Client                *s3.S3
                                                                                      	InstructionFileSuffix string
                                                                                      }

                                                                                        S3LoadStrategy will load the instruction file from s3

                                                                                        func (S3LoadStrategy) Load

                                                                                        func (load S3LoadStrategy) Load(req *request.Request) (Envelope, error)

                                                                                          Load from a given instruction file suffix

                                                                                          type S3SaveStrategy

                                                                                          type S3SaveStrategy struct {
                                                                                          	Client                *s3.S3
                                                                                          	InstructionFileSuffix string
                                                                                          }

                                                                                            S3SaveStrategy will save the metadata to a separate instruction file in S3

                                                                                            func (S3SaveStrategy) Save

                                                                                            func (strat S3SaveStrategy) Save(env Envelope, req *request.Request) error

                                                                                              Save will save the envelope contents to s3.

                                                                                              type SaveStrategy

                                                                                              type SaveStrategy interface {
                                                                                              	Save(Envelope, *request.Request) error
                                                                                              }

                                                                                                SaveStrategy is how the data's metadata wants to be saved

                                                                                                type WrapEntry

                                                                                                type WrapEntry func(Envelope) (CipherDataDecrypter, error)

                                                                                                  WrapEntry is builder that return a proper key decrypter and error

                                                                                                  func NewKMSWrapEntry

                                                                                                  func NewKMSWrapEntry(kmsClient kmsiface.KMSAPI) WrapEntry

                                                                                                    NewKMSWrapEntry builds returns a new KMS key provider and its decrypt handler.

                                                                                                    Example:

                                                                                                    sess := session.New(&aws.Config{})
                                                                                                    customKMSClient := kms.New(sess)
                                                                                                    decryptHandler := s3crypto.NewKMSWrapEntry(customKMSClient)
                                                                                                    
                                                                                                    svc := s3crypto.NewDecryptionClient(sess, func(svc *s3crypto.DecryptionClient{
                                                                                                    	svc.WrapRegistry[KMSWrap] = decryptHandler
                                                                                                    }))