filevault

README

FileVault

Secure, content-addressable storage for sensitive files. Uses random, unique data encrypting key for each file at rest. Exports encrypted file to whitelisted recipient using recipient's PGP public key.

Web API

POST /file

Adds a file to the FileVault

Required parameters

• One of "file" or "path"
  • "file" is the key to an http file upload (i.e., client sends the data)
  • "path" is a full path to the file on a shared filesystem (must be accessible by the FileVault server / service account)
• "subject" - a string describing the content, which cannot be blank (e.g., category.subcategory.scope.source.v1)

Response

• Responds with a 201 status code ("CREATED") if the ingestion succeeds
• Response body is a plain string of the sha256 hexdigest of the file's contents matching /^[a-f0-9]{64}$/
• The sha256 hexdigest is the ID of the file in the FileVault

Tips

• Confirm that the response is a 201; if not, assume the file was not ingested

• Compare the returned digest with one calculated locally to verify that the vault received what you intended to send

  • With sha256sum: sha256sum <file>
  • With openssl: openssl dgst -sha256 -hex <file>

GET /file/{id}

Download an unencrypted copy of the file.

Required parameters

Username and password must be provided using Basic Authentication. FileVault configuration must point to a valid htpasswd file. A simple command line tool for adding and removing users from an htpasswd file is available here.

GET /meta/{id}

Returns metadata associated with a file in the vault as JSON.

Request

• id is the sha256 hexdigest of the file's contents (returned by POST /file)
• id must match /^[a-f0-9]{64}$/

Response

• 404 if file ID is not in the FileVault
• If successful, response is a 200 with a JSON body consisting of:
  • "Subject": a string of the subject provided when the file was stored
  • "Received": a timestamp in RFC 3339 format representing when the file was ingested by the FileVault
  • "Filename": a string of the original file name at time of ingestion
  • "MimeType": the detected MimeType of the file (using only the file extension)
  • "Size": the size of the unencrypted file, in bytes

{
  "Filename": "foo.txt",
  "Subject": "whatever.was.specified.at.ingest",
  "MimeType": "text/plain",
  "Received": "2015-12-19T16:39:57-08:00",
  "Size": 12345684
}

POST /export/{id}

Request the export of a file from the vault to a particular recipient

Required parameters

• recipient: a hex-encoded string of the 20 byte PGP public key fingerprint of the target recipient

Response

• If authorized, the response is plain string with a path to the exported file
• The exported file will be encrypted with the specified PGP public key
• If the public key is not in the whitelisted keyring, the response will be 401 ("Unauthorized")

tips

• From gpg, you can view the fingerprints of the keys in your keyring using gpg --fingerprint
• This FileVault server only exports to a single folder defined in its configuration file

POST /refresh-keys

Causes FileVault to reload the whitelisted export keys from the configured keyring location

Required parameters

NONE

Documentation

Index

• Variables
• func HexStringToFingerprint(input string) (fp [20]byte, err error)
• type Config
  • func (c Config) Validate() (err error)
• type Metadata
• type Vault
  • func NewVault(c *Config) (v Vault, err error)

Variables

var (

	// ErrRecipientNotInWhitelist is returned when the specified key fingerprint
	// is not found in the ProdSupportPubRing
	ErrRecipientNotInWhitelist = errors.New(
		`The recipient's PGP fingerprint is not in the authorized keychain`)

	// ErrInvalidPGPFingerprint is returned when a specified PGP key fingerprint
	// does not parse from hex with optional spaces to exactly 20 bytes
	ErrInvalidPGPFingerprint = errors.New(`The PGP fingerprint provided is not a valid hex encoding of 20 bytes`)

	// ErrKeyFingerprintNotSpecified is returned when an export is requested
	// without specifying the recipient's key fingerprint
	ErrKeyFingerprintNotSpecified = errors.New(`A specific PGP key fingerprint is required, but was not specified`)

	// ErrInvalidID is returned when a specified file ID does not match
	// the regular expression /^[a-z0-9]{64}$/
	ErrInvalidID = errors.New(`Invalid file ID`)

	// ErrIncorrectPassphrase is returned when a passphrase does not decrypt a
	// private key
	ErrIncorrectPassphrase = errors.New(`Incorect private key passphrase`)

	// ErrMasterKeyNotFound is returned when a specified key fingerprint
	// is not found on the provided secret keyring
	ErrMasterKeyNotFound = errors.New(`The specified master key fingerprint was not found on the secure keyring`)
)

var (

	// PGPSettings defines the cipher (AES256) and compression settings
	// (ZLIB with default compression) used by the filevault
	PGPSettings = &packet.Config{
		DefaultCipher:          packet.CipherAES256,
		DefaultCompressionAlgo: packet.CompressionZLIB,
		CompressionConfig:      &packet.CompressionConfig{Level: packet.DefaultCompression},
	}

	// HashAlgo is the cyptographic hashing algorithm used to identify files
	HashAlgo = sha256.New

	// IDLength is the length of file identifiers in bytes
	IDLength = hex.EncodedLen(sha256.Size)

	// SubDirCharLen specifies how many characters of the file ID will be used to
	// generate subdirectories under the file, key and meta root folders. Since
	// each character can be one of 16 possible values ([a-f0-9]),
	// the maximum number of subdirectories will be 16^SubDirCharLen
	SubDirCharLen = 2

	// IDPattern is a regular expression that all valid IDs must match
	IDPattern = regexp.MustCompile(fmt.Sprintf("^[a-f0-9]{%d}$", IDLength))

	// DEKBytes defines the number or random bytes used to generate the data
	// encrypting keys.  The random bytes are base64 encoded and used as a PGP
	// passphrase for symmetric encryption
	DEKBytes = 16
)

Functions

func HexStringToFingerprint

func HexStringToFingerprint(input string) (fp [20]byte, err error)

HexStringToFingerprint converts a string with optional spaces into the 20 byte array format of a PGP key fingerprint

Types

type Config

type Config struct {
	TLSCert, TLSKey             string
	DataRoot, KeyRoot, MetaRoot string
	ProdSupportDir              string
	ProdSupportPubRing          string
	LogFile                     string
	EmailFrom, EmailTo          string
	SMTPServer                  string
	SMTPPort                    int
	SecRing                     string
	MasterKeyPassphrase         string
	MasterKeyFingerprint        string
	HTTPLog                     string
	HtpasswdFile                string
}

Config stores the configuration parameters for a filevault

func (Config) Validate

func (c Config) Validate() (err error)

Validate performs some sanity checks on configuration values

type Metadata

type Metadata struct {
	Filename, Subject, MimeType string
	Received                    time.Time
	Size                        int64
}

Metadata describes a file in the Vault

type Vault

type Vault interface {

	// Store adds a file and associated metadata into the vault
	Store(unencryptedData io.Reader, md *Metadata) (id string, err error)

	// Export re-encrypts a file in the vault with a specified public key
	// and stores the output in a pre-configured location
	Export(id string, recipientFingerprint [20]byte) (path string, err error)

	// Get copies the unencrypted data of a file to the provided Writer
	Get(id string, dest io.Writer, requester string) (err error)

	// GetMetadata loads available metadata for a specified file
	GetMetadata(id string, md *Metadata) error

	// LoadExportKeyring reloads approved export recipients from the configured
	// ProdSupportKeyring file path
	LoadExportKeyring(data io.Reader) error
}

Vault stores sensitive files

func NewVault

func NewVault(c *Config) (v Vault, err error)

NewVault returns a new Vault