encrypt

package module
v0.1.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 15, 2022 License: MPL-2.0 Imports: 26 Imported by: 5

README

encrypt package Go Reference

The encrypt package implements a new Filter that supports filtering fields in an event payload using a custom tag named class. This new tag supports two fields. The first field tag is the classification of the data (valid values are public, sensitive and secret). The second field is an optional filter operation to apply (valid values are redact, encrypt, hmac-sha256).

tagged struct example

type testPayloadStruct struct {
    Public    string `class:"public"`
    Sensitive string `class:"sensitive,redact"` // example of classification,operation
    Secret    []byte `class:"secret"`
}

encrypt.Filter supports filtering the following struct field types within an event payload, when they are tagged with a class tag:

  • string
  • []string
  • []byte
  • [][]byte
  • wrapperspb.StringValue
  • wrapperspb.BytesValue

Note: tagging a map has no affect on it's filtering. Please see Taggable interface for information about how to filter maps.

The following DataClassifications are supported:

  • PublicClassification
  • SensitiveClassification
  • SecretClassification

The following FilterOperations are supported:

  • NoOperation: no filter operation is applied to the field data.
  • RedactOperation: redact the field data.
  • EncryptOperation: encrypts the field data.
  • HmacSha256Operation: HMAC sha-256 the field data.

Taggable interface

Go maps and google.protobuf.Struct in an event payloads can be filtered by implementing a single function Taggable interface, which returns a []PointerTag for fields that must be filtered. You may have payloads and/or payload fields which implement the Taggable interface and also contain fields that are tagged with the class tag.

Important:

In general, the package defaults assume that unclassified data (including map fields) are secrets and filters them appropriately. This means that:

  • When a map doesn't implement Taggable all of it's fields will be filtered as secret data.
  • If a map's Tags(...) function doesn't return a PointerTag for a field, that field will be filtered as secret data.
// Taggable defines an interface for taggable maps
type Taggable interface {
	// Tags will return a set of pointer tags for the map
	Tags() ([]PointerTag, error)
}

// PointerTag provides the pointerstructure pointer string to get/set a key
// within a map or struct.Value along with its DataClassification and
// FilterOperation.
type PointerTag struct {
	// Pointer is the pointerstructure pointer string to get/set a key within a
	// map[string]interface{}  See: https://github.com/mitchellh/pointerstructure
	Pointer string

	// Classification is the DataClassification of data pointed to by the
	// Pointer
	Classification DataClassification

	// Filter is the FilterOperation to apply to the data pointed to by the
	// Pointer.  This is optional and the default operations (or overrides) will
	// apply when not specified
	Filter FilterOperation
}

Filter operation overrides

The Filter node will contain an optional field:

FilterOperationOverrides map[DataClassification]FilterOperation

This map can provide an optional set of runtime overrides for the FilterOperations to be applied to DataClassifications.

Normally, the filter operation applied to a field is determined by the operation specified in its class tag. If no operation is specified in the tag, then a set of reasonable default filter operations are applied.

FilterOperationOverrides provides the ability to override an event's "class" tag settings.

Default filter operations

  • PublicClassification: NoOperation
  • SensitiveClassification: EncryptOperation
  • SecretClassification: RedactOperation
  • NoClassification: RedactOperation

Note: The function encrypt.DefaultFilterOperations() returns a map[DataClassification]FilterOperation of these defaults.

Documentation

Overview

Package encrypt implements a new Filter that supports filtering fields in an event payload using a custom tag named "class".

See: README.md

Index

Examples

Constants

View Source 
const (
	// RedactedData is the value that replaces redacted data (secrets)
	RedactedData = "[REDACTED]"

	// DataClassificationTagName is the tag name for classifying data into
	// DataClassification's
	DataClassificationTagName = "class"
)
View Source 
const TestMapField = "foo"

TestMapField defines a const for a field name used for testing TestTaggedMap

View Source 
const TestPublicMapField = "public-foo"

Variables

View Source 
var ErrInvalidParameter = errors.New("invalid parameter")

ErrInvalidParameter defines a value for invalid parameter errors

Functions

func DefaultFilterOperations 

func DefaultFilterOperations() map[DataClassification]FilterOperation

DefaultFilterOperations returns a map of DataClassification to its default FilterOperation (when no overrides are configured for the filter node).

func NewDerivedReader 

func NewDerivedReader(ctx context.Context, wrapper wrapping.Wrapper, lenLimit int64, salt, info []byte) (*io.LimitedReader, error)

DerivedReader returns a reader from which keys can be read, using the given wrapper, reader length limit, salt and context info. Salt and info can be nil.

Example: 

reader, _ := NewDerivedReader(wrapper, userId, jobId)
key := ed25519.GenerateKey(reader)

func NewEventWrapper 

func NewEventWrapper(ctx context.Context, wrapper wrapping.Wrapper, eventId string) (wrapping.Wrapper, error)

NewEventWrapper is used by the Filter to derive a wrapper to use for a specific event. The event must implement the WrapperPayload interface for per event wrappers to be derived.

func TestDecryptValue 

func TestDecryptValue(t *testing.T, w wrapping.Wrapper, value []byte) []byte

func TestHmacSha256 

func TestHmacSha256(t *testing.T, data []byte, w wrapping.Wrapper, salt, info []byte) string

func TestWrapper 

func TestWrapper(t *testing.T) wrapping.Wrapper

TestWrapper initializes an AEAD wrapping.Wrapper for testing

Types

type DataClassification 

type DataClassification string

DataClassification defines a type for classification of data into categories like: public, sensitive, secret, etc. DataClassifications are used with as values for tags using DataClassificationTagName to declare a type of classification. 

const (
	UnknownClassification DataClassification = "unknown"

	// PublicClassification declares a field as public data.  No filter
	// operations are ever performed on public data.
	PublicClassification DataClassification = "public"

	// SensitiveClassification declares a field as sensitive data.  By default,
	// sensitive data is encrypted unless the tag includes an overriding FilterOperation.
	SensitiveClassification DataClassification = "sensitive"

	// SecretClassification declares a field as secret data.  By default,
	// secret data is redacted unless the tag includes an overriding FilterOperation.
	SecretClassification DataClassification = "secret"
)

type EventWrapperInfo 

type EventWrapperInfo interface {
	// Event ID to use when deriving keys for crypto operations on the event
	// payload
	EventId() string

	// HmacSalt to use for the event hmac-sha256 operations
	HmacSalt() []byte

	// HmacInfo to use for the event hmac-sha256 operations
	HmacInfo() []byte
}

EventWrapperInfo defines and interface for eventlogger payloads which include data used to derive a per event wrapper.

type Filter 

type Filter struct {
	// Wrapper to encrypt or hmac-sha256 string and []byte fields which are
	// tagged as SensitiveClassification.  This may be rotated with an event
	// that has a payload satisfying the RotateWrapper interface.  If an
	// event's payload satisfies the EventWrapperInfo interface, an event
	// specify wrapper will be derived from this wrapper using that
	// EventWrapperInfo.
	Wrapper wrapping.Wrapper

	// Salt for deriving a hmac-sha256 operations key (can be nil). This may be
	// rotated with an event that has a payload satisfying the RotateWrapper
	// interface. If an event's payload satisfies the EventWrapperInfo
	// interface, event specific HmacSalt will be used for operations on that
	// specific event.
	HmacSalt []byte

	// Info for deriving a hmac-sha256 operations key (can be nil). This may be
	// rotated with an event that has a payload satisfying the RotateWrapper
	// interface.  If an event's payload satisfies the
	// EventWrapperInfo interface, event specific HmacInfo will be used for
	// operations on that specific event.
	HmacInfo []byte

	// FilterOperationOverrides provide an optional a set of runtime overrides
	// for the FilterOperations to be applied to DataClassifications.
	//
	// Normally, the filter operation applied to a field is determined by the
	// operation specified in it's "class" tag. If no operation is
	// specified in the tag, then a set of reasonable default filter operations
	// are applied.
	//
	// FilterOperationOverrides provides the ability to override an event's
	// "class" tag settings.
	FilterOperationOverrides map[DataClassification]FilterOperation

	// IgnoreTypes provides the ability to supply optional types that will be
	// ignored when filtering.
	// For example: if you want all Google protobuf field masks to be ignored
	// (never filtered), you could set IgnoreTypes to:
	//	IgnoreTypes: []reflect.Type{reflect.TypeOf(&fieldmaskpb.FieldMask{})}
	IgnoreTypes []reflect.Type
	// contains filtered or unexported fields
}

Filter is an eventlogger Filter Node which will filter string and []byte fields in an event. Fields with tags that designate SecretClassification will be redacted. Fields with tags that designate SensitiveClassification will either be encrypted or hmac-sha256.

Example 
package main

import (
	"context"
	"crypto/rand"
	"encoding/base64"
	"fmt"
	"os"
	"time"

	"github.com/hashicorp/eventlogger"
	"github.com/hashicorp/eventlogger/filters/encrypt"
	"github.com/hashicorp/eventlogger/sinks/writer"
	wrapping "github.com/hashicorp/go-kms-wrapping/v2"
	"github.com/hashicorp/go-kms-wrapping/v2/aead"
)

func main() {
	then := time.Date(
		2009, 11, 17, 20, 34, 58, 651387237, time.UTC)
	// Create a broker
	b := eventlogger.NewBroker()

	b.StopTimeAt(then) // setting this so the output timestamps are predictable for testing.

	wrapper := exampleWrapper()

	// A gated.Filter for events
	f := &encrypt.Filter{
		Wrapper:  wrapper,
		HmacSalt: []byte("salt"),
		HmacInfo: []byte("info"),
	}
	// Marshal to JSON
	jsonFmt := &eventlogger.JSONFormatter{}

	// Send the output to stdout
	stdoutSink := &writer.Sink{
		Writer: os.Stdout,
	}

	// Register the nodes with the broker
	nodes := []eventlogger.Node{f, jsonFmt, stdoutSink}
	nodeIDs := make([]eventlogger.NodeID, len(nodes))
	for i, node := range nodes {
		id := eventlogger.NodeID(fmt.Sprintf("node-%d", i))
		err := b.RegisterNode(id, node)
		if err != nil {
			// handle error
		}
		nodeIDs[i] = id
	}

	et := eventlogger.EventType("test-event")
	// Register a pipeline for our event type
	err := b.RegisterPipeline(eventlogger.Pipeline{
		EventType:  et,
		PipelineID: "encrypt-filter-pipeline",
		NodeIDs:    nodeIDs,
	})
	if err != nil {
		panic(err)
	}

	p := examplePayload{
		NoClassification: "no classification",
		Public:           "public",
		Sensitive:        "sensitive",
		Secret:           "secret",
		TaggableMap: map[string]string{
			noClassificationField: "no classification",
			publicField:           "public",
			sensitiveField:        "sensitive",
			secretField:           "secret",
		},
	}

	ctx := context.Background()

	if status, err := b.Send(ctx, et, p); err != nil {
		// handle err and status.Warnings
		fmt.Println("err: ", err)
		fmt.Println("warnings: ", status.Warnings)
	}

}

const (
	sensitiveField        = "sensitive"
	secretField           = "secret"
	publicField           = "public"
	noClassificationField = "no-classification"
)

type examplePayload struct {
	NoClassification string
	Public           string `class:"public"`
	Sensitive        string `class:"sensitive,redact"`
	Secret           string `class:"secret,redact"`
	TaggableMap      map[string]string
}

func (p examplePayload) Tags() ([]encrypt.PointerTag, error) {
	return []encrypt.PointerTag{
		{Pointer: "/TaggableMap/" + secretField, Classification: encrypt.SecretClassification, Filter: encrypt.RedactOperation},
		{Pointer: "/TaggableMap/" + sensitiveField, Classification: encrypt.SensitiveClassification, Filter: encrypt.RedactOperation},
		{Pointer: "/TaggableMap/" + publicField, Classification: encrypt.PublicClassification, Filter: encrypt.RedactOperation},
	}, nil
}

// exampleWrapper initializes an AEAD wrapping.Wrapper for examples
func exampleWrapper() wrapping.Wrapper {
	rootKey := make([]byte, 32)
	n, err := rand.Read(rootKey)
	if err != nil {
		panic(err)
	}
	if n != 32 {
		panic("unable to read 32 bytes from rand")
	}
	root := aead.NewWrapper()
	_, err = root.SetConfig(
		context.Background(),
		wrapping.WithKeyId(base64.StdEncoding.EncodeToString(rootKey)),
	)
	if err != nil {
		panic(err)
	}
	err = root.SetAesGcmKeyBytes(rootKey)
	if err != nil {
		panic(err)
	}
	return root
}

Output:

{"created_at":"2009-11-17T20:34:58.651387237Z","event_type":"test-event","payload":{"NoClassification":"no classification","Public":"public","Sensitive":"sensitive","Secret":"secret","TaggableMap":{"no-classification":"[REDACTED]","public":"public","secret":"[REDACTED]","sensitive":"[REDACTED]"}}}

func (*Filter) Process 

func (ef *Filter) Process(ctx context.Context, e *eventlogger.Event) (*eventlogger.Event, error)

Process will encrypt or hmac-sha256 string and []byte fields which are tagged as SensitiveClassification. Fields that are tagged SecretClassification will be redacted.

If the event payload satisfies the WrapperPayload interface, then the payload's Wrapper(), HmacSalt() and HmacInfo() will be used to rotate the filter's wrappers for ongoing filtering operations. Events matching this WrapperPayload interface are not sent along in the pipeline and a nil with no errors is immediately returned after the wrapper has been rotated.

If the event payload satisfies the EventWrapperInfo interface, then the payload's EventId(), HmacSalt() and HmacInfo() will be used to for filtering operations for just the single event being processed.

func (*Filter) Reopen 

func (af *Filter) Reopen() error

Reopen is a no op for Filters.

func (*Filter) Rotate 

func (ef *Filter) Rotate(opt ...Option)

Rotate supports rotating the filter's wrapper, salt and info via the options: WithWrapper, WithSalt, WithInfo

func (*Filter) Type 

func (ef *Filter) Type() eventlogger.NodeType

Type describes the type of the node as a Filter.

type FilterOperation 

type FilterOperation string

FilterOperation defines a type for filtering operations like: redact, encrypt, hmac-sha256, etc. Used in combination with a DataClassification, it allows developers to override the default filter operations on tag fields. 

const (
	NoOperation         FilterOperation = ""
	UnknownOperation    FilterOperation = "unknown"
	RedactOperation     FilterOperation = "redact"
	EncryptOperation    FilterOperation = "encrypt"
	HmacSha256Operation FilterOperation = "hmac-sha256"
)

type Option 

type Option func(*options)

Option - how Options are passed as arguments.

func WithInfo 

func WithInfo(info []byte) Option

WithInfo defines optional info.

func WithSalt 

func WithSalt(salt []byte) Option

WithSalt defines optional salt.

func WithWrapper 

func WithWrapper(wrapper wrapping.Wrapper) Option

WithWrapper defines an optional wrapper.

type PointerTag 

type PointerTag struct {

	// Pointer is the pointerstructure pointer string to get/set a key within a
	// map[string]interface{}  See: https://github.com/mitchellh/pointerstructure
	Pointer string

	// Classification is the DataClassification of data pointed to by the
	// Pointer
	Classification DataClassification

	// Filter is the FilterOperation to apply to the data pointed to by the
	// Pointer.  This is optional and the default operations (or overrides) will
	// apply when not specified
	Filter FilterOperation
}

PointerTag provides the pointerstructure pointer string to get/set a key within a map[string]interface{} along with its DataClassification and FilterOperation.

type RotateWrapper 

type RotateWrapper interface {
	// Wrapper to use for event encryption or hmac-sha256 operations
	Wrapper() wrapping.Wrapper

	// HmacSalt to use for event hmac-sha256 operations
	HmacSalt() []byte

	// HmacInfo to use for event hmac-sha256 operations
	HmacInfo() []byte
}

RotateWrapper defines an interface for eventlogger payloads which include rotated wrapper data. This interface allows for the rotation of the wrapper, salt and info

type Taggable 

type Taggable interface {

	// Tags will return a set of pointer tags for the map
	Tags() ([]PointerTag, error)
}

Taggable defines an interface for taggable maps

type TestTaggedMap 

type TestTaggedMap map[string]interface{}

TestTaggedMap is a map that implements the Taggable interface for testing

func (TestTaggedMap) Tags 

func (t TestTaggedMap) Tags() ([]PointerTag, error)

Tags implements the taggable interface for the TestTaggedMap type

Source Files

View all Source files

Directories

Path Synopsis
testing
resources/protopayload

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.