gotidus

README

gotidus

gotidus is a Golang library which allows automatic view generation for every table in an SQL database. The purpose of the views is to anonymize the contents of select columns to ensure that no confidential information leaves the database while still providing access to the data in general.

It is also a port from the tidus Ruby Gem.

Install

go get github.com/Barzahlen/gotidus

Usage

Please see the example below as well as the godoc reference.

Example

fooTable := gotidus.NewTable()
// Define columns on the table to anonymize in a specific way.
// Other columns will just contain their normal value.
// Note: Any column defined but not actually in the table will be ignored.
fooTable.AddAnonymizer(
    "bar",
    gotidus.NewStaticAnonymizer("staticValue", "TEXT"),
)
generator := gotidus.NewGenerator(postgres.NewQueryBuilder())
// Define tables that should have specifically anonymized columns.
// Tables that are not supposed to be anonymized specifically,
// do not have to be defined.
// 
// Note: Any table defined but not actually in the database will be ignored.
generator.AddTable("foo", fooTable)

// Clear existing views
err := generator.ClearViews(db)
if err != nil {
    log.Fatal(err)
}

// ... database migration

// Create new views
err = generator.CreateViews(db)
if err != nil {
    log.Fatal(err)
}

Backup and Restore

You can use the bash example script located in examples to backup and restore databases prepared with tidus easily. tidus_backup_restore.sh can be called with any parameter other than -d|-r|--dump|--restore to get help for it's usage. The tidus_seq_rst.sql file is necessary for restores since it's will reset all sequences after restore for you - it's not necessary for backups only. You also need the tidus_credentials.conf with the IP/DNS, User and Password of the Dump and Restore users. If you use tidus_backup_restore.sh on separate machines for backup and restore, you can split up the credentials file and only provide the information necessary to backup and restore.

Basic usage

Before dumping or restoring you have to provide the tidus_credentials.conf file with all the informations needed for dumping and restoring. Those parameters are not exposed into the commandline due to security considerations. Also manually edit the tidus_backup_restore.sh and check the dump_it and restore_it functions and add the databases you want to dump or restore as well as the database names in your staging environment and the staging user which will get the permissions after restore.

• ./tidus_backup_restore.sh /path/to/tidus_credentials.conf -d /path/to/the/dumps/folder
  • Add all databases you want to dump from in the dump_it function!
• ./tidus_backup_restore.sh /path/to/tidus_credentials.conf -r /path/to/the/dumps/folder <Backup-Set-No>
  • Add all databases you want to restore - as well as the destination database names and users - in the restore_it function!
  • Be sure to have the tidus_seq_rst.sqlin the same folder as the script which is required for a successful restore!

Bugs and Contribution

For bugs and feature requests open an issue on Github. For code contributions fork the repo, make your changes and create a pull request.

Extending functionality

The number of anonymizers implemented so far is limited. A new anonymization strategy can be easily defined through implementation of the gotidus.Anonymizer interface. It is furthermore possible to add support for other databases by implementing the gotidus.QueryBuilder interface.

License

LICENSE

Documentation

Overview

Package gotidus is an SQL anonymization view builder for go

Example:

fooTable := gotidus.NewTable()
// Define columns on the table to anonymize in a specific way.
// Other columns will just contain their normal value.
// Note: Any column defined but not actually in the table will be ignored.
fooTable.AddAnonymizer(
    "bar",
    gotidus.NewStaticAnonymizer("staticValue", "TEXT"),
)
generator := gotidus.NewGenerator(postgres.NewQueryBuilder())
// Define tables that should have specifically anonymized columns.
// Tables that are not supposed to be anonymized specifically,
// do not have to be defined.
//
// Note: Any table defined but not actually in the database will be ignored.
generator.AddTable("foo", fooTable)

// Clear existing views
err := generator.ClearViews(db)
if err != nil {
    log.Fatal(err)
}

// ... database migration

// Create new views
err = generator.CreateViews(db)
if err != nil {
    log.Fatal(err)
}

Index

• Constants
• func FullColumnName(tableName, columnName string) string
• type Anonymizer
• type Generator
  • func NewGenerator(queryBuilder QueryBuilder, options ...GeneratorOption) *Generator
  • func (g *Generator) AddTable(name string, table *Table) *Generator
  • func (g *Generator) ClearViews(db *sql.DB) error
  • func (g *Generator) CreateViews(db *sql.DB) error
  • func (g *Generator) GetTable(name string) *Table
  • func (g *Generator) ViewName(tableName string) string
• type GeneratorOption
  • func WithViewPostfix(viewPostfix string) GeneratorOption
• type NoopAnonymizer
  • func NewNoopAnonymizer() *NoopAnonymizer
  • func (a *NoopAnonymizer) Build(tableName, columnName string) string
• type QueryBuilder
• type StaticAnonymizer
  • func NewStaticAnonymizer(staticValue, dataType string) *StaticAnonymizer
  • func (a *StaticAnonymizer) Build(tableName, columnName string) string
• type Table
  • func NewTable() *Table
  • func (t *Table) AddAnonymizer(columnName string, anonymizer Anonymizer) *Table
  • func (t *Table) GetAnonymizer(columnName string) Anonymizer

Constants

const DefaultViewPostfix = "anonymized"

DefaultViewPostfix defines the postfix given to views to distinguish them from the table names.

Functions

func FullColumnName

func FullColumnName(tableName, columnName string) string

FullColumnName is a helper function that allows building the name based on the table and column names.

Types

type Anonymizer

type Anonymizer interface {
	Build(tableName, columnName string) string
}

Anonymizer is the interface for functions that build the query snippet to anonymize a specific column.

type Generator

type Generator struct {
	// contains filtered or unexported fields
}

Generator is the type orchestrating the view clearing and creation, based on the table config.

func NewGenerator

func NewGenerator(queryBuilder QueryBuilder, options ...GeneratorOption) *Generator

NewGenerator initializes a new Generator object. It requires a QueryBuilder object and can be enhanced with GeneratorOption functions.

func (*Generator) AddTable

func (g *Generator) AddTable(name string, table *Table) *Generator

AddTable adds a Table configuration to the generator with the given name. If this function is called again with the same name, it will overwrite the existing table.

func (*Generator) ClearViews

func (g *Generator) ClearViews(db *sql.DB) error

ClearViews removes any potentially existing views that exist with the configured postfix.

func (*Generator) CreateViews

func (g *Generator) CreateViews(db *sql.DB) error

CreateViews creates views named <table_name>_<postfix> for each table that could be found. It uses the configuration set before CreateViews was called.

func (*Generator) GetTable

func (g *Generator) GetTable(name string) *Table

GetTable retrieves a Table from the config. If a Table was configured for the given name, that Table object will be returned. If no Table was configured for the given name, a blank table configuration is returned.

func (*Generator) ViewName

func (g *Generator) ViewName(tableName string) string

ViewName builds the view name from the table name and the postfix to <table_name>_<postfix>.

type GeneratorOption

type GeneratorOption func(*Generator)

GeneratorOption is a function type following the option function pattern. It can be used to define methods of configuring the Generator object.

func WithViewPostfix

func WithViewPostfix(viewPostfix string) GeneratorOption

WithViewPostfix is a GeneratorOption builder, which allows configuring the view postfix.

type NoopAnonymizer

type NoopAnonymizer struct{}

NoopAnonymizer is an Anonymizer interface implementation which returns the column value is as. It is also the default anonymizer for every column unless otherwise defined.

func NewNoopAnonymizer

func NewNoopAnonymizer() *NoopAnonymizer

NewNoopAnonymizer initializes a new NoopAnonymizer object

func (*NoopAnonymizer) Build

func (a *NoopAnonymizer) Build(tableName, columnName string) string

Build returns the column name build from the table and column name

type QueryBuilder

type QueryBuilder interface {
	ListViewsQuery() string
	DropViewQuery(viewName string) string

	ListTablesQuery() string

	ListColumnsQuery() string

	CreateViewQuery(viewName string, tableName string, columns []string) string
}

QueryBuilder is the interface used to implement support for different databases.

type StaticAnonymizer

type StaticAnonymizer struct {
	// contains filtered or unexported fields
}

StaticAnonymizer is an Anonymizer interfface implementation that ensures that every row returns the same static value.

func NewStaticAnonymizer

func NewStaticAnonymizer(staticValue, dataType string) *StaticAnonymizer

NewStaticAnonymizer initializes a new StaticAnonymizer object

func (*StaticAnonymizer) Build

func (a *StaticAnonymizer) Build(tableName, columnName string) string

Build returns a partial query from the static value and data type given on object initialization. table and column name are ignored here.

type Table

type Table struct {
	// contains filtered or unexported fields
}

Table is the type holding the column configuration

func NewTable

func NewTable() *Table

NewTable initializes a Table object wich blank columns.

func (*Table) AddAnonymizer

func (t *Table) AddAnonymizer(columnName string, anonymizer Anonymizer) *Table

AddAnonymizer allows setting a specific Anonymizer for a column of the given name. If an Anonymizer was previously configured for a column name, it will be overwritten.

func (*Table) GetAnonymizer

func (t *Table) GetAnonymizer(columnName string) Anonymizer

GetAnonymizer retrieves an Anonymizer from the Table configuration. If an Anonymizer was configured for the given name, that Anonymizer will be returned. If no Anonymizer was configured for the given name, the NoopAnonymizer will be returned.