datastore

package
v0.0.13 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 2, 2023 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Import the package and use DataStore interface to interact with the data access layer. If you want DAL to use a Postgres database, ensure you have the following environment variables set to relevant values: [DB_ADMIN_USERNAME], [DB_PORT], [DB_NAME], [DB_ADMIN_PASSWORD], [DB_HOST], [SSL_MODE]. You can also set [LOG_LEVEL] environment variable to debug/trace, if you want logging at a specific level (default is [Info])

Define structs that will be persisted using datastore similar to any gorm Models, for reference https://gorm.io/docs/models.html

  • At least one field must be a primary key with `gorm:"primaryKey"` tag
  • For multi-tenancy support, add `gorm:"column:org_id"` as tag to a filed
  • For revision support to block concurrent updates, add `gorm:"column:revision"` as tag
  • For multi-instance support, add `gorm:"column:instance_id"` as tag

DataStore interface exposes basic methods like Find/FindAll/Upsert/Delete. For richer queries and performing a set of operations within a transaction, please, use GetTransaction() method. For more info, refer to Gorm's transactions page: https://gorm.io/docs/transactions.html

Index

Examples

Constants

View Source 
const (
	DbConfigOrgId      = "multitenant.orgId"      // Name of Postgres run-time config. parameter that will store current user's org. ID
	DbConfigInstanceId = "multitenant.instanceId" // Name of Postgres run-time config. parameter that will store current session's instance ID

	MaxIdleConns = 1
)
View Source 
const (
	// Env. variable names.
	DB_NAME_ENV_VAR           = "DB_NAME"
	DB_PORT_ENV_VAR           = "DB_PORT"
	DB_HOST_ENV_VAR           = "DB_HOST"
	SSL_MODE_ENV_VAR          = "SSL_MODE"
	DB_ADMIN_USERNAME_ENV_VAR = "DB_ADMIN_USERNAME"
	DB_ADMIN_PASSWORD_ENV_VAR = "DB_ADMIN_PASSWORD"

	// SQL Error Codes.
	ERROR_DUPLICATE_KEY      = "SQLSTATE 23505"
	ERROR_DUPLICATE_DATABASE = "SQLSTATE 42P04"
)
View Source 
const (
	DEFAULT_OFFSET = 0
	DEFAULT_LIMIT  = 1000
	DEFAULT_SORTBY = ""
)
View Source 
const (
	// Struct Field Names.
	FIELD_ORGID      = "OrgId"
	FIELD_INSTANCEID = "InstanceId"

	// SQL Columns.
	COLUMN_ORGID      = "org_id"
	COLUMN_INSTANCEID = "instance_id"
	COLUMN_REVISION   = "revision"

	// Messages.
	REVISION_OUTDATED_MSG = "Invalid update - outdated "
)

Variables

This section is empty.

Functions

func DBCreate added in v0.0.6 

func DBCreate(cfg DBConfig) error

Create a Postgres DB using the provided config if it doesn't exist.

func DBExists added in v0.0.6 

func DBExists(cfg DBConfig) bool

Checks if a Postgres DB exists and returns true.

func GetFieldValue added in v0.0.5 

func GetFieldValue(record Record, fieldName, columnName string) (string, bool)

Returns the requested fields value from record, which is a pointer to a struct implementing Record interface. Uses a tag rather than field name to find the desired field. Returns an empty string and false if such a field is not present.

func GetInstanceId added in v0.0.5 

func GetInstanceId(record Record) (string, bool)

Returns the requested InstanceId field's value from record, which is a pointer to a struct implementing Record interface. Uses a tag rather than field name to find the desired field. Returns an empty string and false if such a field is not present.

func GetOrgId 

func GetOrgId(record Record) (string, bool)

Returns the requested OrgId field's value from record, which is a pointer to a struct implementing Record interface. Uses a tag rather than field name to find the desired field. Returns an empty string and false if such a field is not present.

func GetTableName 

func GetTableName(x interface{}) (tableName string)

Extracts struct's name, which will serve as DB table name, using reflection.

func IsColumnPresent added in v0.0.5 

func IsColumnPresent(x Record, tableName, columnName string) bool

func IsMultiInstanced added in v0.0.5 

func IsMultiInstanced(x Record, tableName string, instancerConfigured bool) bool

Checks if multiple deployment instances are supported in the given table.

func IsMultiTenanted added in v0.0.5 

func IsMultiTenanted(x Record, tableName string) bool

Checks if multiple tenants are supported in the given table.

func IsPointerToStruct 

func IsPointerToStruct(x interface{}) (isPtrType bool)

func IsRevisioned added in v0.0.5 

func IsRevisioned(x Record, tableName string) bool

Checks if revisioning is supported in the given table.

func IsRowLevelSecurityRequired added in v0.0.5 

func IsRowLevelSecurityRequired(record Record, tableName string, instancerConfigured bool) bool

Row Level Security to used to partition tables for multi-tenancy and multi-instance support.

func SetFieldValue added in v0.0.5 

func SetFieldValue(record Record, fieldName, columnName, value string) bool

func SetInstanceId added in v0.0.5 

func SetInstanceId(record Record, value string) bool

func TypeName 

func TypeName(x interface{}) string

TypeName returns name of the data type of the given variable.

Types

type DBConfig 

type DBConfig struct {
	// contains filtered or unexported fields
}

func ConfigFromEnv added in v0.0.6 

func ConfigFromEnv(dbName string) DBConfig

Returns DBConfig constructed from the env variables. If dbName is set, it is used instead of DB_NAME env variable. All env variables are required and if not set, this method would panic.

type DataStore 

type DataStore interface {
	Find(ctx context.Context, record Record) error
	FindSoftDeleted(ctx context.Context, record Record) error
	FindAll(ctx context.Context, records interface{}, pagination *Pagination) error
	FindAllIncludingSoftDeleted(ctx context.Context, records interface{}, pagination *Pagination) error
	FindWithFilter(ctx context.Context, filter Record, records interface{}, pagination *Pagination) error
	FindWithFilterIncludingSoftDeleted(ctx context.Context, filter Record, records interface{}, pagination *Pagination) error
	Insert(ctx context.Context, record Record) (int64, error)
	SoftDelete(ctx context.Context, record Record) (int64, error)
	Delete(ctx context.Context, record Record) (int64, error)
	Update(ctx context.Context, record Record) (int64, error)
	Upsert(ctx context.Context, record Record) (int64, error)
	GetTransaction(ctx context.Context, record ...Record) (tx *gorm.DB, err error)

	// Create a DB table for the given struct. Enables RLS in it if it is multi-tenant.
	// Generates Postgres roles and policies based on the provided role mapping and applies them
	// to the created DB table.
	// roleMapping - maps service roles to DB roles to be used for the generated DB table
	// There are 4 possible DB roles to choose from:
	// - READER, which gives read access to all the records in the table
	// - WRITER, which gives read & write access to all the records in the table
	// - TENANT_READER, which gives read access to current tenant's records
	// - TENANT_WRITER, which gives read & write access to current tenant's records.
	Register(ctx context.Context, roleMapping map[string]dbrole.DbRole, records ...Record) error
	Reset()

	GetAuthorizer() authorizer.Authorizer
	GetInstancer() authorizer.Instancer
	Helper() Helper
	TestHelper() TestHelper
}
Example (MultiInstance)

Example for the multi-instance feature, illustrates one can create records for different instances and that each instance context can access the data that belongs to specific instance only. 

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/authorizer"
	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/datastore"
	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/dbrole"
	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/logutils"
)

type Person struct {
	Id         string `gorm:"primaryKey"`
	Name       string
	Age        int
	InstanceId string `gorm:"primaryKey"`
}

func (p Person) String() string {
	return fmt.Sprintf("[%s/%s] %s: %d", p.InstanceId, p.Id, p.Name, p.Age)
}

// Example for the multi-instance feature, illustrates one can create records for different instances
// and that each instance context can access the data that belongs to specific instance only.
func main() {
	uId := "P1337"
	p1 := &Person{uId, "Bob", 31, "Dev"}
	p2 := &Person{uId, "John", 36, "Prod"}
	p3 := &Person{"P3", "Pat", 39, "Dev"}

	SERVICE_ADMIN := "service_admin"
	SERVICE_AUDITOR := "service_auditor"
	mdAuthorizer := &authorizer.MetadataBasedAuthorizer{}
	instancer := &authorizer.SimpleInstancer{}

	ServiceAdminCtx := mdAuthorizer.GetAuthContext("", SERVICE_ADMIN)
	DevInstanceCtx := instancer.WithInstanceId(ServiceAdminCtx, "Dev")
	ProdInstanceCtx := instancer.WithInstanceId(ServiceAdminCtx, "Prod")

	// Initializes the Datastore using the metadata authorizer and connection details obtained from the ENV variables.
	ds, err := datastore.FromEnvWithDB(logutils.GetCompLogger(), mdAuthorizer, instancer, "ExampleDataStore_multiInstance")
	defer ds.Reset()
	if err != nil {
		log.Fatalf("datastore initialization from env errored: %s", err)
	}

	// Registers the necessary structs with their corresponding role mappings.
	roleMapping := map[string]dbrole.DbRole{
		SERVICE_AUDITOR: dbrole.INSTANCE_READER,
		SERVICE_ADMIN:   dbrole.INSTANCE_WRITER,
	}
	if err = ds.Register(context.TODO(), roleMapping, &Person{}); err != nil {
		log.Fatalf("Failed to create DB tables: %+v", err)
	}

	// Inserts a record with a given Id (uId) using the context of the Dev instance.
	rowsAffected, err := ds.Insert(DevInstanceCtx, p1)
	fmt.Println(rowsAffected, err)
	// Inserts another record with the same uId using the context of the Prod instance.
	rowsAffected, err = ds.Insert(ProdInstanceCtx, p2)
	fmt.Println(rowsAffected, err)
	// Inserts a third record with a different uId using the context of the Dev instance.
	rowsAffected, err = ds.Insert(DevInstanceCtx, p3)
	fmt.Println(rowsAffected, err)

	// Finds a record using the context of the Dev instance and the specified uId.
	q1 := &Person{Id: uId}
	err = ds.Find(DevInstanceCtx, q1)
	fmt.Println(q1, err)
	// Finds a record using the context of the Prod instance and the same uId.
	q2 := &Person{Id: uId}
	err = ds.Find(ProdInstanceCtx, q2)
	fmt.Println(q2, err)
	// Finds a record using the correct context of the Dev instance.
	q3 := &Person{Id: "P3"}
	err = ds.Find(DevInstanceCtx, q3)
	fmt.Println(q3, err)
	// Attempts to find a record using the incorrect context of the Prod instance and should error out.
	q4 := &Person{Id: "P3"}
	err = ds.Find(ProdInstanceCtx, q4)
	fmt.Printf("err != nil - %t\n", err != nil)

	// Deletes a record using the context of the Dev instance and the specified uId.
	rowsAffected, err = ds.Delete(DevInstanceCtx, q1)
	fmt.Println(rowsAffected, err)
	// Deletes a record using the context of the Prod instance and the same uId.
	rowsAffected, err = ds.Delete(ProdInstanceCtx, q2)
	fmt.Println(rowsAffected, err)
	// Attempts to delete a record using an invalid context of the Prod instance, which should not affect the database.
	rowsAffected, err = ds.Delete(ProdInstanceCtx, q4)
	fmt.Println(rowsAffected, err)
	// Deletes a record using a valid context of the Dev instance.
	rowsAffected, err = ds.Delete(DevInstanceCtx, q3)
	fmt.Println(rowsAffected, err)
}

Output:

1 <nil>
1 <nil>
1 <nil>
[Dev/P1337] Bob: 31 <nil>
[Prod/P1337] John: 36 <nil>
[Dev/P3] Pat: 39 <nil>
err != nil - true
1 <nil>
1 <nil>
0 <nil>
1 <nil>
Example (MultiTenancy)

Example for the multi-tenancy feature, illustrates one can create records for different tenants and that each tenant context can access the data that belongs to specific tenant only. 

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/authorizer"
	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/datastore"
	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/dbrole"
	"github.com/vmware-labs/multi-tenant-persistence-for-saas/pkg/logutils"
)

type User struct {
	Id    string `gorm:"primaryKey"`
	Name  string
	Age   int
	OrgId string `gorm:"primaryKey"`
}

func (p User) String() string {
	return fmt.Sprintf("[%s/%s] %s: %d", p.OrgId, p.Id, p.Name, p.Age)
}

// Example for the multi-tenancy feature, illustrates one can create records for different tenants
// and that each tenant context can access the data that belongs to specific tenant only.
func main() {
	uId := "P1337"
	p1 := &User{uId, "Bob", 31, "Coke"}
	p2 := &User{uId, "John", 36, "Pepsi"}
	p3 := &User{"P3", "Pat", 39, "Coke"}

	TENANT_ADMIN := "tenant_admin"
	TENANT_AUDITOR := "tenant_auditor"
	mdAuthorizer := &authorizer.MetadataBasedAuthorizer{}
	CokeOrgCtx := mdAuthorizer.GetAuthContext("Coke", TENANT_ADMIN)
	PepsiOrgCtx := mdAuthorizer.GetAuthContext("Pepsi", TENANT_ADMIN)

	// Initializes the Datastore using the metadata authorizer and connection details obtained from the ENV variables.
	ds, err := datastore.FromEnvWithDB(logutils.GetCompLogger(), mdAuthorizer, nil, "ExampleDataStore_multiTenancy")
	defer ds.Reset()
	if err != nil {
		log.Fatalf("datastore initialization from env errored: %s", err)
	}

	// Registers the necessary structs with their corresponding tenant role mappings.
	roleMapping := map[string]dbrole.DbRole{
		TENANT_AUDITOR: dbrole.TENANT_READER,
		TENANT_ADMIN:   dbrole.TENANT_WRITER,
	}
	if err = ds.Register(context.TODO(), roleMapping, &User{}); err != nil {
		log.Fatalf("Failed to create DB tables: %+v", err)
	}

	// Inserts a record with a given Id (uId) using the context of the Coke organization.
	rowsAffected, err := ds.Insert(CokeOrgCtx, p1)
	fmt.Println(rowsAffected, err)
	// Inserts another record with the same uId using the context of the Pepsi organization.
	rowsAffected, err = ds.Insert(PepsiOrgCtx, p2)
	fmt.Println(rowsAffected, err)
	// Inserts a third record with a different uId using the context of the Coke organization.
	rowsAffected, err = ds.Insert(CokeOrgCtx, p3)
	fmt.Println(rowsAffected, err)

	// Finds a record using the context of the Coke organization and the specified uId.
	q1 := &User{Id: uId}
	err = ds.Find(CokeOrgCtx, q1)
	fmt.Println(q1, err)
	// Finds a record using the context of the Pepsi organization and the same uId.
	q2 := &User{Id: uId}
	err = ds.Find(PepsiOrgCtx, q2)
	fmt.Println(q2, err)
	// Finds a record using the correct context of the Coke organization.
	q3 := &User{Id: "P3"}
	err = ds.Find(CokeOrgCtx, q3)
	fmt.Println(q3, err)
	// Attempts to find a record using the incorrect context of the Pepsi organization and should error out.
	q4 := &User{Id: "P3"}
	err = ds.Find(PepsiOrgCtx, q4)
	fmt.Printf("err != nil - %t\n", err != nil)

	// Deletes a record using the context of the Coke organization and the specified uId.
	rowsAffected, err = ds.Delete(CokeOrgCtx, q1)
	fmt.Println(rowsAffected, err)
	// Deletes a record using the context of the Pepsi organization and the same uId.
	rowsAffected, err = ds.Delete(PepsiOrgCtx, q2)
	fmt.Println(rowsAffected, err)
	// Attempts to delete a record using an invalid context of the Pepsi organization, which should not affect the database.
	rowsAffected, err = ds.Delete(PepsiOrgCtx, q4)
	fmt.Println(rowsAffected, err)
	// Deletes a record using a valid context of the Coke organization.
	rowsAffected, err = ds.Delete(CokeOrgCtx, q3)
	fmt.Println(rowsAffected, err)

}

Output:

1 <nil>
1 <nil>
1 <nil>
[Coke/P1337] Bob: 31 <nil>
[Pepsi/P1337] John: 36 <nil>
[Coke/P3] Pat: 39 <nil>
err != nil - true
1 <nil>
1 <nil>
0 <nil>
1 <nil>

func FromConfig 

func FromConfig(l *logrus.Entry, a authorizer.Authorizer, instancer authorizer.Instancer, cfg DBConfig) (d DataStore, err error)

func FromEnv 

func FromEnv(l *logrus.Entry, a authorizer.Authorizer, instancer authorizer.Instancer) (d DataStore, err error)

func FromEnvWithDB added in v0.0.9 

func FromEnvWithDB(l *logrus.Entry, a authorizer.Authorizer, instancer authorizer.Instancer, dbName string) (d DataStore, err error)

type Helper 

type Helper interface {
	FindInTable(ctx context.Context, tableName string, record Record, softDelete bool) (err error)
	FindAllInTable(ctx context.Context, tableName string, records interface{}, pagination *Pagination, softDelete bool) error
	FindWithFilterInTable(ctx context.Context, tableName string, record Record, records interface{}, pagination *Pagination, softDelete bool) (err error)
	GetDBTransaction(ctx context.Context, tableName string, record Record) (tx *gorm.DB, err error)

	RegisterHelper(ctx context.Context, roleMapping map[string]dbrole.DbRole, tableName string, record Record) error
}

type Pagination added in v0.0.4 

type Pagination struct {
	Offset int
	Limit  int
	SortBy string
}

func DefaultPagination added in v0.0.4 

func DefaultPagination() *Pagination

func GetPagination added in v0.0.4 

func GetPagination(offset int, limit int, sortBy string) *Pagination

func NoPagination added in v0.0.4 

func NoPagination() *Pagination

type Record 

type Record interface {
}

func GetRecordInstanceFromSlice 

func GetRecordInstanceFromSlice(x interface{}) Record

type TenancyInfo added in v0.0.5 

type TenancyInfo struct {
	DbRole     dbrole.DbRole
	InstanceId string
	OrgId      string
}

type TestHelper 

type TestHelper interface {
	DropTables(records ...Record) error                       // Drop DB tables by records
	Truncate(tableNames ...string) error                      // Truncates DB tables
	TruncateCascade(cascade bool, tableNames ...string) error // Truncates DB tables, with an option to truncate them in a cascading fashion
	HasTable(tableName string) (bool, error)                  // Checks if DB table exists
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.