dsc

package module
Version: v0.16.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 20, 2020 License: Apache-2.0 Imports: 25 Imported by: 26

README

Datastore Connectivity (dsc)

Datastore Connectivity library for Go. GoDoc

This library is compatible with Go 1.10+

Please refer to CHANGELOG.md if you encounter breaking changes.

Motivation

This library was developed as part of dsunit (Datastore unit testibility library) to provide unified access to SQL, noSQL, or any other store that deals with structured data in SQL-ish way.

Usage:

The following is a very simple example of CRUD operations with dsc

package main

import (
)


func main() {


	config := dsc.NewConfig("mysql", "[user]:[password]@[url]", "user:root,password:dev,url:tcp(127.0.0.1:3306)/mydb?parseTime=true")
	factory := NewManagerFactory()
	manager, err := factory.Create(config)
    if err != nil {
        panic(err.Error())
	}

    // manager := factory.CreateFromURL("file:///etc/myapp/datastore.json")
  
  
    interest := Interest{}
    
    success, err:= manager.ReadSingle(&interest, SELECT id, name, expiry, category FROM interests WHERE id = ?", []interface{}{id},nil)
	if err != nil {
        panic(err.Error())
	}

    var intersts = make([]Interest, 0)
    err:= manager.ReadAll(&interests, SELECT id, name, expiry, category FROM interests", nil ,nil)
    if err != nil {
        panic(err.Error())
    }

    
    intersts := []Interest {
        Interest{Name:"Abc", ExpiryTimeInSecond:3600, Category:"xyz"},
        Interest{Name:"Def", ExpiryTimeInSecond:3600, Category:"xyz"},
        Interest{Id:20, Name:"Ghi", ExpiryTimeInSecond:3600, Category:"xyz"},
    }


	inserted, updated, err:= manager.PersistAll(&intersts, "interests", nil)
	if err != nil {
        panic(err.Error())
   	}
    deleted, err := manager.DeleteAll(&intersts, "intersts", nil)
    if err != nil {
        panic(err.Error())
   	}
 	fmt.Printf("Inserted %v, updated: %v\n", deleted)
  
}

More examples illustrating the use of the API are located in the examples directory.

Details about the API are available in the docs directory.

Prerequisites

Go version v1.5+ is required.

To install the latest stable version of Go, visit http://golang.org/dl/

Target

Installation:

  1. Install Go 1.5+ and setup your environment as Documented here.
  2. Get the client in your GOPATH : go get github.com/viant/dsc
  • To update the client library: go get -u github.com/viant/dsc
Some Hints:
  • To run a go program directly: go run <filename.go>
  • to build: go build -o <output> <filename.go>

API Documentation

API documentation is available in the docs directory.

Tests

This library is packaged with a number of tests. Tests require Testify library.

Before running the tests, you need to update the dependencies:

$ go get .

To run all the test cases with race detection:

$ go test

Examples

A simple CRUD applications is provided in the examples directory.

GoCover

GoCover

License

The source code is made available under the terms of the Apache License, Version 2, as stated in the file LICENSE.

Individual files may be made available under their own specific license, all compatible with Apache License, Version 2. Please see individual files for details.

Credits and Acknowledgements

Library Author: Adrian Witas

Contributors: Sudhakaran Dharmaraj

Documentation

Overview

Package dsc - datastore connectivity library This library provides connection capabilities to SQL, noSQL datastores or structured files providing sql layer on top of it.

For native database/sql it is just a ("database/sql") proxy, and for noSQL it supports simple SQL that is being translated to put/get,scan,batch native NoSQL operations.

Datastore Manager implements read, batch (no insert nor update), and delete operations. Read operation requires data record mapper, Persist operation requires dml provider. Delete operation requries key provider.

Datastore Manager provides default record mapper and dml/key provider for a struct, if no actual implementation is passed in.

The following tags can be used

1 column - name of datastore field/column

2 autoincrement - boolean flag to use autoincrement, in this case on insert the value can be automatically set back on application model class

3 primaryKey - boolean flag primary key

4 dateLayout - date layout check string to time.Time conversion

4 dateFormat - date format check java simple date format

5 sequence - name of sequence used to generate next id

6 transient - boolean flag to not map a field with record data

7 valueMap - value maping that will be applied after fetching a record and before writing it to datastore.

For instance valueMap:"yes:true,no:false" would map yes to true, no to false

Usage:

	type Interest struct {
		Id int	`autoincrement:"true"`
		Name string
		ExpiryTimeInSecond int `column:"expiry"`
		Category string
	}

 	manager := factory.CreateFromURL("file:///etc/mystore-config.json")
        interest := Interest{}

    	intersts = make([]Interest, 0)
        err:= manager.ReadAll(&interests, SELECT id, name, expiry, category FROM interests", nil ,nil)
    	if err != nil {
        	panic(err.Error())
    	}
    	...

	inserted, updated, err:= manager.PersistAll(&intersts, "interests", nil)
	if err != nil {
        	panic(err.Error())
   	}

Index

Constants

View Source
const (
	//SQLTypeInsert 0 constant for DML insert statement provider.
	SQLTypeInsert = 0
	//SQLTypeUpdate 1 constant for DML update statement provider.
	SQLTypeUpdate = 1
	//SQLTypeDelete 2 constant for DML delete statement provider.
	SQLTypeDelete = 2
)
View Source
const BatchSizeKey = "batchSize"

BatchSizeKey represents a config batch size parameter

Variables

View Source
var BulkInsertAllType = "insertAll"
View Source
var CopyLocalInsert = "copyLocalInsert"
View Source
var UnionSelectInsert = "unionSelectInsert"

Functions

func NewBooleanPredicate

func NewBooleanPredicate(leftOperand bool, operator string) toolbox.Predicate

NewBooleanPredicate returns a new boolean predicate. It takes left operand and logical operator: 'OR' or 'AND'

func NewSQLCriteriaPredicate

func NewSQLCriteriaPredicate(parameters toolbox.Iterator, sqlCriteria *SQLCriteria) (toolbox.Predicate, error)

NewSQLCriteriaPredicate create a new sql criteria predicate, it takes binding parameters iterator, and actual criteria.

func NewSQLCriterionPredicate

func NewSQLCriterionPredicate(criterion *SQLCriterion, parameters toolbox.Iterator) (toolbox.Predicate, error)

NewSQLCriterionPredicate create a new predicate for passed in SQLCriterion

func NewSQLDatastoreDialect

func NewSQLDatastoreDialect(tablesSQL, sequenceSQL, schemaSQL, allSchemaSQL, keySQL, disableForeignKeyCheck, enableForeignKeyCheck, autoIncrementSQL, tableInfoSQL string, schmeaIndex int, dialect DatastoreDialect) *sqlDatastoreDialect

NewSQLDatastoreDialect creates a new default sql dialect

func NewSQLResult

func NewSQLResult(rowsAffected, lastInsertID int64) sql.Result

NewSQLResult returns a new SqlResult

func RegisterDatastoreDialect

func RegisterDatastoreDialect(driver string, dialectable DatastoreDialect)

RegisterDatastoreDialect register DatastoreDialect for a driver.

func RegisterManagerFactory

func RegisterManagerFactory(driver string, factory ManagerFactory)

RegisterManagerFactory registers manager factory for passed in driver.

func ScanRow

func ScanRow(scanner Scanner) ([]interface{}, []string, error)

ScanRow takes scanner to scans row.

func StdoutLogger

func StdoutLogger(format string, args ...interface{})

StdoutLogger represents stdout logger

func VoidLogger

func VoidLogger(format string, args ...interface{})

VoidLogger represent logger that do not log

Types

type AbstractConnection

type AbstractConnection struct {
	Connection
	// contains filtered or unexported fields
}

AbstractConnection represents an abstract connection

func NewAbstractConnection

func NewAbstractConnection(config *Config, connectionPool chan Connection, connection Connection) *AbstractConnection

NewAbstractConnection create a new abstract connection

func (*AbstractConnection) Begin

func (ac *AbstractConnection) Begin() error

Begin starts a transaction - this method is an abstract method

func (*AbstractConnection) Close

func (ac *AbstractConnection) Close() error

Close closes connection if pool is full or send it back to the pool

func (*AbstractConnection) Commit

func (ac *AbstractConnection) Commit() error

Commit finishes current transaction - this method is an abstract method

func (*AbstractConnection) Config

func (ac *AbstractConnection) Config() *Config

Config returns a datastore config

func (*AbstractConnection) ConnectionPool

func (ac *AbstractConnection) ConnectionPool() chan Connection

ConnectionPool returns a connection channel

func (*AbstractConnection) LastUsed

func (ac *AbstractConnection) LastUsed() *time.Time

LastUsed returns a last used time

func (*AbstractConnection) Rollback

func (ac *AbstractConnection) Rollback() error

Rollback - discards transaction - this method is an abstract method

func (*AbstractConnection) SetLastUsed

func (ac *AbstractConnection) SetLastUsed(ts *time.Time)

SetLastUsed sets last used time

type AbstractConnectionProvider

type AbstractConnectionProvider struct {
	ConnectionProvider
	// contains filtered or unexported fields
}

AbstractConnectionProvider represents an abstract/superclass ConnectionProvider

func NewAbstractConnectionProvider

func NewAbstractConnectionProvider(config *Config, connectionPool chan Connection, connectionProvider ConnectionProvider) *AbstractConnectionProvider

NewAbstractConnectionProvider create a new AbstractConnectionProvider

func (*AbstractConnectionProvider) Close

func (cp *AbstractConnectionProvider) Close() error

Close closes a datastore connection or returns it to the pool (Config.PoolSize and Config.MaxPoolSize).

func (*AbstractConnectionProvider) Config

func (cp *AbstractConnectionProvider) Config() *Config

Config returns a datastore config,

func (*AbstractConnectionProvider) ConnectionPool

func (cp *AbstractConnectionProvider) ConnectionPool() chan Connection

ConnectionPool returns a ConnectionPool

func (*AbstractConnectionProvider) Get

Get returns a new datastore connection or error.

func (*AbstractConnectionProvider) SpawnConnectionIfNeeded

func (cp *AbstractConnectionProvider) SpawnConnectionIfNeeded()

SpawnConnectionIfNeeded creates a new connection if connection pool has not reached size controlled by Config.PoolSize

type AbstractManager

type AbstractManager struct {
	Manager
	// contains filtered or unexported fields
}

AbstractManager represent general abstraction for datastore implementation. Note that ExecuteOnConnection, ReadAllOnWithHandlerOnConnection may need to be implemented for particular datastore.

func NewAbstractManager

func NewAbstractManager(config *Config, connectionProvider ConnectionProvider, self Manager) *AbstractManager

NewAbstractManager create a new abstract manager, it takes config, conneciton provider, and target (sub class) manager

func (*AbstractManager) Acquire added in v0.7.0

func (m *AbstractManager) Acquire()

Acquire if max request per second is specified this function will throttle any request exceeding specified max

func (*AbstractManager) ClassifyDataAsInsertableOrUpdatable

func (m *AbstractManager) ClassifyDataAsInsertableOrUpdatable(connection Connection, dataPointer interface{}, table string, provider DmlProvider) ([]interface{}, []interface{}, error)

ClassifyDataAsInsertableOrUpdatable classifies passed in data as insertable or updatable.

func (*AbstractManager) Config

func (m *AbstractManager) Config() *Config

Config returns a config.

func (*AbstractManager) ConnectionProvider

func (m *AbstractManager) ConnectionProvider() ConnectionProvider

ConnectionProvider returns a connection provider.

func (*AbstractManager) DeleteAll

func (m *AbstractManager) DeleteAll(dataPointer interface{}, table string, keyProvider KeyGetter) (deleted int, err error)

DeleteAll deletes all rows for passed in table, key provider is used to extract primary keys. It returns number of deleted rows or error.

func (*AbstractManager) DeleteAllOnConnection

func (m *AbstractManager) DeleteAllOnConnection(connection Connection, dataPointer interface{}, table string, keyProvider KeyGetter) (deleted int, err error)

DeleteAllOnConnection deletes all rows on connection from table, key provider is used to extract primary keys. It returns number of deleted rows or error. If driver allows this operation is executed in one transaction.

func (*AbstractManager) DeleteSingle

func (m *AbstractManager) DeleteSingle(dataPointer interface{}, table string, keyProvider KeyGetter) (bool, error)

DeleteSingle deletes single row from table on for passed in data pointer, key provider is used to extract primary keys. It returns boolean if successful, or error.

func (*AbstractManager) DeleteSingleOnConnection

func (m *AbstractManager) DeleteSingleOnConnection(connection Connection, dataPointer interface{}, table string, keyProvider KeyGetter) (bool, error)

DeleteSingleOnConnection deletes data on connection from table on for passed in data pointer, key provider is used to extract primary keys. It returns true if successful.

func (*AbstractManager) Execute

func (m *AbstractManager) Execute(sql string, sqlParameters ...interface{}) (result sql.Result, err error)

Execute executes passed in sql with parameters. It returns sql result, or an error.

func (*AbstractManager) ExecuteAll

func (m *AbstractManager) ExecuteAll(sqls []string) ([]sql.Result, error)

ExecuteAll passed in SQL. It returns sql result, or an error.

func (*AbstractManager) ExecuteAllOnConnection

func (m *AbstractManager) ExecuteAllOnConnection(connection Connection, sqls []string) ([]sql.Result, error)

ExecuteAllOnConnection executes passed in SQL on connection. It returns sql result, or an error.

func (*AbstractManager) ExpandSQL

func (m *AbstractManager) ExpandSQL(sql string, arguments []interface{}) string

ExpandSQL expands sql with passed in arguments

func (*AbstractManager) PersistAll

func (m *AbstractManager) PersistAll(dataPointer interface{}, table string, provider DmlProvider) (int, int, error)

PersistAll persists all table rows, dmlProvider is used to generate insert or update statement. It returns number of inserted, updated or error. If driver allows this operation is executed in one transaction.

func (*AbstractManager) PersistAllOnConnection

func (m *AbstractManager) PersistAllOnConnection(connection Connection, dataPointer interface{}, table string, provider DmlProvider) (inserted int, updated int, err error)

PersistAllOnConnection persists on connection all table rows, dmlProvider is used to generate insert or update statement. It returns number of inserted, updated or error.

func (*AbstractManager) PersistData

func (m *AbstractManager) PersistData(connection Connection, data interface{}, table string, keySetter KeySetter, sqlProvider func(item interface{}) *ParametrizedSQL) (int, error)

PersistData batch data on connection on table, keySetter is used to optionally set autoincrement column, sqlProvider handler will generate ParametrizedSQL with Insert or Update statement.

func (*AbstractManager) PersistSingle

func (m *AbstractManager) PersistSingle(dataPointer interface{}, table string, provider DmlProvider) (inserted int, updated int, err error)

PersistSingle persists single table row, dmlProvider is used to generate insert or update statement. It returns number of inserted, updated or error.

func (*AbstractManager) PersistSingleOnConnection

func (m *AbstractManager) PersistSingleOnConnection(connection Connection, dataPointer interface{}, table string, provider DmlProvider) (inserted int, updated int, err error)

PersistSingleOnConnection persists on connection single table row, dmlProvider is used to generate insert or udpate statement. It returns number of inserted, updated or error.

func (AbstractManager) ReadAll

func (m AbstractManager) ReadAll(resultSlicePointer interface{}, query string, queryParameters []interface{}, mapper RecordMapper) error

ReadAll executes query with parameters and fetches all table rows. The row is mapped to result slice pointer with record mapper.

func (*AbstractManager) ReadAllOnConnection

func (m *AbstractManager) ReadAllOnConnection(connection Connection, resultSlicePointer interface{}, query string, queryParameters []interface{}, mapper RecordMapper) error

ReadAllOnConnection executes query with parameters on passed in connection and fetches all table rows. The row is mapped to result slice pointer with record mapper.

func (*AbstractManager) ReadAllWithHandler

func (m *AbstractManager) ReadAllWithHandler(query string, queryParameters []interface{}, readingHandler func(scanner Scanner) (toContinue bool, err error)) error

ReadAllWithHandler executes query with parameters and for each fetch row call reading handler with a scanner, to continue reading next row, scanner needs to return true.

func (*AbstractManager) ReadSingle

func (m *AbstractManager) ReadSingle(resultPointer interface{}, query string, queryParameters []interface{}, mapper RecordMapper) (success bool, err error)

ReadSingle executes query with parameters and reads on connection single table row. The row is mapped to result pointer with record mapper.

func (*AbstractManager) ReadSingleOnConnection

func (m *AbstractManager) ReadSingleOnConnection(connection Connection, resultPointer interface{}, query string, queryParameters []interface{}, mapper RecordMapper) (success bool, err error)

ReadSingleOnConnection executes query with parameters on passed in connection and reads single table row. The row is mapped to result pointer with record mapper.

func (*AbstractManager) RegisterDescriptorIfNeeded

func (m *AbstractManager) RegisterDescriptorIfNeeded(table string, instance interface{}) (*TableDescriptor, error)

RegisterDescriptorIfNeeded register a table descriptor if there it is not present, returns a pointer to a table descriptor.

func (*AbstractManager) TableDescriptorRegistry

func (m *AbstractManager) TableDescriptorRegistry() TableDescriptorRegistry

TableDescriptorRegistry returns a table descriptor registry

type BaseStatement

type BaseStatement struct {
	*SQLCriteria
	SQL     string
	Table   string
	Alias   string
	Columns []*SQLColumn
}

BaseStatement represents a base query and dml statement

func (BaseStatement) ColumnNames

func (bs BaseStatement) ColumnNames() []string

ColumnNames returns a column names.

type Column

type Column interface {
	ColumnType
	// Name returns the name or alias of the TableColumn.
	Name() string
}

Column represents TableColumn type interface (compabible with *sql.ColumnType

func NewColumn

func NewColumn(name, typeName string, length, precision, scale *int64, scanType reflect.Type, nullable *bool) Column

NewColumn create new TableColumn

func NewSimpleColumn

func NewSimpleColumn(name, typeName string) Column

NewSimpleColumn create simple TableColumn name

type ColumnType added in v0.16.0

type ColumnType interface {
	// Length returns the TableColumn type length for variable length TableColumn types such
	// as text and binary field types. If the type length is unbounded the value will
	// be math.MaxInt64 (any database limits will still apply).
	// If the TableColumn type is not variable length, such as an int, or if not supported
	// by the driver ok is false.
	Length() (length int64, ok bool)

	// DecimalSize returns the scale and precision of a decimal type.
	// If not applicable or if not supported ok is false.
	DecimalSize() (precision, scale int64, ok bool)

	// ScanType returns a Go type suitable for scanning into using Rows.Scan.
	// If a driver does not support this property ScanType will return
	// the type of an empty interface.
	ScanType() reflect.Type

	// Nullable returns whether the TableColumn may be null.
	// If a driver does not support this property ok will be false.
	Nullable() (nullable, ok bool)

	// DatabaseTypeName returns the database system name of the TableColumn type. If an empty
	// string is returned the driver type name is not supported.
	// Consult your driver documentation for a list of driver data types. Length specifiers
	// are not included.
	// Common type include "VARCHAR", "TEXT", "NVARCHAR", "DECIMAL", "BOOL", "INT", "BIGINT".
	DatabaseTypeName() string
}

type ColumnValueProvider

type ColumnValueProvider interface {
	ColumnValues() ([]interface{}, error)
}

ColumnValueProvider provides column values

type Config

type Config struct {
	URL                 string
	DriverName          string
	PoolSize            int
	MaxPoolSize         int
	Descriptor          string
	Parameters          map[string]interface{}
	Credentials         string
	MaxRequestPerSecond int

	CredConfig *cred.Config `json:"-"`
	// contains filtered or unexported fields
}

Config represent datastore config.

func NewConfig

func NewConfig(driverName string, descriptor string, encodedParameters string) *Config

NewConfig creates new Config, it takes the following parameters descriptor - optional datastore connection string with macros that will be looked epxanded from for instance [user]:[password]@[url] encodedParameters should be in the following format: <key1>:<value1>, ...,<keyN>:<valueN>

func NewConfigFromURL

func NewConfigFromURL(URL string) (*Config, error)

NewConfigFromUrl returns new config from url

func NewConfigWithParameters

func NewConfigWithParameters(driverName string, descriptor string, credential string, parameters map[string]interface{}) (*Config, error)

NewConfigWithParameters creates a new config with parameters

func (*Config) ApplyCredentials added in v0.15.0

func (c *Config) ApplyCredentials(config *cred.Config) error

func (*Config) Clone added in v0.10.1

func (c *Config) Clone() *Config

Clone clones config

func (*Config) DsnDescriptor added in v0.8.1

func (c *Config) DsnDescriptor() (string, error)

DsnDescriptor return dsn expanded descriptor or error

func (*Config) Get

func (c *Config) Get(name string) string

Get returns value for passed in parameter name or panic - please use Config.Has to check if value is present.

func (*Config) GetBoolean

func (c *Config) GetBoolean(name string, defaultValue bool) bool

GetBoolean returns value for passed in parameter name or defaultValue

func (*Config) GetDateLayout

func (c *Config) GetDateLayout() string

GetDateLayout returns date layout

func (*Config) GetDuration

func (c *Config) GetDuration(name string, multiplier time.Duration, defaultValue time.Duration) time.Duration

GetDuration returns value for passed in parameter name or defaultValue

func (*Config) GetFloat

func (c *Config) GetFloat(name string, defaultValue float64) float64

GetFloat returns value for passed in parameter name or defaultValue

func (*Config) GetInt

func (c *Config) GetInt(name string, defaultValue int) int

GetInt returns value for passed in parameter name or defaultValue

func (*Config) GetMap

func (c *Config) GetMap(name string) map[string]interface{}

Get returns value for passed in parameter name or panic - please use Config.Has to check if value is present.

func (*Config) GetString

func (c *Config) GetString(name string, defaultValue string) string

GetString returns value for passed in parameter name or defaultValue

func (*Config) Has

func (c *Config) Has(name string) bool

Has returns true if parameter with passed in name is present, otherwise it returns false.

func (*Config) HasDateLayout

func (c *Config) HasDateLayout() bool

HasDateLayout returns true if config has date layout, it checks dateLayout or dateFormat parameter names.

func (*Config) Init

func (c *Config) Init() error

Init makes parameter map from encoded parameters if presents, expands descriptor with parameter value using [param_name] matching pattern.

type Connection

type Connection interface {
	Config() *Config

	ConnectionPool() chan Connection

	//Close closes connection or it returns it back to the pool
	Close() error

	CloseNow() error //closes connecition, it does not return it back to the pool

	Unwrap(target interface{}) interface{}

	LastUsed() *time.Time

	SetLastUsed(ts *time.Time)

	TransactionManager
}

Connection represents a datastore connection

type ConnectionProvider

type ConnectionProvider interface {
	Get() (Connection, error)

	Config() *Config

	ConnectionPool() chan Connection

	SpawnConnectionIfNeeded()

	NewConnection() (Connection, error)

	Close() error
}

ConnectionProvider represents a datastore connection provider.

type DatastoreDialect

type DatastoreDialect interface {
	GetDatastores(manager Manager) ([]string, error)

	GetTables(manager Manager, datastore string) ([]string, error)

	DropTable(manager Manager, datastore string, table string) error

	CreateTable(manager Manager, datastore string, table string, specification interface{}) error

	CanCreateDatastore(manager Manager) bool

	CreateDatastore(manager Manager, datastore string) error

	CanDropDatastore(manager Manager) bool

	DropDatastore(manager Manager, datastore string) error

	GetCurrentDatastore(manager Manager) (string, error)

	//GetSequence returns a sequence number
	GetSequence(manager Manager, name string) (int64, error)

	//GetKeyName returns a name of TableColumn name that is a key, or coma separated list if complex key
	GetKeyName(manager Manager, datastore, table string) string

	//GetColumns returns TableColumn info
	GetColumns(manager Manager, datastore, table string) ([]Column, error)

	//IsAutoincrement returns true if autoicrement
	IsAutoincrement(manager Manager, datastore, table string) bool

	//Flag if data store can batch batch
	CanPersistBatch() bool

	//BulkInsert type
	BulkInsertType() string

	//DisableForeignKeyCheck disables fk check
	DisableForeignKeyCheck(manager Manager, connection Connection) error

	//EnableForeignKeyCheck disables fk check
	EnableForeignKeyCheck(manager Manager, connection Connection) error

	//IsKeyCheckSwitchSessionLevel returns true if key check is controlled on connection level (as opposed to globally on table level)
	IsKeyCheckSwitchSessionLevel() bool

	//Normalizes sql i.e for placeholders:   dsc uses '?' for placeholder if some dialect use difference this method should take care of it
	NormalizeSQL(SQL string) string

	//EachTable iterate all current connection manager datastore tables
	EachTable(manager Manager, handler func(table string) error) error

	//ShowCreateTable returns DDL showing create table statement or error
	ShowCreateTable(manager Manager, table string) (string, error)

	//Init initializes connection
	Init(manager Manager, connection Connection) error

	//CanHandleTransaction returns true if driver can handle transaction
	CanHandleTransaction() bool

	//Checks if database is online
	Ping(manager Manager) error
}

DatastoreDialect represents datastore dialects.

func GetDatastoreDialect

func GetDatastoreDialect(driver string) DatastoreDialect

GetDatastoreDialect returns DatastoreDialect for passed in driver.

func NewDefaultDialect

func NewDefaultDialect() DatastoreDialect

NewDefaultDialect crates a defulat dialect. DefaultDialect can be used as a embeddable struct (super class).

type DefaultDialect

type DefaultDialect struct{}

func (DefaultDialect) BulkInsertType added in v0.9.0

func (d DefaultDialect) BulkInsertType() string

func (DefaultDialect) CanCreateDatastore

func (d DefaultDialect) CanCreateDatastore(manager Manager) bool

func (DefaultDialect) CanDropDatastore

func (d DefaultDialect) CanDropDatastore(manager Manager) bool

func (DefaultDialect) CanHandleTransaction added in v0.3.0

func (d DefaultDialect) CanHandleTransaction() bool

func (DefaultDialect) CanPersistBatch

func (d DefaultDialect) CanPersistBatch() bool

func (DefaultDialect) CreateDatastore

func (d DefaultDialect) CreateDatastore(manager Manager, datastore string) error

func (DefaultDialect) CreateTable

func (d DefaultDialect) CreateTable(manager Manager, datastore string, table string, options interface{}) error

func (DefaultDialect) DisableForeignKeyCheck

func (d DefaultDialect) DisableForeignKeyCheck(manager Manager, connection Connection) error

DisableForeignKeyCheck disables fk check

func (DefaultDialect) DropDatastore

func (d DefaultDialect) DropDatastore(manager Manager, datastore string) error

func (DefaultDialect) DropTable

func (d DefaultDialect) DropTable(manager Manager, datastore string, table string) error

func (DefaultDialect) EachTable

func (d DefaultDialect) EachTable(manager Manager, handler func(table string) error) error

EachTable iterates each datastore table

func (DefaultDialect) EnableForeignKeyCheck

func (d DefaultDialect) EnableForeignKeyCheck(manager Manager, connection Connection) error

DisableForeignKeyCheck disables fk check

func (DefaultDialect) GetColumns

func (d DefaultDialect) GetColumns(manager Manager, datastore, table string) ([]Column, error)

func (DefaultDialect) GetCurrentDatastore

func (d DefaultDialect) GetCurrentDatastore(manager Manager) (string, error)

func (DefaultDialect) GetDatastores

func (d DefaultDialect) GetDatastores(manager Manager) ([]string, error)

func (DefaultDialect) GetKeyName

func (d DefaultDialect) GetKeyName(manager Manager, datastore, table string) string

func (DefaultDialect) GetSequence

func (d DefaultDialect) GetSequence(manager Manager, name string) (int64, error)

func (DefaultDialect) GetTables

func (d DefaultDialect) GetTables(manager Manager, datastore string) ([]string, error)

func (DefaultDialect) Init

func (d DefaultDialect) Init(manager Manager, connection Connection) error

func (DefaultDialect) IsAutoincrement

func (d DefaultDialect) IsAutoincrement(manager Manager, datastore, table string) bool

func (DefaultDialect) IsKeyCheckSwitchSessionLevel added in v0.14.0

func (d DefaultDialect) IsKeyCheckSwitchSessionLevel() bool

func (DefaultDialect) NormalizeSQL added in v0.3.0

func (d DefaultDialect) NormalizeSQL(SQL string) string

func (DefaultDialect) Ping added in v0.6.0

func (d DefaultDialect) Ping(manager Manager) error

func (DefaultDialect) ShowCreateTable

func (d DefaultDialect) ShowCreateTable(manager Manager, table string) (string, error)

type DelimiteredRecord

type DelimiteredRecord struct {
	Columns   []string
	Delimiter string
	Record    map[string]interface{}
}

type DmlBuilder

type DmlBuilder struct {
	TableDescriptor *TableDescriptor
	NonPkColumns    *[]string
	Columns         *[]string
	InsertSQL       string
	UpdateSQL       string
	DeleteSQL       string
}

DmlBuilder represents a insert,update,delete statement builder.

func NewDmlBuilder

func NewDmlBuilder(descriptor *TableDescriptor) *DmlBuilder

NewDmlBuilder returns a new DmlBuilder for passed in table descriptor.

func (*DmlBuilder) GetParametrizedSQL

func (b *DmlBuilder) GetParametrizedSQL(sqlType int, valueProvider func(column string) interface{}) *ParametrizedSQL

GetParametrizedSQL returns GetParametrizedSQL for passed in sqlType, and value provider.

type DmlParser

type DmlParser struct {
	// contains filtered or unexported fields
}

DmlParser represents dml parser.

func NewDmlParser

func NewDmlParser() *DmlParser

NewDmlParser creates a new NewDmlParser

func (*DmlParser) Parse

func (dp *DmlParser) Parse(input string) (*DmlStatement, error)

Parse parses input to create DmlStatement.

type DmlProvider

type DmlProvider interface {
	Get(operationType int, instance interface{}) *ParametrizedSQL

	KeySetter
	KeyGetter
}

DmlProvider represents dml generator, which is responsible for providing parametrized sql, it takes operation type: SqlTypeInsert = 0 SqlTypeUpdate = 1 SqlTypeDelete = 2 and instance to the application abstraction

func NewDmlProviderIfNeeded

func NewDmlProviderIfNeeded(provider DmlProvider, table string, targetType reflect.Type) (DmlProvider, error)

NewDmlProviderIfNeeded returns a new NewDmlProvider for a table and target type if passed provider was nil.

func NewMapDmlProvider

func NewMapDmlProvider(descriptor *TableDescriptor) DmlProvider

type DmlStatement

type DmlStatement struct {
	*BaseStatement
	Type   string
	Values []interface{}
}

DmlStatement represents dml statement.

func (DmlStatement) ColumnValueMap

func (ds DmlStatement) ColumnValueMap(parameters toolbox.Iterator) (map[string]interface{}, error)

ColumnValueMap returns map of column with its values extracted from passed in parameters

func (DmlStatement) ColumnValues

func (ds DmlStatement) ColumnValues(parameters toolbox.Iterator) ([]interface{}, error)

ColumnValues returns values of columns extracted from binding parameters

type FileManager

type FileManager struct {
	*AbstractManager
	// contains filtered or unexported fields
}

FileManager represents a line delimiter, JSON file manager. Current implementation is brute/force full file scan on each operation. Do not use it on production, it is not performance optimized. Idea behind is to be able to load data log and test it. You can easily add other managers providing your custom encoder and decoder factories i.e. protobuf, avro.

func NewFileManager

func NewFileManager(encoderFactory toolbox.EncoderFactory, decoderFactory toolbox.DecoderFactory, valuesDelimiter string, config *Config) *FileManager

NewFileManager creates a new file manager.

func (*FileManager) ExecuteOnConnection

func (m *FileManager) ExecuteOnConnection(connection Connection, sql string, sqlParameters []interface{}) (sql.Result, error)

ExecuteOnConnection executs passed in sql on connection. It takes connection, sql and sql parameters. It returns number of rows affected, or error. This method support basic insert, updated and delete operations.

func (*FileManager) Init

func (m *FileManager) Init() error

func (*FileManager) PersistTableData

func (m *FileManager) PersistTableData(tableURL string, data []byte) error

func (*FileManager) ReadAllOnWithHandlerOnConnection

func (m *FileManager) ReadAllOnWithHandlerOnConnection(connection Connection, query string, sqlParameters []interface{}, readingHandler func(scanner Scanner) (toContinue bool, err error)) error

ReadAllOnWithHandlerOnConnection reads all records on passed in connection.

type FileScanner

type FileScanner struct {
	Values map[string]interface{}
	// contains filtered or unexported fields
}

FileScanner represents a file scanner to transfer record to destinations.

func NewFileScanner

func NewFileScanner(config *Config, columns []string, columnTypes []ColumnType) *FileScanner

NewFileScanner create a new file scanner, it takes config, and columns as parameters.

func (*FileScanner) ColumnTypes added in v0.16.0

func (s *FileScanner) ColumnTypes() ([]ColumnType, error)

Columns returns columns of the processed record.

func (*FileScanner) Columns

func (s *FileScanner) Columns() ([]string, error)

Columns returns columns of the processed record.

func (*FileScanner) Scan

func (s *FileScanner) Scan(destinations ...interface{}) (err error)

Scan reads file record values to assign it to passed in destinations.

type KeyGetter

type KeyGetter interface {
	//Key returns key/id for the the application domain instance.
	Key(instance interface{}) []interface{}
}

KeyGetter represents id/key accessor.

func NewKeyGetterIfNeeded

func NewKeyGetterIfNeeded(keyGetter KeyGetter, table string, targetType reflect.Type) (KeyGetter, error)

NewKeyGetterIfNeeded returns a new key getter if supplied keyGetter was nil for the target type

type KeySetter

type KeySetter interface {
	//SetKey sets autoincrement/sql value to the application domain instance.
	SetKey(instance interface{}, seq int64)
}

KeySetter represents id/key mutator.

type Limiter added in v0.7.0

type Limiter struct {
	// contains filtered or unexported fields
}

Limiter represents resource limter

func NewLimiter added in v0.7.0

func NewLimiter(duration time.Duration, max int) *Limiter

NewLimiter creates a new limiter

func (*Limiter) Acquire added in v0.7.0

func (l *Limiter) Acquire()

Acquire checks if limit for current time window was not exhausted or sleep

type Log

type Log func(format string, args ...interface{})

Log represent log function

var Logf Log = VoidLogger

Logf - function to log debug info

type Manager

type Manager interface {
	Config() *Config

	//ConnectionProvider returns connection provider
	ConnectionProvider() ConnectionProvider

	//Execute executes provided sql, with the arguments, '?' is used as placeholder for and arguments
	Execute(sql string, parameters ...interface{}) (sql.Result, error)

	//ExecuteAll executes all provided sql
	ExecuteAll(sqls []string) ([]sql.Result, error)

	//ExecuteOnConnection executes sql on passed in connection, this allowes to maintain transaction if supported
	ExecuteOnConnection(connection Connection, sql string, parameters []interface{}) (sql.Result, error)

	//ExecuteAllOnConnection executes all sql on passed in connection, this allowes to maintain transaction if supported
	ExecuteAllOnConnection(connection Connection, sqls []string) ([]sql.Result, error)

	//ReadSingle fetches a single record of data, it takes pointer to the result, sql query, binding parameters, record to application instance mapper
	ReadSingle(resultPointer interface{}, query string, parameters []interface{}, mapper RecordMapper) (success bool, err error)

	//ReadSingleOnConnection fetches a single record of data on connection, it takes connection, pointer to the result, sql query, binding parameters, record to application instance mapper
	ReadSingleOnConnection(connection Connection, resultPointer interface{}, query string, parameters []interface{}, mapper RecordMapper) (success bool, err error)

	//ReadAll reads all records, it takes pointer to the result slice , sql query, binding parameters, record to application instance mapper
	ReadAll(resultSlicePointer interface{}, query string, parameters []interface{}, mapper RecordMapper) error

	//ReadAllOnConnection reads all records, it takes connection, pointer to the result slice , sql query, binding parameters, record to application instance mapper
	ReadAllOnConnection(connection Connection, resultSlicePointer interface{}, query string, parameters []interface{}, mapper RecordMapper) error

	//ReadAllWithHandler reads data for passed in query and parameters, for each row reading handler will be called, to continue reading next row it needs to return true
	ReadAllWithHandler(query string, parameters []interface{}, readingHandler func(scanner Scanner) (toContinue bool, err error)) error

	//ReadAllOnWithHandlerOnConnection reads data for passed in query and parameters, on connection,  for each row reading handler will be called, to continue reading next row, it needs to return true
	ReadAllOnWithHandlerOnConnection(connection Connection, query string, parameters []interface{}, readingHandler func(scanner Scanner) (toContinue bool, err error)) error

	//ReadAllOnWithHandlerOnConnection persists all passed in data to the table, it uses dml provider to generate DML for each row.
	PersistAll(slicePointer interface{}, table string, provider DmlProvider) (inserted int, updated int, err error)

	//PersistAllOnConnection persists all passed in data on connection to the table, it uses dml provider to generate DML for each row.
	PersistAllOnConnection(connection Connection, dataPointer interface{}, table string, provider DmlProvider) (inserted int, updated int, err error)

	//PersistSingle persists single row into table, it uses dml provider to generate DML to the row.
	PersistSingle(dataPointer interface{}, table string, provider DmlProvider) (inserted int, updated int, err error)

	//PersistSingleOnConnection persists single row on connection into table, it uses dml provider to generate DML to the row.
	PersistSingleOnConnection(connection Connection, dataPointer interface{}, table string, provider DmlProvider) (inserted int, updated int, err error)

	//connection persists all all row of data to passed in table, it uses key setter to optionally set back autoincrement value, and func to generate parametrized sql for the row.
	PersistData(connection Connection, data interface{}, table string, keySetter KeySetter, sqlProvider func(item interface{}) *ParametrizedSQL) (int, error)

	//DeleteAll deletes all record for passed in slice pointer from table, it uses key provider to take id/key for the record.
	DeleteAll(slicePointer interface{}, table string, keyProvider KeyGetter) (deleted int, err error)

	//DeleteAllOnConnection deletes all record on connection for passed in slice pointer from table, it uses key provider to take id/key for the record.
	DeleteAllOnConnection(connection Connection, resultPointer interface{}, table string, keyProvider KeyGetter) (deleted int, err error)

	//DeleteSingle deletes single row of data from table, it uses key provider to take id/key for the record.
	DeleteSingle(resultPointer interface{}, table string, keyProvider KeyGetter) (success bool, err error)

	//DeleteSingleOnConnection deletes single row of data on connection  from table, it uses key provider to take id/key for the record.
	DeleteSingleOnConnection(connection Connection, resultPointer interface{}, table string, keyProvider KeyGetter) (success bool, err error)

	//ClassifyDataAsInsertableOrUpdatable classifies records are inserable and what are updatable.
	ClassifyDataAsInsertableOrUpdatable(connection Connection, slicePointer interface{}, table string, provider DmlProvider) (insertables, updatables []interface{}, err error)

	//TableDescriptorRegistry returns Table Descriptor Registry
	TableDescriptorRegistry() TableDescriptorRegistry
}

Manager represents datastore manager.

type ManagerFactory

type ManagerFactory interface {
	//Creates manager, takes config pointer.
	Create(config *Config) (Manager, error)

	//Creates manager from url, can url needs to point to Config JSON.
	CreateFromURL(url string) (Manager, error)
}

ManagerFactory represents a manager factory.

func GetManagerFactory

func GetManagerFactory(driver string) (ManagerFactory, error)

GetManagerFactory returns a manager factory for passed in driver, or error.

func NewManagerFactory

func NewManagerFactory() ManagerFactory

NewManagerFactory create a new manager factory.

type ManagerRegistry

type ManagerRegistry interface {
	Get(name string) Manager

	Register(name string, manager Manager)
}

ManagerRegistry represents a manager registry.

func NewManagerRegistry

func NewManagerRegistry() ManagerRegistry

NewManagerRegistry create a new ManagerRegistry

type ParametrizedSQL

type ParametrizedSQL struct {
	SQL    string        //Sql
	Values []interface{} //binding parameter values
	Type   int
}

ParametrizedSQL represents a parmetrized SQL with its binding values, in order of occurrence.

type QueryBuilder

type QueryBuilder struct {
	QueryHint       string
	TableDescriptor *TableDescriptor
}

QueryBuilder represetns a query builder. It builds simple select sql.

func NewQueryBuilder

func NewQueryBuilder(descriptor *TableDescriptor, queryHint string) QueryBuilder

NewQueryBuilder create anew QueryBuilder, it takes table descriptor and optional query hit to include it in the queries

func (*QueryBuilder) BuildBatchedQueryOnPk

func (qb *QueryBuilder) BuildBatchedQueryOnPk(columns []string, pkRowValues [][]interface{}, batchSize int) []*ParametrizedSQL

BuildBatchedQueryOnPk builds batches of ParametrizedSQL for passed in query columns and pk values. Batch size specifies number of rows in one parametrized sql.

func (*QueryBuilder) BuildQueryAll

func (qb *QueryBuilder) BuildQueryAll(columns []string) *ParametrizedSQL

BuildQueryAll builds query all data without where clause

func (*QueryBuilder) BuildQueryOnPk

func (qb *QueryBuilder) BuildQueryOnPk(columns []string, pkRowValues [][]interface{}) *ParametrizedSQL

BuildQueryOnPk builds ParametrizedSQL for passed in query columns and pk values.

type QueryParser

type QueryParser struct {
	// contains filtered or unexported fields
}

QueryParser represents a simple SQL query parser.

func NewQueryParser

func NewQueryParser() *QueryParser

NewQueryParser represents basic SQL query parser.

func (*QueryParser) Parse

func (qp *QueryParser) Parse(query string) (*QueryStatement, error)

Parse parses SQL query to build QueryStatement

type QueryStatement

type QueryStatement struct {
	*BaseStatement
	AllField    bool
	UnionTables []string
	GroupBy     []*SQLColumn
}

QueryStatement represents SQL query statement.

type RecordMapper

type RecordMapper interface {
	//Maps data record by passing to the scanner references to the application abstraction
	Map(scanner Scanner) (interface{}, error)
}

RecordMapper represents a datastore record mapper, it is responsible for mapping data record into application abstraction.

func NewColumnarRecordMapper

func NewColumnarRecordMapper(usePointer bool, targetSlice reflect.Type) RecordMapper

NewColumnarRecordMapper creates a new ColumnarRecordMapper, to map a data record to slice.

func NewMapRecordMapper

func NewMapRecordMapper(usePointer bool, targetSlice reflect.Type) RecordMapper

NewMapRecordMapper creates a new ColumnarRecordMapper, to map a data record to slice.

func NewMetaRecordMapped

func NewMetaRecordMapped(targetType interface{}, usePointer bool) RecordMapper

NewMetaRecordMapped creates a new MetaRecordMapped to map a data record to a struct, it takes target struct and flag if it is a pointer as parameters.

func NewRecordMapper

func NewRecordMapper(targetType reflect.Type) RecordMapper

NewRecordMapper create a new record mapper, if struct is passed it would be MetaRecordMapper, or for slice ColumnRecordMapper

func NewRecordMapperIfNeeded

func NewRecordMapperIfNeeded(mapper RecordMapper, targetType reflect.Type) RecordMapper

NewRecordMapperIfNeeded create a new mapper if passed in mapper is nil. It takes target type for the record mapper.

type SQLColumn

type SQLColumn struct {
	Name              string
	Alias             string
	Expression        string
	Function          string
	FunctionArguments string
}

SQLColumn represents a sql column

type SQLCriteria

type SQLCriteria struct {
	Criteria        []*SQLCriterion
	LogicalOperator string
}

SQLCriteria represents SQL criteria

func (*SQLCriteria) CriteriaValues

func (s *SQLCriteria) CriteriaValues(parameters toolbox.Iterator) ([]interface{}, error)

CriteriaValues returns criteria values extracted from binding parameters, starting from parametersOffset,

func (*SQLCriteria) Expression added in v0.4.0

func (s *SQLCriteria) Expression() string

type SQLCriterion

type SQLCriterion struct {
	Criteria      *SQLCriteria
	LeftOperand   interface{}
	Operator      string
	RightOperand  interface{}
	RightOperands []interface{}
	Inverse       bool // if not operator presents
}

SQLCriterion represents single where clause condiction

func (*SQLCriterion) Expression added in v0.4.0

func (c *SQLCriterion) Expression() string

type SQLScanner

type SQLScanner struct {
	Values map[string]interface{}
	// contains filtered or unexported fields
}

func NewSQLScanner

func NewSQLScanner(query *QueryStatement, config *Config, columns []string) *SQLScanner

NewSQLScanner creates a new sql scanner

func NewSQLScannerWithTypes added in v0.16.0

func NewSQLScannerWithTypes(query *QueryStatement, config *Config, columns []string, types []ColumnType) *SQLScanner

NewSQLScannerWithTypes create a scanner with type

func (*SQLScanner) ColumnTypes added in v0.16.0

func (s *SQLScanner) ColumnTypes() ([]ColumnType, error)

func (*SQLScanner) Columns

func (s *SQLScanner) Columns() ([]string, error)

func (*SQLScanner) Scan

func (s *SQLScanner) Scan(destinations ...interface{}) error

type Scanner

type Scanner interface {
	//Returns all columns specified in select statement.
	Columns() ([]string, error)

	//ColumnTypes return column types
	ColumnTypes() ([]ColumnType, error)

	//Scans datastore record data to convert and assign it to provided destinations, a destination needs to be pointer.
	Scan(dest ...interface{}) error
}

Scanner represents a datastore data scanner. This abstraction provides the ability to convert and assign datastore record of data to provided destination

func NewScanner

func NewScanner(s Scanner) Scanner

type TableColumn

type TableColumn struct {
	ColumnName       string      `column:"column_name"`
	DataType         string      `column:"data_type"`
	DataTypeLength   *int64      `column:"data_type_length"`
	NumericPrecision *int64      `column:"numeric_precision"`
	NumericScale     *int64      `column:"numeric_scale"`
	IsNullable       interface{} `column:"is_nullable"`
	Position         int         `column:"position"`
	// contains filtered or unexported fields
}

func (*TableColumn) DatabaseTypeName

func (c *TableColumn) DatabaseTypeName() string

Common type include "VARCHAR", "TEXT", "NVARCHAR", "DECIMAL", "BOOL", "INT", "BIGINT".

func (*TableColumn) DecimalSize

func (c *TableColumn) DecimalSize() (precision, scale int64, ok bool)

func (*TableColumn) Length

func (c *TableColumn) Length() (length int64, ok bool)

func (*TableColumn) Name

func (c *TableColumn) Name() string

func (*TableColumn) Nullable

func (c *TableColumn) Nullable() (nullable, ok bool)

func (*TableColumn) ScanType

func (c *TableColumn) ScanType() reflect.Type

type TableDescriptor

type TableDescriptor struct {
	Table          string
	Autoincrement  bool
	PkColumns      []string
	Columns        []string
	ColumnTypes    map[string]string
	Nullables      map[string]bool
	OrderColumns   []string
	Schema         []map[string]interface{} //Schema to be interpreted by NoSQL drivers for create table operation .
	SchemaURL      string                   //url with JSON to the TableDescriptor.Schema.
	FromQuery      string                   //If table is query base then specify FromQuery
	FromQueryAlias string
}

TableDescriptor represents a table details.

func NewTableDescriptor

func NewTableDescriptor(table string, instance interface{}) (*TableDescriptor, error)

NewTableDescriptor creates a new table descriptor for passed in instance, it can use the following tags:"column", "dateLayout","dateFormat", "autoincrement", "primaryKey", "sequence", "transient"

func (*TableDescriptor) From added in v0.10.2

func (t *TableDescriptor) From() string

func (*TableDescriptor) HasSchema

func (d *TableDescriptor) HasSchema() bool

HasSchema check if table desciptor has defined schema.

type TableDescriptorRegistry

type TableDescriptorRegistry interface {
	//Has checks if descriptor is defined for the table.
	Has(table string) bool

	//Get returns a table descriptor for passed in table, it calls panic if descriptor is not found, to avoid it please always use Has check.
	Get(table string) *TableDescriptor

	//Register registers a table descriptor.
	Register(descriptor *TableDescriptor) error

	//Tables returns all registered tables.
	Tables() []string
}

TableDescriptorRegistry represents a registry to store table descriptors by table name.

func NewTableDescriptorRegistry

func NewTableDescriptorRegistry() TableDescriptorRegistry

newTableDescriptorRegistry returns a new newTableDescriptorRegistry

type TransactionManager

type TransactionManager interface {
	Begin() error

	Commit() error

	Rollback() error
}

TransactionManager represents a transaction manager.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL