migrationutils

Documentation

Overview

Package migrationutils implements a general purpose "batcher" for executing multiple SQL statements with a single Postgres call. The intention is that this can be used by any migration code that needs to perform a large number of SQL updates and wishes to do so in a more performant way.

Current limitations:

• Only UPDATE statements are supported.
• Only one column is supported in the WHERE clause.

It is likely that we'll want to enhance this in the future to support INSERT and/or DELETE statements, and to allow for multiple columns in the WHERE clause. The functions have been written with this in mind to make them easy to expand upon.

Index

• func NewPostgresBatcher(tx *pachsql.Tx, action string, table string, columns []string, ...) (*postgresBatcher, error)
• func QuoteBytes(buf []byte) string
• func QuoteString(str string) string
• func SanitizeSQL(sql string, args ...any) (string, error)
• type Part
• type Query
  • func NewQuery(sql string) (*Query, error)
  • func (q *Query) Sanitize(args ...any) (string, error)
• type SimplePostgresBatcher
  • func NewSimplePostgresBatcher(tx *pachsql.Tx) *SimplePostgresBatcher
  • func (pb *SimplePostgresBatcher) Add(ctx context.Context, stmt string, args ...interface{}) error
  • func (pb *SimplePostgresBatcher) Flush(ctx context.Context) error

Functions

func NewPostgresBatcher

func NewPostgresBatcher(tx *pachsql.Tx, action string, table string, columns []string, wColumns []string, batchSize int) (*postgresBatcher, error)

NewPostgresBatcher creates a new batcher.

func QuoteBytes

func QuoteBytes(buf []byte) string

func QuoteString

func QuoteString(str string) string

func SanitizeSQL

func SanitizeSQL(sql string, args ...any) (string, error)

SanitizeSQL replaces placeholder values with args. It quotes and escapes args as necessary. This function is only safe when standard_conforming_strings is on.

Types

type Part

type Part any

Part is either a string or an int. A string is raw SQL. An int is a argument placeholder.

type Query

type Query struct {
	Parts []Part
}

func NewQuery

func NewQuery(sql string) (*Query, error)

func (*Query) Sanitize

func (q *Query) Sanitize(args ...any) (string, error)

type SimplePostgresBatcher

type SimplePostgresBatcher struct {
	// contains filtered or unexported fields
}

SimplePostgresBatcher is an alternative batcher that buffers statements 'stmts' terminated by semicolons until it hits the max number of statements. The statements are then executed in one round-trip query to postgres.

func NewSimplePostgresBatcher

func NewSimplePostgresBatcher(tx *pachsql.Tx) *SimplePostgresBatcher

func (*SimplePostgresBatcher) Add

func (pb *SimplePostgresBatcher) Add(ctx context.Context, stmt string, args ...interface{}) error

Add inserts a statement to the SimplePostgresBatcher's internal buffer. If the number of statements exceeds the maximum after adding, the batcher executes the statements. SimplePostgresBatcher doesn't support buffering arguments, the arguments are sanitized then their values are inlined.

Add does not support SqlNull<X> structs like SqlNullString.

func (*SimplePostgresBatcher) Flush

func (pb *SimplePostgresBatcher) Flush(ctx context.Context) error

Flush triggers the SimplePostgresBatcher to execute any remaining buffered statements.