relationaldb

README

relationaldb

Connect a Go application to an Amazon Aurora DSQL database.

go get github.com/templaterojbar/relationaldb

It carries the two things every client of such a database has to get right, and which would otherwise be rewritten per service:

• Authentication with short-lived IAM tokens. DSQL has no passwords. A token is signed from the caller's credentials and expires, so a long-lived pool has to sign a new one for every connection it opens.
• Retrying on conflict. Concurrency is optimistic: two transactions writing the same row both proceed, and the loser fails at commit with SQLSTATE 40001 instead of waiting on a lock. The whole transaction has to run again.

Connecting

pool, err := relationaldb.Open(ctx, relationaldb.Config{
	Endpoint: os.Getenv("DATABASE_ENDPOINT"),
	Role:     os.Getenv("DATABASE_ROLE"),
	Region:   os.Getenv("AWS_REGION"),
})
if err != nil {
	return err
}
defer pool.Close()

Open returns a *pgxpool.Pool and does not connect yet; the first query does. Every new connection gets a freshly signed token, and connections retire after 45 minutes, before DSQL closes them at one hour.

In a Lambda, open the pool once at start and reuse it across invocations.

Credentials defaults to the caller's own, which for a Lambda are its execution role's. The role it connects as, and what that role may read or write, are granted in the database beforehand.

Writing

err := relationaldb.InTx(ctx, pool, func(tx pgx.Tx) error {
	_, err := tx.Exec(ctx, "UPDATE accounts SET balance = balance - $1 WHERE id = $2", amount, id)
	return err
})

InTx commits, or rolls back and returns the error. On a conflict it runs the whole function again, up to five times, backing off with jitter so the same two writers do not collide again at the same moment. Retry{}.InTx takes a different policy.

Two rules follow from the retry:

• The function must be safe to run more than once. Anything outside the database — publishing a message, calling an API — goes after InTx returns, not inside it.
• Every attempt starts from scratch. Read inside the transaction rather than passing in values read before it.

When every attempt loses, the error wraps ErrRetriesExhausted and the last conflict.

License

MIT

Documentation

Overview

Package relationaldb connects an application to an Amazon Aurora DSQL database.

It carries the two decisions every client of such a database has to get right, and which would otherwise be rewritten per service: authenticating with short-lived IAM tokens instead of a password, and retrying transactions that lose an optimistic-concurrency conflict.

The database, the role and the grants are created by the platform's relational-database recipe and its migrator; this package only connects as a role it already prepared.

Index

• Variables
• func InTx(ctx context.Context, db Beginner, fn func(pgx.Tx) error) error
• func Open(ctx context.Context, config Config) (*pgxpool.Pool, error)
• type Beginner
• type Config
• type Retry
  • func (r Retry) InTx(ctx context.Context, db Beginner, fn func(pgx.Tx) error) error

Variables

var (
	ErrMissingEndpoint = errors.New("endpoint is required")
	ErrMissingRole     = errors.New("role is required")
	ErrMissingRegion   = errors.New("region is required")
)

var DefaultRetry = Retry{Attempts: 5, BaseDelay: 20 * time.Millisecond}

DefaultRetry is what InTx uses. Conflicts resolve in milliseconds, so the delays are short; what matters is that they grow and carry jitter, or the same two writers keep colliding.

var ErrRetriesExhausted = errors.New("transaction kept conflicting")

ErrRetriesExhausted wraps the last conflict when every attempt lost.

Functions

func InTx

func InTx(ctx context.Context, db Beginner, fn func(pgx.Tx) error) error

InTx runs fn in a transaction, retrying the whole transaction while DSQL reports a conflict. fn must be safe to run more than once: keep effects outside the database, such as publishing a message, after InTx returns.

func Open

func Open(ctx context.Context, config Config) (*pgxpool.Pool, error)

Open returns a pool that authenticates every new connection with a freshly signed token, because tokens expire while a long-lived pool does not. It does not connect yet; the first query does.

Types

type Beginner

type Beginner interface {
	Begin(ctx context.Context) (pgx.Tx, error)
}

Beginner starts transactions. *pgxpool.Pool satisfies it.

type Config

type Config struct {
	// Endpoint is the cluster hostname, as published by the recipe's endpoint output.
	Endpoint string
	// Role is the database role to connect as: client_<key> for the key the stack
	// declared for this function.
	Role string
	// Region of the cluster, used to sign the authentication token.
	Region string
	// Credentials signs the tokens. Leave nil to use the function's own credentials.
	Credentials aws.CredentialsProvider
	// MaxConnections caps the pool. Zero uses the default.
	MaxConnections int32
}

Config describes the database to connect to. Endpoint, Role and Region come from the function's environment; the recipe sets the first and Lambda sets the region.

type Retry

type Retry struct {
	Attempts  int
	BaseDelay time.Duration
}

Retry is how many times a conflicting transaction runs again, and how long it waits.

func (Retry) InTx

func (r Retry) InTx(ctx context.Context, db Beginner, fn func(pgx.Tx) error) error

InTx runs fn with this retry policy instead of the default one.