The highest tagged major version is
README
¶
DynamoDB Lock Client for Go v4
This repository is covered by this SLA.
The dymanoDB Lock Client for Go is a general purpose distributed locking library built for DynamoDB. The dynamoDB Lock Client for Go supports both fine-grained and coarse-grained locking as the lock keys can be any arbitrary string, up to a certain length. Please create issues in the GitHub repository with questions, pull request are very much welcome.
It inspired by the Amazon's original dynamodb-lock-client using the AWS's latest Go SDK.
It is simpler than the original and its predecessor in the same that it offers more basic primitives and it is up to the caller to weave everything together.
Main differences:
- There is no SessionMonitor anymore. It was a leaky abstraction that was always problematic.
- Heartbeats are manual, and it is up to the caller to implement their own heartbeat policies. A helper method was added to help transition from dynamolock/v2.
Use cases
A common use case for this lock client is: let's say you have a distributed system that needs to periodically do work on a given campaign (or a given customer, or any other object) and you want to make sure that two boxes don't work on the same campaign/customer at the same time. An easy way to fix this is to write a system that takes a lock on a customer, but fine-grained locking is a tough problem. This library attempts to simplify this locking problem on top of DynamoDB.
Another use case is leader election. If you only want one host to be the leader, then this lock client is a great way to pick one. When the leader fails, it will fail over to another host within a customizable leaseDuration that you set.
Getting Started
To use the DynamoDB Lock Client for Go, you must make it sure it is present in
$GOPATH or in your vendor directory.
$ go get -u cirello.io/dynamolock/v4
This package has the go.mod file to be used with Go's module system.
Then, you need to set up a DynamoDB table that has a hash key on a key with the
name key. For your convenience, there is a function in the package called
CreateTable that you can use to set up your table, but it is also possible to
set up the table in the AWS Console. The table should be created in advance,
since it takes a couple minutes for DynamoDB to provision your table for you.
The package level documentation comment has an example of how to use this
package.
First you have to create the table and wait for DynamoDB to complete:
package main
import (
"log"
"cirello.io/dynamolock/v4"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
func main() {
cfg, err := config.LoadDefaultConfig(context.Background(), config.WithRegion("us-west-2"))
if err != nil {
log.Fatal(err)
}
c, err := dynamolock.New(dynamodb.NewFromConfig(cfg),
"locks",
dynamolock.WithLeaseDuration(3*time.Second),
)
if err != nil {
log.Fatal(err)
}
defer c.Close()
log.Println("ensuring table exists")
_, err = c.CreateTable("locks",
dynamolock.WithProvisionedThroughput(&types.ProvisionedThroughput{
ReadCapacityUnits: aws.Int64(5),
WriteCapacityUnits: aws.Int64(5),
}),
dynamolock.WithCustomPartitionKeyName("key"),
)
if err != nil {
log.Fatal(err)
}
}
Once you see the table is created in the DynamoDB console, you should be ready to run:
package main
import (
"log"
"cirello.io/dynamolock/v4"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
)
func main() {
cfg, err := config.LoadDefaultConfig(context.Background(), config.WithRegion("us-west-2"))
if err != nil {
log.Fatal(err)
}
c, err := dynamolock.New(dynamodb.NewFromConfig(cfg),
"locks",
dynamolock.WithLeaseDuration(3*time.Second),
)
if err != nil {
log.Fatal(err)
}
defer c.Close()
data := []byte("some content a")
lockedItem, err := c.AcquireLock("spock",
dynamolock.WithData(data),
dynamolock.ReplaceData(),
)
if err != nil {
log.Fatal(err)
}
// here you can periodically call if you need a long lived lock:
// err := c.SendHeartbeat(ctx, lockedItem)
log.Println("lock content:", string(lockedItem.Data()))
if got := string(lockedItem.Data()); string(data) != got {
log.Println("losing information inside lock storage, wanted:", string(data), " got:", got)
}
log.Println("cleaning lock")
success, err := c.ReleaseLock(lockedItem)
if !success {
log.Fatal("lost lock before release")
}
if err != nil {
log.Fatal("error releasing lock:", err)
}
log.Println("done")
}
Selected Features
Read the data in a lock without acquiring it
You can read the data in the lock without acquiring it, and find out who owns the lock. Here's how:
lock, err := lockClient.Get("kirk");
Logic to avoid problems with clock skew
The lock client never stores absolute times in DynamoDB -- only the relative "lease duration" time is stored in DynamoDB. The way locks are expired is that a call to acquireLock reads in the current lock, checks the RecordVersionNumber of the lock (which is a GUID) and starts a timer. If the lock still has the same GUID after the lease duration time has passed, the client will determine that the lock is stale and expire it.
What this means is that, even if two different machines disagree about what time it is, they will still avoid clobbering each other's locks.
Required DynamoDB Actions
For an IAM role to take full advantage of dynamolock/**v2**, it must be allowed to
perform all of the following actions on the DynamoDB table containing the locks:
GetItemPutItemUpdateItemDeleteItemCreateTable
What did happen to Dynamolock v3?
At some point, few people from the community got involved and decided to start debating changes on the library. At the time, they seemed backward-compatibility breaking changes, so I opened the v3. Once it became clear these would-be contributors would not be able to work on this library anymore, I closed the v3 branch and never released it. So to avoid recycling a major number, I bumped this release to v4.
Documentation
¶
Overview ¶
Package dynamolock provides a simple utility for using DynamoDB's consistent read/write feature to use it for managing distributed locks.
In order to use this package, the client must create a table in DynamoDB, although the client provides a convenience method for creating that table (CreateTable).
Basic usage:
import (
"log"
"cirello.io/dynamolock/v4"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
//---
cfg, err := config.LoadDefaultConfig(context.Background(), config.WithRegion("us-west-2"))
if err != nil {
log.Fatal(err)
}
c, err := dynamolock.New(dynamodb.NewFromConfig(cfg),
"locks",
dynamolock.WithLeaseDuration(3*time.Second),
)
if err != nil {
log.Fatal(err)
}
defer c.Close()
log.Println("ensuring table exists")
c.CreateTable("locks",
dynamolock.WithProvisionedThroughput(&types.ProvisionedThroughput{
ReadCapacityUnits: aws.Int64(5),
WriteCapacityUnits: aws.Int64(5),
}),
dynamolock.WithCustomPartitionKeyName("key"),
)
//-- at this point you must wait for DynamoDB to complete the creation.
data := []byte("some content a")
lockedItem, err := c.AcquireLock("spock",
dynamolock.WithData(data),
dynamolock.ReplaceData(),
)
if err != nil {
log.Fatal(err)
}
// here you can periodically call if you need a long lived lock:
// err := c.SendHeartbeat(ctx, lockedItem)
log.Println("lock content:", string(lockedItem.Data()))
if got := string(lockedItem.Data()); string(data) != got {
log.Println("losing information inside lock storage, wanted:", string(data), " got:", got)
}
log.Println("cleaning lock")
success, err := c.ReleaseLock(lockedItem)
if !success {
log.Fatal("lost lock before release")
}
if err != nil {
log.Fatal("error releasing lock:", err)
}
log.Println("done")
This package is covered by this SLA: https://github.com/cirello-io/public/blob/master/SLA.md
Index ¶
- Variables
- func Heartbeat(ctx context.Context, c *Client, lockItem *Lock, opts ...SendHeartbeatOption) error
- type AcquireLockOption
- func FailIfLocked() AcquireLockOption
- func ReplaceData() AcquireLockOption
- func WithAdditionalAttributes(attr map[string]types.AttributeValue) AcquireLockOption
- func WithAdditionalTimeToWaitForLock(d time.Duration) AcquireLockOption
- func WithData(b []byte) AcquireLockOption
- func WithDeleteLockOnRelease() AcquireLockOption
- func WithRefreshPeriod(d time.Duration) AcquireLockOption
- type Client
- func (c *Client) AcquireLock(ctx context.Context, key string, opts ...AcquireLockOption) (*Lock, error)
- func (c *Client) Close(ctx context.Context) error
- func (c *Client) CreateTable(ctx context.Context, tableName string, opts ...CreateTableOption) (*dynamodb.CreateTableOutput, error)
- func (c *Client) Get(ctx context.Context, key string) (*Lock, error)
- func (c *Client) ReleaseLock(ctx context.Context, lockItem *Lock, opts ...ReleaseLockOption) error
- func (c *Client) SendHeartbeat(ctx context.Context, lockItem *Lock, opts ...SendHeartbeatOption) error
- type ClientOption
- type ContextLogger
- type CreateTableOption
- type DynamoDBClient
- type Lock
- type LockNotGrantedError
- type Logger
- type ReleaseLockOption
- type SendHeartbeatOption
- type TimeoutError
Constants ¶
This section is empty.
Variables ¶
var ( ErrCannotReleaseNullLock = errors.New("cannot release null lock item") ErrOwnerMismatched = errors.New("lock owner mismatched") )
ErrClientClosed reports the client cannot be used because it is already closed.
ErrReadOnlyLockHeartbeat indicates that the given *Lock is not really a lock, but a read-only copy from a Get call.
Functions ¶
Types ¶
type AcquireLockOption ¶
type AcquireLockOption func(*acquireLockOptions)
AcquireLockOption allows to change how the lock is actually held by the client.
func FailIfLocked ¶
func FailIfLocked() AcquireLockOption
FailIfLocked will not retry to acquire the lock, instead returning.
func ReplaceData ¶
func ReplaceData() AcquireLockOption
ReplaceData will force the new content to be stored in the key.
func WithAdditionalAttributes ¶
func WithAdditionalAttributes(attr map[string]types.AttributeValue) AcquireLockOption
WithAdditionalAttributes stores some additional attributes with each lock. This can be used to add any arbitrary parameters to each lock row.
func WithAdditionalTimeToWaitForLock ¶
func WithAdditionalTimeToWaitForLock(d time.Duration) AcquireLockOption
WithAdditionalTimeToWaitForLock defines how long to wait in addition to the lease duration (if set to 10 minutes, this will try to acquire a lock for at least 10 minutes before giving up and returning an error).
func WithData ¶
func WithData(b []byte) AcquireLockOption
WithData stores the content into the lock itself.
func WithDeleteLockOnRelease ¶
func WithDeleteLockOnRelease() AcquireLockOption
WithDeleteLockOnRelease defines whether or not the lock should be deleted when Close() is called on the resulting LockItem will force the new content to be stored in the key.
func WithRefreshPeriod ¶
func WithRefreshPeriod(d time.Duration) AcquireLockOption
WithRefreshPeriod defines how long to wait before trying to get the lock again (if set to 10 seconds, for example, it would attempt to do so every 10 seconds).
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is a dynamoDB based distributed lock client.
func New ¶
func New(dynamoDB DynamoDBClient, tableName string, opts ...ClientOption) *Client
New creates a new dynamoDB based distributed lock client.
func (*Client) AcquireLock ¶
func (c *Client) AcquireLock(ctx context.Context, key string, opts ...AcquireLockOption) (*Lock, error)
AcquireLock holds the defined lock. The given context is passed down to the underlying dynamoDB call.
func (*Client) Close ¶
Close releases all of the locks. The given context is passed down to the underlying dynamoDB calls.
func (*Client) CreateTable ¶
func (c *Client) CreateTable(ctx context.Context, tableName string, opts ...CreateTableOption) (*dynamodb.CreateTableOutput, error)
CreateTable prepares a DynamoDB table with the right schema for it to be used by this locking library. The table should be set up in advance, because it takes a few minutes for DynamoDB to provision a new instance. Also, if the table already exists, it will return an error. The given context is passed down to the underlying dynamoDB call.
func (*Client) Get ¶
Get loads the given lock, but does not acquire the lock. It returns the metadata currently associated with the given lock. If the client pointer is the one who acquired the lock, it will return the lock, and operations such as releaseLock will work. However, if the client is not the one who acquired the lock, then operations like releaseLock will not work (after calling Get, the caller should check lockItem.isExpired() to figure out if it currently has the lock.) If the context is canceled, it is going to return the context error on local cache hit. The given context is passed down to the underlying dynamoDB call.
func (*Client) ReleaseLock ¶
ReleaseLock releases the given lock if the current user still has it, returning nil if the lock was successfully released, and false if someone else already stole the lock or a problem happened. Deletes the lock item if it is released and deleteLockItemOnClose is set.
type ClientOption ¶
type ClientOption func(*Client)
ClientOption reconfigure the lock client creation.
func WithContextLogger ¶
func WithContextLogger(l ContextLogger) ClientOption
WithContextLogger injects a logger into the client, so its internals can be recorded.
func WithLeaseDuration ¶
func WithLeaseDuration(d time.Duration) ClientOption
WithLeaseDuration defines how long should the lease be held.
func WithLogger ¶
func WithLogger(l Logger) ClientOption
WithLogger injects a logger into the client, so its internals can be recorded.
func WithOwnerName ¶
func WithOwnerName(s string) ClientOption
WithOwnerName changes the owner linked to the client, and by consequence to locks.
func WithPartitionKeyName ¶
func WithPartitionKeyName(s string) ClientOption
WithPartitionKeyName defines the key name used for asserting keys uniqueness.
func WithSortKey ¶
func WithSortKey(name string, value string) ClientOption
WithSortKey defines the sort key name and value to use for asserting keys uniqueness. If not set, the sort key will not be used in DynamoDB calls.
type ContextLogger ¶
ContextLogger defines a logger interface that can be used to pass extra information to the implementation. For example, if you use zap, you may have extra fields you want to add to the log line. You can add those extra fields to the parent context of calls like AcquireLock, and then retrieve them in your implementation of ContextLogger.
type CreateTableOption ¶
type CreateTableOption func(*createDynamoDBTableOptions)
CreateTableOption is an options type for the CreateTable method in the lock client. This allows the user to create a DynamoDB table that is lock client-compatible and specify optional parameters such as the desired throughput and whether or not to use a sort key.
func WithCustomPartitionKeyName ¶
func WithCustomPartitionKeyName(s string) CreateTableOption
WithCustomPartitionKeyName changes the partition key name of the table. If not specified, the default "key" will be used.
func WithProvisionedThroughput ¶
func WithProvisionedThroughput(provisionedThroughput *types.ProvisionedThroughput) CreateTableOption
WithProvisionedThroughput changes the billing mode of DynamoDB and tells DynamoDB to operate in a provisioned throughput mode instead of pay-per-request.
func WithSortKeyName ¶
func WithSortKeyName(s string) CreateTableOption
WithSortKeyName creates the table with a sort key. If not specified, the table will not have a sort key.
func WithTags ¶
func WithTags(tags []types.Tag) CreateTableOption
WithTags changes the tags of the table. If not specified, the table will have empty tags.
type DynamoDBClient ¶
type DynamoDBClient interface {
GetItem(ctx context.Context, params *dynamodb.GetItemInput, optFns ...func(*dynamodb.Options)) (*dynamodb.GetItemOutput, error)
PutItem(ctx context.Context, params *dynamodb.PutItemInput, optFns ...func(*dynamodb.Options)) (*dynamodb.PutItemOutput, error)
UpdateItem(ctx context.Context, params *dynamodb.UpdateItemInput, optFns ...func(*dynamodb.Options)) (*dynamodb.UpdateItemOutput, error)
DeleteItem(ctx context.Context, params *dynamodb.DeleteItemInput, optFns ...func(*dynamodb.Options)) (*dynamodb.DeleteItemOutput, error)
CreateTable(ctx context.Context, params *dynamodb.CreateTableInput, optFns ...func(*dynamodb.Options)) (*dynamodb.CreateTableOutput, error)
}
DynamoDBClient defines the public interface that must be fulfilled for testing doubles.
type Lock ¶
type Lock struct {
// contains filtered or unexported fields
}
Lock item properly speaking.
func (*Lock) AdditionalAttributes ¶
func (l *Lock) AdditionalAttributes() map[string]types.AttributeValue
AdditionalAttributes returns the lock's additional data stored during acquisition.
func (*Lock) Data ¶
Data returns the content of the lock, if any is available.
func (*Lock) IsExpired ¶
IsExpired returns if the lock is expired, released, or neither.
type LockNotGrantedError ¶
type LockNotGrantedError struct {
// contains filtered or unexported fields
}
LockNotGrantedError indicates that an AcquireLock call has failed to establish a lock because of its current lifecycle state.
func (*LockNotGrantedError) Error ¶
func (e *LockNotGrantedError) Error() string
func (*LockNotGrantedError) Unwrap ¶
func (e *LockNotGrantedError) Unwrap() error
Unwrap reveals the underlying cause why the lock was not granted.
type Logger ¶
type Logger interface {
Println(v ...any)
}
Logger defines the minimum desired logger interface for the lock client.
type ReleaseLockOption ¶
type ReleaseLockOption func(*releaseLockOptions)
ReleaseLockOption provides options for releasing a lock when calling the releaseLock() method. This class contains the options that may be configured during the act of releasing a lock.
func WithDataAfterRelease ¶
func WithDataAfterRelease(data []byte) ReleaseLockOption
WithDataAfterRelease is the new data to persist to the lock (only used if deleteLock=false.) If the data is null, then the lock client will keep the data as-is and not change it.
func WithDeleteLock ¶
func WithDeleteLock(deleteLock bool) ReleaseLockOption
WithDeleteLock defines whether or not to delete the lock when releasing it. If set to false, the lock row will continue to be in DynamoDB, but it will be marked as released.
type SendHeartbeatOption ¶
type SendHeartbeatOption func(*sendHeartbeatOptions)
SendHeartbeatOption allows to proceed with Lock content changes in the heartbeat cycle.
func DeleteData ¶
func DeleteData() SendHeartbeatOption
DeleteData removes the Lock data on heartbeat.
func MatchOwnerOnly ¶
func MatchOwnerOnly() SendHeartbeatOption
MatchOwnerOnly helps dealing with network transient errors by ignoring internal record version number and matching only against the owner and the partition key name. If lock owner is globally unique, then this feature is safe to use.
func ReplaceHeartbeatData ¶
func ReplaceHeartbeatData(data []byte) SendHeartbeatOption
ReplaceHeartbeatData overrides the content of the Lock in the heartbeat cycle.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package internal provides a boundary to prevent external packages from using dynamolock's internal interfaces that are subject to change.
|
Package internal provides a boundary to prevent external packages from using dynamolock's internal interfaces that are subject to change. |