dynamocity

README

dynamocity

import "github.com/edwardsmatt/dynamocity"

Package dynamocity provides helpful types for using dynamodb; however, the OverrideEndpointResolver theoretically supports creating a client for any AWS service client using AWS Go SDK V2

The core reason dynamocity exits is to provide a convenient implementation of dynamodbattribute.Marshaler and dynamodbattribute.Unmarshaller which enforces fixed timestamp precision when marshalling for DynamoDB, making it safe for using time.Time as a DynamoDB range key in a string type.

Background

From the standard go library time.RFC3339Nano documentation

The RFC3339Nano format removes trailing zeros from the seconds field and thus may not sort correctly once formatted.

Given the existing AWS Go SDK V2 uses time.RFC3339Nano, it is not suitable to use time.Time as a Dynamo DB Sort Key in a string attribute type.

Designing an efficient Dynamo Single Table design which leverages a generic composite key structure can often use string attribute types; and for correct sortability in this case truncating trailing zeros would be detrimental.

As an aside please I highly recommend the Dynamo DB Book from Alex DeBrie if you're interested in some top shelf DynamoDB learning resources

Why dynamocity?

Well to be honest, I was working with a chap, let's call him JimmyD. We were working through designing a dynamo table schema for a few different components. Whilst wrapping his head around the schema and patterns, JimmyD refered to the Dynamo schema as a monstrosity and then the Dynamo Monstricity, and afterwards - it became affectionately known as Dynamocity.

And that's it. Thanks JimmyD

Index

• NanoTime
• MillisTime
• SecondsTime
• OverrideEndpointResolver

Types

All of the below Time types implement fixed precision when marshalled to strings, and are therefore sortable as an index type in Dynamo, or anywhere that a string representation needs to be string sortable. Also, all of the following types implement:

• dynamodbattribute.Marshaler
• dynamodbattribute.Unmarshaller
• fmt.Stringer
• json.Unmarshaler
• json.Marshaler

Implementing these types make them safe for JSON, string, or dynamo (un)marshalling.

NanoTime

NanoTime represents a sortable strict RFC3339 Timestamp with fixed nanosecond precision. Example Usage:

dynamocity.NanoTime(time.Date(2020, time.April, 1, 14, 0, 0, 999000000, time.UTC)),

MillisTime

MillisTime represents a sortable strict RFC3339 Timestamp with fixed millisecond precision. Example Usage:

dynamocity.MillisTime(time.Date(2020, time.April, 1, 14, 0, 0, 999000000, time.UTC)),

SecondsTime

SecondsTime represents a sortable strict RFC3339 Timestamp with fixed second precision. Example Usage:

dynamocity.SecondsTime(time.Date(2020, time.April, 1, 14, 0, 0, 999000000, time.UTC)),

OverrideEndpointResolver

The OverrideEndpointResolver can be used to provide a simple Client factory function. For example, creating a *dynamodb.Client with overrides could be as follows:

// Dynamo is a utility function to return a *dynamodb.Client
func Dynamo(overrides map[string]string) (*dynamodb.Client, error) {
    awsConfig, err := external.LoadDefaultAWSConfig()
    if err != nil {
        return nil, err
    }

    awsConfig.EndpointResolver = dynamocity.MakeEndpointResolver(overrides)

    client := dynamodb.New(awsConfig)

    return client, nil
}

However, as previously mentioned this pattern could theoretically be used for any AWS Service AWS Go SDK V2 - For example:

// Lambda is a utility function to return a *lambda.Client
func Lambda(overrides map[string]string) (*lambda.Client, error) {
    awsConfig, err := external.LoadDefaultAWSConfig()
    if err != nil {
        return nil, err
    }

    awsConfig.EndpointResolver = dynamocity.MakeEndpointResolver(overrides)

    client := lambda.New(awsConfig)

    return client, nil
}

Prerequisites

• docker-compose
• go 1.12 (in alignment with the AWS Go SDK V2)

Getting Started

Execute the following to provide an explanation of tasks that are commonly used for development.

make help

The output explains the common make targets and what they do:

Perform common development tasks
Usage: make [TARGET]
Targets:
    clean     Clean removes the vendor directory, go.mod, and go.sum files
    prepare   Sets up a go.mod, go.sum and downloads all vendor dependencies
    test      Starts a dynamo local dynamo container and runs unit and integration tests

Documentation

Overview

Package dynamocity provides helpful types for working with DynamoDB

OverrideEndpointResolver is a utility resolver for replacing the endpoint for a dynamo service to use http://localhost:8000 for testing with a dynamodb-local.

Index

• Constants
• func BetweenEndInc(t, startExclusive, endInclusive time.Time) bool
• func BetweenExclusive(t, startExclusive, endExclusive time.Time) bool
• func BetweenInclusive(t, start, end time.Time) bool
• func BetweenStartInc(t, startInclusive, endExclusive time.Time) bool
• func MakeEndpointResolver(services map[string]string) aws.EndpointResolver
• type Date
  • func ParseDate(str string) (Date, error)
  • func (t Date) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)
  • func (t Date) MarshalJSON() ([]byte, error)
  • func (t Date) String() string
  • func (t Date) Time() time.Time
  • func (t *Date) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error
  • func (t *Date) UnmarshalJSON(b []byte) error
• type MillisTime
  • func (t MillisTime) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)
  • func (t MillisTime) MarshalJSON() ([]byte, error)
  • func (t MillisTime) String() string
  • func (t MillisTime) Time() time.Time
  • func (t *MillisTime) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error
  • func (t *MillisTime) UnmarshalJSON(b []byte) error
• type NanoTime
  • func (t NanoTime) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)
  • func (t NanoTime) MarshalJSON() ([]byte, error)
  • func (t NanoTime) String() string
  • func (t NanoTime) Time() time.Time
  • func (t *NanoTime) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error
  • func (t *NanoTime) UnmarshalJSON(b []byte) error
• type OverrideEndpointResolver
  • func (o *OverrideEndpointResolver) ResolveEndpoint(service, region string) (aws.Endpoint, error)
• type SecondsTime
  • func (t SecondsTime) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)
  • func (t SecondsTime) MarshalJSON() ([]byte, error)
  • func (t SecondsTime) String() string
  • func (t SecondsTime) Time() time.Time
  • func (t *SecondsTime) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error
  • func (t *SecondsTime) UnmarshalJSON(b []byte) error

Constants

const FlexibleNanoFmt = time.RFC3339Nano

FlexibleNanoFmt is the standard library time.RFC3339Nano, which applies a flexible compatible nanosecond precision marshalling and unmarshalling capability. However, when marshalling the standard library time.RFC3339Nano format removes trailing zeros from the seconds field, and thus may not sort correctly once formatted.

Therefore, this format is unsafe for marshalling to dynamo or JSON if the resultant value is expected to be sortable by string.

const StrictDateFmt = "2006-01-02"

StrictDateFmt is a representation of the Date portion of a time.RFC3339 format.

Unmarshalling using StrictDateFmt will result in errors if the string does not strictly match the time.RFC3339 format. For example: `2006-01-02`

const StrictMillisFmt = "2006-01-02T15:04:05.000Z07:00"

StrictMillisFmt applies a strict millisecond precision marshalling of a dynamocity.NanoTime. This ensures that trailing zeros are never stripped. The standard library time.RFC3339Nano format removes trailing zeros from the seconds field, and thus may not sort correctly once formatted.

Unmarshalling using StrictMillisFmt will result in errors if the string does not strictly match the RFC3339 format with millisecond precision. For example: `2006-01-02T15:04:05.000Z07:00`

const StrictNanoFmt = "2006-01-02T15:04:05.000000000Z07:00"

StrictNanoFmt applies a strict nanosecond precision marshalling of a dynamocity.NanoTime. This ensures that trailing zeros are never stripped. The standard library time.RFC3339Nano format removes trailing zeros from the seconds field, and thus may not sort correctly once formatted.

Unmarshalling using StrictNanoFmt will result in errors if the string does not strictly match the RFC3339 format with fixed nanosecond precision. For example: `2006-01-02T15:04:05.000000000Z07:00`

const StrictSecondsFmt = time.RFC3339

StrictSecondsFmt is the standard library time.RFC3339, which applies a strict second precision marshalling and unmarshalling capability.

Unmarshalling using StrictSecondsFmt will result in errors if the string does not strictly match the time.RFC3339 format. For example: `2006-01-02T15:04:05Z07:00`

Functions

func BetweenEndInc

func BetweenEndInc(t, startExclusive, endInclusive time.Time) bool

BetweenEndInc will return true if this time.Time is after the start and before or equal to the end

func BetweenExclusive

func BetweenExclusive(t, startExclusive, endExclusive time.Time) bool

BetweenExclusive will return true if this time.Time is after the start and before to the end

func BetweenInclusive

func BetweenInclusive(t, start, end time.Time) bool

BetweenInclusive will return true if this time.Time is after or equal to the start and before or equal to the end

func BetweenStartInc

func BetweenStartInc(t, startInclusive, endExclusive time.Time) bool

BetweenStartInc will return true if this time.Time is after or equal to the start and before the end

func MakeEndpointResolver

func MakeEndpointResolver(services map[string]string) aws.EndpointResolver

MakeEndpointResolver is a factory function for creating an aws.EndpointResolver

Types

type Date

type Date time.Time

Date represents a sortable Date with fixed date precision.

Date implements attributevalue.Marshaler specifically for "YYYY-MM-DD" format which does not permit any timestamp; however, it can unmarshall from any Timestamp with nanosecond precision.

func ParseDate

func ParseDate(str string) (Date, error)

ParseDate will attempt to parse any RFC3339 Timestamp or date with format YYYY-MM-DD to a dynamocity.Date

func (Date) MarshalDynamoDBAttributeValue

func (t Date) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)

MarshalDynamoDBAttributeValue implements the attributevalue.Marshaler interface to marshal a dynamocity.Date into a DynamoDB AttributeValue string value with specific second precision

func (Date) MarshalJSON

func (t Date) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface to marshal RFC3339 timestamps with second precision

func (Date) String

func (t Date) String() string

String implements the fmt.Stringer interface to supply a native String representation for a value in time.RFC3339 Format with second precision

func (Date) Time

func (t Date) Time() time.Time

Time is a handler func to return an instance of dynamocity.Date as time.Time

func (*Date) UnmarshalDynamoDBAttributeValue

func (t *Date) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error

UnmarshalDynamoDBAttributeValue implements the attributevalue.Unmarshaler interface to unmarshal a attributevalue.AttributeValue into a dynamocity.Date. This unmarshal is flexible and supports any timestamp with nanosecond precision

func (*Date) UnmarshalJSON

func (t *Date) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface to unmarshal a date or RFC3339 timestamp

type MillisTime

type MillisTime time.Time

MillisTime represents a sortable strict RFC3339 Timestamp with fixed millisecond precision, making it string sortable. MillisTime implements attributevalue.Marshaler, dynamodbattribute.Unmarshaller The standard library time.RFC3339Nano format removes trailing zeros from the seconds field and thus may not sort correctly once formatted.

func (MillisTime) MarshalDynamoDBAttributeValue

func (t MillisTime) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)

MarshalDynamoDBAttributeValue implements the attributevalue.Marshaler interface to marshal a dynamocity.MillisTime into a DynamoDB AttributeValue string value with specific millisecond precision

func (MillisTime) MarshalJSON

func (t MillisTime) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface to marshal RFC3339 timestamps with millisecond precision

func (MillisTime) String

func (t MillisTime) String() string

String implements the fmt.Stringer interface to supply a native String representation for a value in RFC3339 Format with millisecond precision

func (MillisTime) Time

func (t MillisTime) Time() time.Time

Time is a handler func to return an instance of dynamocity.MillisTime as time.Time

func (*MillisTime) UnmarshalDynamoDBAttributeValue

func (t *MillisTime) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error

UnmarshalDynamoDBAttributeValue implements the attributevalue.Marshaler interface to unmarshal a types.AttributeValue into a dynamocity.MillisTime. This unmarshal is flexible on millisecond precision

func (*MillisTime) UnmarshalJSON

func (t *MillisTime) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface to marshal RFC3339 timestamps with millisecond precision

type NanoTime

type NanoTime time.Time

NanoTime represents a sortable strict RFC3339 Timestamp with fixed nanosecond precision, making it string sortable. NanoTime implements attributevalue.Marshaler, attributevalue.Unmarshaller The standard library time.RFC3339Nano format removes trailing zeros from the fractional seconds field and thus may not sort correctly once formatted.

func (NanoTime) MarshalDynamoDBAttributeValue

func (t NanoTime) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)

MarshalDynamoDBAttributeValue implements the attributevalue.Marshaler interface to marshal a dynamocity.NanoTime into a DynamoDB AttributeValue string value with specific nanosecond precision

func (NanoTime) MarshalJSON

func (t NanoTime) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface to marshal RFC3339 timestamps with nanosecond precision

func (NanoTime) String

func (t NanoTime) String() string

String implements the fmt.Stringer interface to supply a native String representation for a value in RFC3339 Format with nanosecond precision

func (NanoTime) Time

func (t NanoTime) Time() time.Time

Time is a handler func to return an instance of dynamocity.NanoTime as time.Time

func (*NanoTime) UnmarshalDynamoDBAttributeValue

func (t *NanoTime) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error

UnmarshalDynamoDBAttributeValue implements the dynamodb.Unmarshaler interface to unmarshal a types.AttributeValue into a dynamocity.NanoTime. This unmarshal is flexible on nanosecond precision

func (*NanoTime) UnmarshalJSON

func (t *NanoTime) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface to marshal RFC3339 timestamps with nanosecond precision

type OverrideEndpointResolver

type OverrideEndpointResolver struct {
	// contains filtered or unexported fields
}

OverrideEndpointResolver is an endpoint resolver for providing overridden endpoints for AWS services Overriding the endpoints for services is helpful for testing, including running dynamodb-local

func (*OverrideEndpointResolver) ResolveEndpoint

func (o *OverrideEndpointResolver) ResolveEndpoint(service, region string) (aws.Endpoint, error)

ResolveEndpoint implements the EndpointResolver interface which resolves an endpoint for a service endpoint id and region.

type SecondsTime

type SecondsTime time.Time

SecondsTime represents a sortable strict RFC3339 Timestamp with fixed second precision, making it string sortable. SecondsTime implements dynamodbattribute.Marshaler, dynamodbattribute.Unmarshaller specifically for the time.RFC3339 format which does not permit fractional seconds; however, once this format is marshalled it may be sorted correctly in a string value

func (SecondsTime) MarshalDynamoDBAttributeValue

func (t SecondsTime) MarshalDynamoDBAttributeValue() (types.AttributeValue, error)

MarshalDynamoDBAttributeValue implements the dynamodb.Marshaler interface to marshal a dynamocity.SecondsTime into a DynamoDB AttributeValue string value with specific second precision

func (SecondsTime) MarshalJSON

func (t SecondsTime) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface to marshal RFC3339 timestamps with second precision

func (SecondsTime) String

func (t SecondsTime) String() string

String implements the fmt.Stringer interface to supply a native String representation for a value in time.RFC3339 Format with second precision

func (SecondsTime) Time

func (t SecondsTime) Time() time.Time

Time is a handler func to return an instance of dynamocity.SecondsTime as time.Time

func (*SecondsTime) UnmarshalDynamoDBAttributeValue

func (t *SecondsTime) UnmarshalDynamoDBAttributeValue(av types.AttributeValue) error

UnmarshalDynamoDBAttributeValue implements the dynamodb.Unmarshaler interface to unmarshal a dynamodb.AttributeValue into a dynamocity.SecondsTime. This unmarshal is flexible on fractional second precision

func (*SecondsTime) UnmarshalJSON

func (t *SecondsTime) UnmarshalJSON(b []byte) error

UnmarshalJSON implements the json.Unmarshaler interface to marshal RFC3339 timestamps with second precision