norman

README

Readme if you like

Support: This library is for Golang and targetted against the AWS v2 SDK for Golang.
Purpose: Helps to build Go structs for a single table DynamoDB database driven application.
Why: Building backends in Go for DynamoDB felt like a game of battleships in Go.
Norman?: NORMAN - Native Object Relational Mapper Against NoSQL.

Background

First off - NORMAN is simple and designed to do a couple of things; Stay sane and write less code.

I dislike building a 'schema' for DynamoDB in one tool, then building a Go struct, then tying it together with a set of filters, projections, input attribtues and key data. It felt too disconnected and one typo leads to a world of pain. I much prefer to drive my table design straight from the programming language I use regularly, which is Go. So, with NORMAN, you can build your Go data types that provide structure for inputting and retrieving data against DynamoDB and maintain some sanity.

NORMAN offers:

1. DynamoDB schema creation through composing tagged native Go data types. This promotes re-use of structs, maintains a single table design paradigm and more importantly, helps you to stay sane when driving the data modelling exercise directly from Go, instead of on a tool.

2. AttributeValue Input list generation from native Go data types. That's right, load your Go structs up with data and use them in operations like PutItem etc without using the AWS API directly. It's only a little abstracted and designed for speed and less code.

Note: DynamoDB is schemaless, and unless you are changing your PK data, your modelled schema is only important for your application; DynamoDB doesn't give a monkeys. So, when I say schema, this is a sanity keeping tool for modelling your application data for DynamoDB in Go.

See examples directory for more information.

Usage

Look in the examples directory. It provides you with just enough help to build a schema and minimise the amount of AWS boilerplate code you need.

examples/build_schema shows you how to build a schema and what struct field tags are required.

examples/use_schema shows you how to use this project to simplify your access patterns.

In summary, your project's types need to be in their own module and imported into a project with the sole purpose of creating a schema. These are the tags you can use.

    // TAG INFORMATION
	// ddb_name tag is overloaded with data
	
    // ddb_name = "pk:partitionkeyname"
	// ddb_name = "sk:sortkeyname"
	// ddb_name = "att:attributename"
	// Global secondary indexes are indexed.
	// ddb_name = "gsi#<gsi_name>:pk:<pk_name>"
	// ddb_name = "gsi#<gsi_name>:sk:<sk_name>"

    // Example
    type PartitionData struct {
        PK string `json:"PK" ddb_name:"pk:PK"`
    }

    type OptionalData struct {
        OptionalField string `json:"optionalField" ddb_name:"att:optionalField"`
    }

    type Test struct {
        PartitionData
        *OptionalField
        TestField string `json:"testField" ddb_name:"att:testField"`
    }

If there is a slice or embedded struct that's optional, it's imperative to make these entires pointers, else DynamoDB will be upset. Them being empty will still include them for processing. Making them a pointer (with it being nil), will exclude optional elements from processing.

License

Apache 2.0. Read the license file.

TODO

Lots. Some areas are not tested very well and no test files exist to date. This is WIP.

That said, I expect the struct field tags to remain stable and the main API.

Contributing

Open an issue, discuss the issue. Propose an idea and then provide some code. If you just hit me unsolicited, I won't help. I'm too busy to decipher your issue or need.

Donating & Funding This Work

If you find this helpful, you can help support the effort by making a financial contribution. In turn, I shall probably most likely spend this on coffee.

https://www.buymeacoffee.com/davedotdev

Documentation

Index

• func DeepFields(iface interface{}) []reflect.Value
• func DeepTypes(iface interface{}) []reflect.StructField
• func GetItems(input interface{}) (map[string]types.AttributeValue, error)
• type Attribute
• type DDBField
• type DDBFieldType
  • func GetFieldData(fieldName string) (DDBFieldType, []string, error)
• type DDBSchema
  • func NewDDBSchema(name, author, description string) DDBSchema
  • func (ddb *DDBSchema) Process(inputs ...interface{}) error
• type DataModel
• type GSI
• type GlobalSecondaryIndexes
• type KeyAttributes

Functions

func DeepFields

func DeepFields(iface interface{}) []reflect.Value

DeepFields returns flattened slice of struct fields

func DeepTypes

func DeepTypes(iface interface{}) []reflect.StructField

DeepTypes function returns slice of flattened struct fields

func GetItems

func GetItems(input interface{}) (map[string]types.AttributeValue, error)

GetItems returns map of type Attributes for PutItem.

Types

type Attribute

type Attribute struct {
	AttributeName string `json:"AttributeName,omitempty"`
	AttributeType string `json:"AttributeType,omitempty"`
}

Component of DDBSchema

type DDBField

type DDBField struct {
	Type DDBFieldType
	Name string
}

DDBField

type DDBFieldType

type DDBFieldType int

const (
	UNKNOWN DDBFieldType = iota
	DDB_PK
	DDB_SK
	DDB_ATT
	DDB_GSIPK
	DDB_GSISK
)

func GetFieldData

func GetFieldData(fieldName string) (DDBFieldType, []string, error)

GetFieldData returns the enum type of field, a list of parts and an error

type DDBSchema

type DDBSchema struct {
	ModelName     string `ddb_modelname:"ModelName"`
	ModelMetadata struct {
		Author           string `json:"Author"`
		DateCreated      string `json:"DateCreated"`
		DateLastModified string `json:"DateLastModified"`
		Description      string `json:"Description"`
		AWSService       string `json:"AWSService"`
		Version          string `json:"Version"`
	} `json:"ModelMetadata"`
	DataModel []DataModel `json:"DataModel"`
}

DynamoDB 2.0 Schema

func NewDDBSchema

func NewDDBSchema(name, author, description string) DDBSchema

NewDDBSchema returns a new DDBSchema object.

func (*DDBSchema) Process

func (ddb *DDBSchema) Process(inputs ...interface{}) error

Process accepts a struct with ddb tags and adds it to the schema. Warning: You can accidentally overwrite the PK/SK information if you're not careful.

type DataModel

type DataModel struct {
	TableName     string        `json:"TableName"`
	KeyAttributes KeyAttributes `json:"KeyAttributes"`

	NonKeyAttributes       []Attribute              `json:"NonKeyAttributes"`
	GlobalSecondaryIndexes []GlobalSecondaryIndexes `json:"GlobalSecondaryIndexes,omitempty"`
}

Component of DDBSchema

type GSI

type GSI struct {
	Index int
	Attribute
}

Internal type to NORMAN

type GlobalSecondaryIndexes

type GlobalSecondaryIndexes struct {
	KeyAttributes struct {
		PartitionKey Attribute  `json:"PartitionKey"`
		SortKey      *Attribute `json:"SortKey,omitempty"`
	} `json:"KeyAttributes"`
	IndexName  string `json:"IndexName"`
	Projection struct {
		ProjectionType string `json:"ProjectionType"`
	} `json:"Projection"`
}

Component of DDBSchema

type KeyAttributes

type KeyAttributes struct {
	PartitionKey Attribute  `json:"PartitionKey"`
	SortKey      *Attribute `json:"SortKey,omitempty"`
}

KeyAttributes is a component of DataModel