dyndump

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 5, 2015 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package dyndump provides a scan-based method for dumping an entire DynamoDB table using scan.

It supports parallel connections to the server for increased throughput along with rate limiting to a specific read capacity.

Items are written to an ItemWriter interface until the table is exhausted, or the Stop method is called.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Fetcher

type Fetcher struct {
	Dyn            *dynamodb.DynamoDB
	TableName      string
	ConsistentRead bool       // Setting to true will use double the read capacity.
	MaxParallel    int        // Maximum number of parallel requests to make to Dynamo.
	MaxItems       int64      // Maximum (approximately) number of items to read from Dynamo.
	ReadCapacity   float64    // Average global read capacity to use for the scan.
	Writer         ItemWriter // Retrieved items are sent to this ItemWriter.
	// contains filtered or unexported fields
}

Fetcher fetches data from DynamoDB at a specified capacity and writes fetched items to a writer implementing the ItemWriter interface.

func (*Fetcher) Run

func (f *Fetcher) Run() error

Run executes the fetcher, starting as many parallel reads as specified by the MaxParallel option and returns when the read has finished, failed, or been stopped.

func (*Fetcher) Stats

func (f *Fetcher) Stats() Stats

Stats returns current aggregate statistics about an ongoing or completed run. It is safe to call from concurrent goroutines.

func (*Fetcher) Stop

func (f *Fetcher) Stop()

Stop requests a clean shutdown of active readers. Active readers will complete the current request and then exit.

type ItemWriter

type ItemWriter interface {
	WriteItem(item map[string]*dynamodb.AttributeValue) error
}

ItemWriter is the interface expected by a Fetcher when writing retrieved DynamoDB items. Implementors must take care to support writes from concurrent goroutines.

type JSONItemEncoder

type JSONItemEncoder struct {
	// contains filtered or unexported fields
}

JSONItemEncoder implements the ItemWriter interface, formatting incoming items as JSON and writing them to the supplied writer.

func NewJSONItemEncoder

func NewJSONItemEncoder(w io.Writer, encodeTypes, numbersAsStrings bool) *JSONItemEncoder

NewJSONItemEncoder returns a JSONItmeEncoder writing to the supplied io.Writer.

func (*JSONItemEncoder) WriteItem

func (jie *JSONItemEncoder) WriteItem(item map[string]*dynamodb.AttributeValue) error

WriteItem JSON encodes a single DynamoDB item.

type Stats

type Stats struct {
	ItemsRead    int64
	CapacityUsed float64
}

Stats is returned by Fetcher.Stats to return current global throughput statistics.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL