digest

Documentation

Index

• func Create(config *Config) (map[uint64]uint64, map[uint64][]string, error)
• type Addition
• type Config
  • func NewConfig(r io.Reader, primaryKey Positions, valueColumns Positions, ...) *Config
• type Deletion
• type Differences
  • func Diff(baseConfig, deltaConfig Config) (Differences, error)
• type Digest
  • func CreateDigest(csv []string, separator string, pKey Positions, pRow Positions) Digest
• type Engine
  • func NewEngine(config Config) *Engine
  • func (e Engine) GenerateFileDigest() (*FileDigest, error)
  • func (e Engine) StreamDigests() (chan []Digest, chan error)
• type FileDigest
  • func NewFileDigest() *FileDigest
  • func (f *FileDigest) Append(d Digest)
  • func (f *FileDigest) SafeAppend(d Digest)
• type Modification
• type Positions
  • func (p Positions) Append(additional Positions) Positions
  • func (p Positions) Contains(position int) bool
  • func (p Positions) Join(csv []string, separator string) string
  • func (p Positions) String(csv []string, separator rune) string

Functions

func Create

func Create(config *Config) (map[uint64]uint64, map[uint64][]string, error)

Create can create a Digest using the Configurations passed. It returns the digest as a map[uint64]uint64. It can also keep track of the Source line.

Types

type Addition

type Addition []string

Addition is a row appearing in delta but missing in base

type Config

type Config struct {
	Key        Positions
	Value      Positions
	Include    Positions
	Reader     io.Reader
	Separator  rune
	LazyQuotes bool
}

Config represents configurations that can be passed to create a Digest.

Key: The primary key positions Value: The Value positions that needs to be compared for diff Include: Include these positions in output. It is Value positions by default.

func NewConfig

func NewConfig(
	r io.Reader,
	primaryKey Positions,
	valueColumns Positions,
	includeColumns Positions,
	separator rune,
	lazyQuotes bool,
) *Config

NewConfig creates an instance of Config struct.

type Deletion

type Deletion []string

Deletion is a row appearing in base but missing in delta

type Differences

type Differences struct {
	Additions     []Addition
	Modifications []Modification
	Deletions     []Deletion
}

Differences represents the differences between 2 csv content

func Diff

func Diff(baseConfig, deltaConfig Config) (Differences, error)

Diff finds the Differences between baseConfig and deltaConfig

type Digest

type Digest struct {
	Key    uint64
	Value  uint64
	Source []string
}

Digest represents the binding of the key of each csv line and the digest that gets created for the entire line

func CreateDigest

func CreateDigest(csv []string, separator string, pKey Positions, pRow Positions) Digest

CreateDigest creates a Digest for each line of csv. There will be one Digest per line

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine to create a FileDigest

func NewEngine

func NewEngine(config Config) *Engine

NewEngine instantiates an engine

func (Engine) GenerateFileDigest

func (e Engine) GenerateFileDigest() (*FileDigest, error)

GenerateFileDigest generates FileDigest with thread safety

func (Engine) StreamDigests

func (e Engine) StreamDigests() (chan []Digest, chan error)

StreamDigests starts creating digests in the background Returns 2 buffered channels, a digestChannel and an errorChannel

digestChannel has all digests errorChannel has any errors created during processing

If there are any errors while processing csv, all existing go routines to creates digests are waited to be closed and the digestChannel is closed at the end. Only after that an error is created on the errorChannel.

type FileDigest

type FileDigest struct {
	Digests   map[uint64]uint64
	SourceMap map[uint64][]string
	// contains filtered or unexported fields
}

FileDigest represents the digests created from one file

func NewFileDigest

func NewFileDigest() *FileDigest

NewFileDigest to instantiate a new FileDigest

func (*FileDigest) Append

func (f *FileDigest) Append(d Digest)

Append a Digest to a FileDigest This operation is not thread safe

func (*FileDigest) SafeAppend

func (f *FileDigest) SafeAppend(d Digest)

SafeAppend a Digest to a FileDigest This operation is thread safe

type Modification

type Modification struct {
	Original []string
	Current  []string
}

Modification is a row present in both delta and base with the values column changed in delta

type Positions

type Positions []int

Positions represents positions of columns in a CSV array.

func (Positions) Append

func (p Positions) Append(additional Positions) Positions

Append additional positions to existing positions. Imp: Removes Duplicate. Does not mutate the original array

func (Positions) Contains

func (p Positions) Contains(position int) bool

Contains returns true if position is already present in Positions

func (Positions) Join

func (p Positions) Join(csv []string, separator string) string

Join plucks the values from CSV from their respective positions and concatenates them using separator as a string.

func (Positions) String

func (p Positions) String(csv []string, separator rune) string

String method converts to csv mapping to positions escapes necessary characters