export

README

Export: Dumping tabular data from Go

Package export provides tools to dump tabular data from Go data structures in various formats.

Documentation

Overview

Package export provides tools to dump tabulated data.

Export allows to dump tabular data in different output formats. The main type is Extractor which determines which data is output and in which order. An Extractor is constructed from (almost) any slice type and may access nested fields and/or methods of the slice elements.

Example

Given a struct type S with a method M and a slice of S data

type S struct {
    A int
    B string
    C struct{T time.Time}
}

func (s S) M() float64 { return float64(s.A)/2 }

data := []S{
    {4, "Hello"},
    {5, "World!"},
}

an Extractor ex for data could be constructed like

ex, _ := NewExtractor(data, "B", "M()", "A", "C.T", "C.T.Day()")

This Extractor can be used to dump data in CSV format like this:

csvdumper := CSVDumper{Writer: csv.NewWriter(os.Stdout)}
csvdumper.Dump(ex, DefaultFormat)

Column Specifiers

A columns specifier during construction of an Extractor determines which field, method, nested field, method on nested field, and so on shall be exported:

• Only exported fields can be exported.
• Accessing a nested field (in the example T) inside a field (C in the example) is written as T.C
• Methods require "()" in the columne specifier (here "M()").
• Methods may not take arguments.
• Only methods returnig one value or a (value, error) pair may be used.
• Pointers are dereferenced automatically.
• Nil Pointers and method calls returning a non-nil error result in a NA value for this field.

The final field (or the type returned by a final method call) must be one of:

• bool
• uint8, uint16, ..., int64
• float32 and float64
• complex64 and complex128
• string
• time.Time and time.Duration

This package handles floats and int as 64bit values and complex values as complex128. Thus an uint64 may overflow without notice.

Dumping

Dumping the data bound to an Extractor is done via a Dumper. This package provides three types: CSVDumper, TabDumper and RVecDumper. It is the dumpers responsibility to iterate over the rows and columns of an Extractor and generating values via the the Columns Print method which takes a Formater which does the actual string generation.

Example

package main

import (
	"errors"
	"os"
	"text/tabwriter"
	"time"
)

// Some is some structure.
type Some struct {
	Flt    float64
	Str    string
	IntP   *int
	Other  Other
	OtherP *Other
}

func (s Some) Method1() int {
	return int(s.Flt + 0.5)
}

// Method2 may fail.
func (s Some) Method2() (bool, error) {
	if s.Str == "" {
		return false, errors.New("empty")
	}
	return len(s.Str) > 5, nil
}

type Other struct {
	Start time.Time
}

func (o Other) Unix() int64 {
	return o.Start.Unix()
}

func main() {
	// Set up some values.
	eight, nine := 8, 9
	t0 := time.Date(2009, 12, 28, 8, 45, 0, 0, time.UTC)
	t1 := time.Date(2014, 12, 12, 23, 59, 59, 0, time.UTC)
	t2 := time.Date(2099, 1, 1, 0, 1, 0, 0, time.UTC)

	// Data is a slice of Some things.
	var data = []Some{
		Some{3.14, "Hello", &eight, Other{t0}, &Other{t1}},
		Some{2.72, "Go", nil, Other{t1}, &Other{t2}},
		Some{1.41, "", &nine, Other{t2}, nil},
	}
	extractor, err := NewExtractor(data,
		"Flt", "Str", "IntP", // Accessing fields and pointer fields
		"Method1()", "Method2()", // Accessing results of methods.
		"Other.Start", "OtherP.Unix()", // Accessing nested elements.
		"Other.Start.Day()") // Accessing methods on nested elements.
	if err != nil {
		panic(err.Error())
	}

	// Rename the last column which defaults to "Other.Start.Day".
	extractor.Columns[7].Name = "DayOfMonth"

	w := &tabwriter.Writer{}
	w.Init(os.Stdout, 1, 8, 1, ' ', 0)
	tab := TabDumper{Writer: w}
	format := DefaultFormat // A human readable format. Missing values are omited.
	format.TimeLoc = nil    // Clear location to output in original (UTC) location.
	tab.Dump(extractor, format)
	w.Flush()

}

Output:
Flt  Str   IntP Method1 Method2 Other.Start         OtherP.Unix DayOfMonth
3.14 Hello 8    3       false   2009-12-28T08:45:00 1418428799  28
2.72 Go         3       false   2014-12-12T23:59:59 4070908860  12
1.41       9    1               2099-01-01T00:01:00             1

Index

• Variables
• type CSVDumper
  • func (d CSVDumper) Dump(e *Extractor, format Format) error
• type Column
  • func (c Column) Print(f Formater, i int) string
  • func (c Column) Type() Type
• type Dumper
• type Extractor
  • func NewExtractor(data interface{}, columnSpecs ...string) (*Extractor, error)
  • func (e *Extractor) Bind(data interface{})
• type Format
  • func (f Format) Bool(b bool) string
  • func (f Format) Complex(c complex128) string
  • func (f Format) Duration(d time.Duration) string
  • func (f Format) Float(x float64) string
  • func (f Format) Int(i int64) string
  • func (f Format) NA() string
  • func (f Format) String(s string) string
  • func (f Format) Time(t time.Time) string
• type Formater
• type RVecDumper
  • func (d RVecDumper) Dump(e *Extractor, format Format) error
• type TabDumper
  • func (d TabDumper) Dump(e *Extractor, format Format) error
• type Type
  • func (t Type) String() string

Examples

• Package

Variables

var DefaultFormat = Format{
	TrueRep:     "true",
	FalseRep:    "false",
	IntFmt:      "%d",
	FloatFmt:    "%.4g",
	StringFmt:   "%s",
	TimeFmt:     "2006-01-02T15:04:05",
	TimeLoc:     time.Local,
	DurationFmt: "%s",
	NARep:       "",
	NaNRep:      "",
	PInfRep:     "+\u221e",
	MInfRep:     "-\u221e",
}

DefaultFormat contains default formating options which produce pleasant human readable output.

var PreciseFormat = Format{
	TrueRep:     "true",
	FalseRep:    "false",
	IntFmt:      "%d",
	FloatFmt:    "%g",
	StringFmt:   "%q",
	TimeFmt:     time.RFC3339Nano,
	DurationFmt: "%s",
	TimeLoc:     nil,
	NARep:       "",
	NaNRep:      "NaN",
	PInfRep:     "+\u221e",
	MInfRep:     "-\u221e",
}

PreciseFormat contains formatin options which tries to preserve the original data pretty well.

var RFormat = Format{
	TrueRep:     "TRUE",
	FalseRep:    "FALSE",
	IntFmt:      "%d",
	FloatFmt:    "%.9g",
	StringFmt:   "%q",
	TimeFmt:     `as.POSIXct("2006-01-02 15:04:05")`,
	DurationFmt: "%d",
	TimeLoc:     time.Local,
	NARep:       "NA",
	NaNRep:      "NA",
	PInfRep:     "Inf",
	MInfRep:     "-Inf",
}

RFormat contains formating options usefull if you want to read the generated dumps into R.

Types

type CSVDumper

type CSVDumper struct {
	Writer     *csv.Writer // Writer is the csv writer to output the data.
	OmitHeader bool        // OmitHeader suppresses the header line in the generated CSV.
}

CSVDumper dumps values to a csv writer.

func (CSVDumper) Dump

func (d CSVDumper) Dump(e *Extractor, format Format) error

Dump implements the Dump method of a Dumper.

type Column

type Column struct {
	// Name is the name of the column. It is created based on the
	// column spec during construction of a new Extractor and may
	// be changed afterwards.
	Name string
	// contains filtered or unexported fields
}

Column represents one column in the export. Columns are created during construction of an Extractor only.

func (Column) Print

func (c Column) Print(f Formater, i int) string

Print the i'th entry of column c with the given format.

func (Column) Type

func (c Column) Type() Type

Type returns the type of the column c.

type Dumper

type Dumper interface {
	// Dump the data defined in e in the given format.
	Dump(e *Extractor, format Format) error
}

Dumper is the interface which wrapps the Dump methods

type Extractor

type Extractor struct {
	// N is the numer of elements in the currently bound data.
	N int

	// Columns contains all the columns to extract. After
	// creation of an Extractor Columns may be manipulated, e.g.
	// setting a custom name for a column or rearanging or dropping
	// columns.
	Columns []Column
	// contains filtered or unexported fields
}

Extractor provides access to fields and methods of tabular data. An extractor must be constructed with NewExtractor and can be rebound to new data sets anytime by Bind.

func NewExtractor

func NewExtractor(data interface{}, columnSpecs ...string) (*Extractor, error)

NewExtractor returns an extractor for the given column specifications of data.

func (*Extractor) Bind

func (e *Extractor) Bind(data interface{})

Bind (re)binds e to data which must be of the same type as the data used during the construction of e.

type Format

type Format struct {
	TrueRep, FalseRep string // String values of boolean true and false.
	IntFmt            string // Package fmt style verb for int printing.
	FloatFmt          string // Package fmt style verb for float and complex printing.
	StringFmt         string // Package fmt style verb for string printing.
	TimeFmt           string // A package time layout string.
	DurationFmt       string // Either %s (human redable) or %d (nanoseconds)

	// TimeLoc is the location in which times are presented.
	// If a nil TimeLoc is used the times are presented in their
	// original location.
	TimeLoc *time.Location

	NARep            string // Representation of a missing value.
	NaNRep           string // Representation of a floating point NaN.
	PInfRep, MInfRep string // Positiv and negativ infinite. Complex uses PInf only
}

Format describes how different fields types will be formated, either by specifying a literal representation, a package fmt style verb or a package time time format string.

func (Format) Bool

func (f Format) Bool(b bool) string

func (Format) Complex

func (f Format) Complex(c complex128) string

func (Format) Duration

func (f Format) Duration(d time.Duration) string

func (Format) Float

func (f Format) Float(x float64) string

func (Format) Int

func (f Format) Int(i int64) string

func (Format) NA

func (f Format) NA() string

func (Format) String

func (f Format) String(s string) string

func (Format) Time

func (f Format) Time(t time.Time) string

type Formater

type Formater interface {
	Bool(b bool) string
	Int(i int64) string
	Float(f float64) string
	Complex(c complex128) string
	String(s string) string
	Time(t time.Time) string
	Duration(d time.Duration) string

	// NA is used to produce missing values for nil pointers or
	// method invocations which returned an error.
	NA() string
}

A Formater can convert baisc types to strings.

type RVecDumper

type RVecDumper struct {
	Writer io.Writer // Writer is the writer to output the data.

	// DataFrame is the name of a R data frame to construct from the
	// individual column vectors. A empty value suppresses the generation
	// of this combining data frame.
	DataFrame string
}

RVecDumper dumps as a R vectors, optionaly combined into a data frame.

func (RVecDumper) Dump

func (d RVecDumper) Dump(e *Extractor, format Format) error

Dump implements the Dump method of a Dumper. The given format must produce suitabel literals for the R values if the dumped data shall be processed as R code; RFormat is suitable.

type TabDumper

type TabDumper struct {
	Writer     *tabwriter.Writer // Writer is the tabwriter to output the data.
	OmitHeader bool              // OmitHeader suppresses the header line in the generated CSV.
}

TabDumper dumps the value to a tabwriter.

func (TabDumper) Dump

func (d TabDumper) Dump(e *Extractor, format Format) error

Dump implements the Dump method of a Dumper. Dump does not call Flush on the underlying tabwriter.

type Type

type Type uint

Type represents the basic type of a column.

const (
	NA Type = iota
	Bool
	Int
	Float
	Complex
	String
	Time
	Duration
)

func (Type) String

func (t Type) String() string

String returns the name of t.