cmd

Documentation

Index

• func Execute(args []string, version string)
• func InputFormat(path string) string
• func ParseRows(path string, opts Options, onRow func(Row) error) error
• func ParseTSV(r io.Reader, opts Options, onRow func(Row) error) error
• func ParseTSVChan(ctx context.Context, r io.Reader, opts Options) (<-chan Row, <-chan error)
• func RowCount(path string) (int64, error)
• type Options
  • func DefaultOptions() Options
  • func (o Options) WithAllowCRLF(allow bool) Options
  • func (o Options) WithPreserveOrder(preserve bool) Options
• type Row

Functions

func Execute

func Execute(args []string, version string)

func InputFormat

func InputFormat(path string) string

func ParseRows

func ParseRows(path string, opts Options, onRow func(Row) error) error

func ParseTSV

func ParseTSV(r io.Reader, opts Options, onRow func(Row) error) error

ParseTSV streams a TSV from r, invoking onRow for each line. It keeps memory bounded by reusing chunk buffers; row data is only valid inside onRow.

func ParseTSVChan

func ParseTSVChan(ctx context.Context, r io.Reader, opts Options) (<-chan Row, <-chan error)

ParseTSVChan exposes rows over a channel. Rows are copied to keep the channel consumer safe from buffer reuse. Errors are sent on errCh after the rows channel closes.

func RowCount

func RowCount(path string) (int64, error)

Types

type Options

type Options struct {
	BufferSize           int  // Size of the bufio.Reader buffer
	ChunkSize            int  // Bytes to read per chunk before splitting into lines
	BatchLines           int  // How many lines to hand to a worker at once
	Workers              int  // Number of parsing workers
	StrictColumns        bool // Enforce a fixed column count (first row if ExpectedColumns == 0)
	ExpectedColumns      int  // Expected column count when StrictColumns is true (0 to infer from first row)
	PreserveOrder        bool // Deliver rows in file order
	AllowCRLF            bool // Trim trailing \r when present
	Progress             *progress
	SkipProgressFirstRow bool
	Timeout              time.Duration
}

Options controls TSV parsing performance characteristics.

func DefaultOptions

func DefaultOptions() Options

DefaultOptions returns a tuned baseline for large TSVs.

func (Options) WithAllowCRLF

func (o Options) WithAllowCRLF(allow bool) Options

WithAllowCRLF toggles CRLF stripping (default: enabled).

func (Options) WithPreserveOrder

func (o Options) WithPreserveOrder(preserve bool) Options

WithPreserveOrder overrides the default ordering behavior.

type Row

type Row struct {
	Line   int64
	Fields [][]byte
}

Row is a view over a TSV line. Fields point into an internal buffer and are only valid for the duration of the callback in ParseTSV.