csv

Documentation

Index

• type CSVStreamParser
  • func DefaultCSVStreamParser(rowsPerDocument int, hasHeader bool) *CSVStreamParser
  • func (p *CSVStreamParser) GetSupportedTypes() []string
  • func (p *CSVStreamParser) Parse(ctx context.Context, content []byte, metadata map[string]any) (*core.Document, error)
  • func (p *CSVStreamParser) ParseStream(ctx context.Context, r io.Reader, metadata map[string]any) (<-chan *core.Document, error)
  • func (p *CSVStreamParser) Supports(contentType string) bool

Types

type CSVStreamParser

type CSVStreamParser struct {
	// contains filtered or unexported fields
}

CSVStreamParser reads CSV files and streams them out. It bundles N rows into a single Document to balance chunking granularity.

func DefaultCSVStreamParser

func DefaultCSVStreamParser(rowsPerDocument int, hasHeader bool) *CSVStreamParser

func (*CSVStreamParser) GetSupportedTypes

func (p *CSVStreamParser) GetSupportedTypes() []string

func (*CSVStreamParser) Parse

func (p *CSVStreamParser) Parse(ctx context.Context, content []byte, metadata map[string]any) (*core.Document, error)

Parse implements core.Parser interface.

func (*CSVStreamParser) ParseStream

func (p *CSVStreamParser) ParseStream(ctx context.Context, r io.Reader, metadata map[string]any) (<-chan *core.Document, error)

func (*CSVStreamParser) Supports

func (p *CSVStreamParser) Supports(contentType string) bool