documentloaders

Documentation

Index

• Constants
• type Option
  • func WithAnswerColumn(s string) Option
  • func WithChunkContentColumn(s string) Option
  • func WithFileNameColumn(s string) Option
  • func WithPageNumberColumn(s string) Option
  • func WithQuestionColumn(s string) Option
• type QACSV
  • func NewQACSV(r io.Reader, fileName string, opts ...Option) QACSV
  • func (c QACSV) Load(_ context.Context) ([]schema.Document, error)
  • func (c QACSV) LoadAndSplit(ctx context.Context, splitter textsplitter.TextSplitter) ([]schema.Document, error)

Constants

const (
	QuestionCol     = "q"
	AnswerCol       = "a"
	FileNameCol     = "file_name"
	PageNumberCol   = "page_number"
	ChunkContentCol = "chunk_content"
	LineNumber      = "line_number"
	QAFileName      = "qafile_name"
)

Types

type Option

type Option func(p *QACSV)

Option is a function type that can be used to modify the client.

func WithAnswerColumn

func WithAnswerColumn(s string) Option

func WithChunkContentColumn

func WithChunkContentColumn(s string) Option

func WithFileNameColumn

func WithFileNameColumn(s string) Option

func WithPageNumberColumn

func WithPageNumberColumn(s string) Option

func WithQuestionColumn

func WithQuestionColumn(s string) Option

type QACSV

type QACSV struct {
	// contains filtered or unexported fields
}

QACSV represents a QA CSV document loader.

func NewQACSV

func NewQACSV(r io.Reader, fileName string, opts ...Option) QACSV

NewQACSV creates a new qa csv loader with an io.Reader and optional column names for filtering.

func (QACSV) Load

func (c QACSV) Load(_ context.Context) ([]schema.Document, error)

Load reads from the io.Reader and returns a single document with the data.

func (QACSV) LoadAndSplit

func (c QACSV) LoadAndSplit(ctx context.Context, splitter textsplitter.TextSplitter) ([]schema.Document, error)

LoadAndSplit reads text data from the io.Reader and splits it into multiple documents using a text splitter.