extractor

Documentation

Index

• Constants
• type Extractor
  • func New(strategy, config string) (Extractor, error)
• type ReadabilityExtractor
  • func (e *ReadabilityExtractor) Extract(html io.Reader, pageURL string, logger *slog.Logger) (string, error)
• type SelectorExtractor
  • func (e *SelectorExtractor) Extract(html io.Reader, pageURL string, logger *slog.Logger) (string, error)

Constants

const (
	StrategyReadability = "readability"
	StrategySelector    = "selector"
)

Strategy constants for extraction methods.

Types

type Extractor

type Extractor interface {
	Extract(html io.Reader, pageURL string, logger *slog.Logger) (string, error)
}

Extractor pulls the main content from an HTML document.

func New

func New(strategy, config string) (Extractor, error)

New returns an Extractor for the given strategy and config. An empty strategy defaults to StrategyReadability.

type ReadabilityExtractor

type ReadabilityExtractor struct{}

ReadabilityExtractor uses the go-readability library to extract article content.

func (*ReadabilityExtractor) Extract

func (e *ReadabilityExtractor) Extract(html io.Reader, pageURL string, logger *slog.Logger) (string, error)

type SelectorExtractor

type SelectorExtractor struct {
	Selector string
}

SelectorExtractor uses a CSS selector to extract a specific element from the page.

func (*SelectorExtractor) Extract

func (e *SelectorExtractor) Extract(html io.Reader, pageURL string, logger *slog.Logger) (string, error)