scraper

Documentation

Index

• type Parameters
• type Scraper
  • func New(parameters Parameters) *Scraper
  • func (s *Scraper) GetWebsite(secure bool) string
  • func (s *Scraper) Log(v ...interface{})
  • func (s *Scraper) Scrape(scrapedEmails *[]string) error

Types

type Parameters

type Parameters struct {
	Website             string
	MaxDepth            int
	Emails              bool
	Recursively         bool
	Async               bool
	PrintLogs           bool
	FollowExternalLinks bool
	JSWait              bool
	Debug               bool
}

Configuration for the scrape

type Scraper

type Scraper struct {
	Parameters
}

Goal of the scrape

func New

func New(parameters Parameters) *Scraper

Initiate new scraper

func (*Scraper) GetWebsite

func (s *Scraper) GetWebsite(secure bool) string

func (*Scraper) Log

func (s *Scraper) Log(v ...interface{})

func (*Scraper) Scrape

func (s *Scraper) Scrape(scrapedEmails *[]string) error

Scrape is responsible for main scraping logic