crawler

Documentation

Index

• Constants
• func RegisterChecks()
• type CrawlerCheck
  • func (c *CrawlerCheck) Merge(mergeCheck config.Check) error
  • func (c *CrawlerCheck) RunCheck()

Constants

const Crawler config.CheckType = "crawler"

Functions

func RegisterChecks

func RegisterChecks()

Types

type CrawlerCheck

type CrawlerCheck struct {
	config.CheckBase `yaml:",inline"`
	Domain           string   `yaml:"domain"`
	ExtraDomains     []string `yaml:"extra_domains"`
	IncludeURLs      []string `yaml:"include_urls"`
	Limit            int      `yaml:"limit"`
}

CrawlerCheck is a lightweight crawler that can be used to determine health of the project.

func (*CrawlerCheck) Merge

func (c *CrawlerCheck) Merge(mergeCheck config.Check) error

Merge implementation for file check.

func (*CrawlerCheck) RunCheck

func (c *CrawlerCheck) RunCheck()

RunCheck gathers input configuration and prepares the colly crawler to make HTTP requests to the project.

@see https://github.com/gocolly/colly/tree/master/_examples