Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CrawlHandler

type CrawlHandler struct {
	// contains filtered or unexported fields
}

func NewCrawlHandler

func NewCrawlHandler(queue *bokchoy.Queue, parser parser.Parser, timeout time.Duration) *CrawlHandler

    NewCrawlHandler initializes a new CrawlHandler instance.

    func (*CrawlHandler) AddCrawl

    func (h *CrawlHandler) AddCrawl(url string, statusCode int)

      AddCrawl adds a new crawl to the storage.

      func (*CrawlHandler) Crawl

      func (h *CrawlHandler) Crawl(ctx context.Context, baseURL string, url string, depth int) (*bokchoy.Task, error)

        Crawl publishes a new task to crawl an url with its base URL.

        func (*CrawlHandler) Crawls

        func (h *CrawlHandler) Crawls() []string

          Crawls returns the crawls.

          func (*CrawlHandler) Handle

          func (h *CrawlHandler) Handle(r *bokchoy.Request) error

            Handle handles a bokchoy.Request.

            Source Files