crawlers

package
Version: v0.0.0-...-ba9c0dd Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 22, 2015 License: MIT Imports: 4 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Crawler

type Crawler struct {
	// contains filtered or unexported fields
}

func NewCrawler

func NewCrawler(fetcher contracts.Fetcher, processor contracts.Processor, maxNbConcurrentFetchers int) Crawler

Create a new crawler.

func (Crawler) Crawl

func (c Crawler) Crawl(url string, depth int)

Crawl recursively starting from a given URL with a maximum depth

Source Files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL