README

crawler

This package contains a basic crawler built on top of Bokchoy.

Usage

Producer
go run main.go -run producer -url https://golang.org/

By default the depth is 1, you can override it with -depth flag.

Worker
go run main.go -run worker

By default the concurrency is 1, you can override it with -concurrency flag.

Expand ▾ Collapse ▴

Documentation

The Go Gopher

There is no documentation for this package.

Source Files

Directories

Path Synopsis