crawl

package module

v0.0.0-...-3990c93 Latest Latest Go to latest Published: Jan 16, 2020 License: AGPL-3.0 Imports: 0 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

crawl is a library written in golang for crawling the web.

See godoc for the complete documentation.

crawl was written by Alexandru Cojocaru.

crawl is Free Software and under the AGPLv3 or any later version license.

There is no documentation for this package.

Path	Synopsis
html
httpcache
outline Package outline extracts information from html pages.	Package outline extracts information from html pages.
robots Package robots parses a robots.txt file as specified by Wikipedia https://en.wikipedia.org/wiki/Robots.txt	Package robots parses a robots.txt file as specified by Wikipedia https://en.wikipedia.org/wiki/Robots.txt
sitemap
stackexchange