extensions

package
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 13, 2019 License: Apache-2.0 Imports: 3 Imported by: 0

Documentation

Overview

Package extensions implements various helper addons for Colly

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func RandomUserAgent 

func RandomUserAgent(c *colly.Collector)

RandomUserAgent generates a random browser user agent on every request

func Referer added in v1.2.0 

func Referer(c *colly.Collector)

Referer sets valid Referer HTTP header to requests. Warning: this extension works only if you use Request.Visit from callbacks instead of Collector.Visit.

func URLLengthFilter 

func URLLengthFilter(c *colly.Collector, URLLengthLimit int)

URLLengthFilter filters out requests with URLs longer than URLLengthLimit

Types

This section is empty.

Source Files

  • extensions.go
  • random_user_agent.go
  • referer.go
  • url_length_filter.go

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.