Documentation

Overview

    Package extensions implements various helper addons for Colly

    Index

    Constants

    This section is empty.

    Variables

    This section is empty.

    Functions

    func RandomUserAgent

    func RandomUserAgent(c *colly.Collector)

      RandomUserAgent generates a random browser user agent on every request

      func Referer

      func Referer(c *colly.Collector)

        Referer sets valid Referer HTTP header to requests. Warning: this extension works only if you use Request.Visit from callbacks instead of Collector.Visit.

        func URLLengthFilter

        func URLLengthFilter(c *colly.Collector, URLLengthLimit int)

          URLLengthFilter filters out requests with URLs longer than URLLengthLimit

          Types

          This section is empty.