scraper

Documentation

Index

• func Run()
• type Config

Functions

func Run

func Run()

main function responsible for initializing the web scraping process. It sets up the logger, creates collector instances, defines URL and limits for the main collector, and specifies callback functions for HTML elements. It initiates the web scraping process by visiting the initial URL and waits for the wait group to finish before printing "DONE" to the console and saving the collected data to a JSON file using the saveToJSON function.

Types

type Config

type Config struct {
	Login    string
	Password string
	Proxys   []string
}