scraping

README

Scraping Handler

Install

# Go 1.16+
go install github.com/vbetsun/scraping@latest

# Go version < 1.16
go get -u github.com/vbetsun/scraping 

Usage

Run program

package main

import (
	"log"
	"net/http"

	"github.com/vbetsun/scraping"
)

func main() {
	http.Handle("/", scraping.Handler())
	log.Fatal(http.ListenAndServe(":3000", nil))
}

Send request

curl --location --request POST 'http://localhost:3000' \
--header 'Content-Type: text/plain' \
--data-raw 'https://google.com
https://example.com'

Example of response

16798

1256

License

Golang Scraping Handler is provided under the MIT License

Documentation

Overview

Package scraping gives a middleware that scraps data from given URLs

Index

• Constants
• func Handler() http.Handler

Constants

const LineBreak = "\n"

LineBreak it's OS-specific symbol of new line

const MaxConnections uint32 = 999

MaxConnections it's maximum of simultaneous connections

Functions

func Handler

func Handler() http.Handler

Handler implements a simple handler for scrapping data from given urls