scraper

Documentation

Overview

scraper/fetch.go

scraper/parse.go

scraper/write.go

Index

• func ExtractArticles(doc *html.Node) []models.Article
• func FetchHTML(url string) (*html.Node, error)
• func WriteToCSV(articles []models.Article, filename string)

Functions

func ExtractArticles

func ExtractArticles(doc *html.Node) []models.Article

ExtractArticles extracts articles from the given HTML document.

func FetchHTML

func FetchHTML(url string) (*html.Node, error)

FetchHTML fetches the HTML content of the given URL and returns a parsed HTML node.

func WriteToCSV

func WriteToCSV(articles []models.Article, filename string)

WriteToCSV writes the scraped articles to a CSV file.