scrape

Documentation

Overview

Package scrape is an internal package of the tool Crawl, responsible for extracting data from web pages. It does not allow for some likely use cases that don't come up in a crawl, like identifying tags based only on the value of some attribute.

Index

• func Attribute(key string, n *html.Node) string
• func Classes(node *html.Node) []string
• func NodeByID(id string, node *html.Node) *html.Node
• func NodesByClassName(class string, node *html.Node) []*html.Node
• func NodesByName(name string, node *html.Node) []*html.Node
• func NodesByTagName(tag string, node *html.Node) []*html.Node
• func Query(tag string, attrs map[string]string, n *html.Node) *html.Node
• func QueryAll(tag string, attrs map[string]string, n *html.Node) []*html.Node
• func Text(n *html.Node) string

Functions

func Attribute

func Attribute(key string, n *html.Node) string

func Classes

func Classes(node *html.Node) []string

func NodeByID

func NodeByID(id string, node *html.Node) *html.Node

func NodesByClassName

func NodesByClassName(class string, node *html.Node) []*html.Node

func NodesByName

func NodesByName(name string, node *html.Node) []*html.Node

func NodesByTagName

func NodesByTagName(tag string, node *html.Node) []*html.Node

func Query

func Query(tag string, attrs map[string]string, n *html.Node) *html.Node

Query returns a the first *html.Node that appears in the tree n, with tag name tag, and whose attributes match those described by attrs.

func QueryAll

func QueryAll(tag string, attrs map[string]string, n *html.Node) []*html.Node

QueryAll returns a list of *html.Node that appear in the tree n, with tag name tag, and whose attributes match those described by attrs.

func Text

func Text(n *html.Node) string