README
¶
Orb
Orb is heavily inspired by Cheerio.js, a elegant solution to traversing/manipulating the DOM model. That being said, Orb combines the power of Go and the API of Cheerio to produce a easy and fast library for gophers.
Table of Contents
Project Status
Currently Orb is stil a work in progress and should not be used for production apps.
Installation
Please make sure that your Go version is v1.1+
$ go get github.com/anthonygedeon/orb
Usage
To see more examples and documentation, please visit godoc.org
package main
import (
"fmt"
"net/http"
"github.com/anthonygedeon/orb"
)
func main() {
resp, err := http.Get("https://golang.org/dl/")
if err != nil {
fmt.Errorf("invalid get request: %s", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
fmt.Errorf("status error: %v", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
fmt.Errorf("invalid body content: %s", err)
}
doc := string(body)
o := o.Load(doc)
o(".downloadtable .download").Each(func(i int, node *orb.Query) {
textNode := node.Text()
fmt.Println(textNode)
})
}
Contributing
First off, I'm glad that you would like to contribute to Orb :)
To contribute please take a look at the contribution guide for instructions.
Related Projects
-
cheeriojs/cheerio - Cheerio parses markup and provides an API for traversing/manipulating the resulting data structure.
-
PuerkitoBio/goquery - a little like that j-thing, only in Go.
-
MontFerret/ferret - declarative web scraping.
-
andybalholm/cascadia - the CSS selector engine.
-
suntong/cascadia - a command-line interface to the cascadia CSS selector library, useful to test selectors.
-
gocolly/colly - a lightning fast and elegant Scraping Framework
-
gnulnx/goperf - a website performance test tool that also fetches static assets.
-
Dataflow kit - Web Scraping framework for Gophers.
-
Geziyor - a fast web crawling & scraping framework for Go. Supports JS rendering.
License
Orb is released under the MIT License. See LICENSE
Documentation
¶
Overview ¶
Package orb provides primitives for scraping HTML from web pages
Index ¶
- func Load(document string) func(sel string) *Query
- type Document
- type Query
- func (qs *Query) AddAttr()
- func (qs *Query) AddClass(className string) *Query
- func (qs *Query) Append()
- func (qs *Query) Attr()
- func (qs *Query) Children() *Query
- func (qs *Query) Data(target string) string
- func (qs *Query) Each(fn func(i int, node *Query))
- func (qs *Query) Filter(selector string)
- func (qs *Query) Find(sel string) *Query
- func (qs *Query) FindAll(sel string) *Query
- func (qs *Query) HasClass(target string) bool
- func (qs *Query) Html()
- func (qs *Query) Index()
- func (qs *Query) Is()
- func (qs *Query) Map(fn func(i int, node *html.Node))
- func (qs *Query) Next() *Query
- func (qs *Query) NextAll() *Query
- func (qs *Query) NextAllFilter(selector string) *Query
- func (qs *Query) NextUntil(selector string) *Query
- func (qs *Query) Parent() *Query
- func (qs *Query) ParentFilter(selector string) *Query
- func (qs *Query) Parents() *Query
- func (qs *Query) ParentsFilter(selector string) *Query
- func (qs *Query) ParentsUntil(selector string) *Query
- func (qs *Query) Prev() *Query
- func (qs *Query) PrevAll() *Query
- func (qs *Query) PrevAllFilter(selector string) *Query
- func (qs *Query) PrevUntil(selector string) *Query
- func (qs *Query) Remove() *Query
- func (qs *Query) RemoveAttr()
- func (qs *Query) RemoveClass() *Query
- func (qs *Query) Text() string
- func (qs *Query) ToggleClass()
- Bugs
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
Notes ¶
Bugs ¶
renders duplicate html
Source Files
Directories