simplebrowser

README

Simple Browser

This provides a simple interface to get the HTML of a page and/or take a screenshot after javascript runs on the page.

This library uses chromedp

Usage

var html string
_ := NewPageRequest("https://google.com").WithHTMLGet(&html).Do(context.Background())
fmt.Println(html)

All modifiers

WithCookies([]http.Cookie)
WithHeaders(network.Headers)
WithProxy(string)
WithWaitTime(time.Duration)
WithActions(...chromedp.Action)
WithHTMLGet(*string)
WithScreenshotGet(*[]byte)
Do(context.Context)

Documentation

Overview

Package simplebrowser makes it easy to get the HTML of a page after javascript runs

Index

• Constants
• type PageRequest
  • func NewPageRequest(URL string) *PageRequest
  • func (p *PageRequest) Do(ctx context.Context) error
  • func (p *PageRequest) WithCookies(cookies []http.Cookie) *PageRequest
  • func (p *PageRequest) WithHTMLGet(html *string) *PageRequest
  • func (p *PageRequest) WithHeaders(headers network.Headers) *PageRequest
  • func (p *PageRequest) WithPostActions(actions ...chromedp.Action) *PageRequest
  • func (p *PageRequest) WithPreActions(actions ...chromedp.Action) *PageRequest
  • func (p *PageRequest) WithProxy(proxy string) *PageRequest
  • func (p *PageRequest) WithScreenSize(width, height int) *PageRequest
  • func (p *PageRequest) WithScreenshotGet(pngScreenshot *[]byte) *PageRequest
  • func (p *PageRequest) WithWaitTime(waitTime time.Duration) *PageRequest

Examples

• PageRequest
• PageRequest (WithContent)

Constants

const (
	DefaultWaitTime = time.Second * 2
)

Types

type PageRequest

type PageRequest struct {
	// contains filtered or unexported fields
}

PageRequest A request object to get a page with optional cookies, headers, and proxy

Example

var html string
_ = NewPageRequest("https://google.com").WithHTMLGet(&html).Do(context.Background())
fmt.Println(html)

Example (WithContent)

// Create testing server
var HTMLExample = `<html>
<script>
    function changeContent() {
        document.getElementById("content").innerHTML = "test";
    }
    window.onload = changeContent;
</script>

<body>
    <div id="content">the content</div>
</body>

</html>`
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	w.Write([]byte(HTMLExample))
}))
defer ts.Close()

// Get page and wait for javascript to change div
var html string
err := NewPageRequest(ts.URL).WithWaitTime(time.Second).WithHTMLGet(&html).Do(context.Background())
if err != nil {
	return
}

fmt.Println(strings.Index(html, `<div id="content">test</div>`))

Output:
185

func NewPageRequest

func NewPageRequest(URL string) *PageRequest

NewPageRequest returns a basic PageRequest with default WaitTime

func (*PageRequest) Do

func (p *PageRequest) Do(ctx context.Context) error

Do Perform the actual PageRequest

func (*PageRequest) WithCookies

func (p *PageRequest) WithCookies(cookies []http.Cookie) *PageRequest

WithCookies Add cookies to PageRequest

func (*PageRequest) WithHTMLGet

func (p *PageRequest) WithHTMLGet(html *string) *PageRequest

WithHTMLGet Will place the html of the page in the string after the request is made

func (*PageRequest) WithHeaders

func (p *PageRequest) WithHeaders(headers network.Headers) *PageRequest

WithHeaders Add headers to PageRequest

func (*PageRequest) WithPostActions

func (p *PageRequest) WithPostActions(actions ...chromedp.Action) *PageRequest

WithPostActions Add actions after navigating to the page and after the sleep time

func (*PageRequest) WithPreActions

func (p *PageRequest) WithPreActions(actions ...chromedp.Action) *PageRequest

WithPreActions Add actions for before navigating to the page

func (*PageRequest) WithProxy

func (p *PageRequest) WithProxy(proxy string) *PageRequest

WithProxy Add Proxy to PageRequest. A proxy can be a string like socks4://ip:port

func (*PageRequest) WithScreenSize

func (p *PageRequest) WithScreenSize(width, height int) *PageRequest

WithScreenSize Sets the screen size of the window

func (*PageRequest) WithScreenshotGet

func (p *PageRequest) WithScreenshotGet(pngScreenshot *[]byte) *PageRequest

WithScreenshotGet Will get a screenshot of the page after the request is made and load it in to the []byte pointer in png format.

func (*PageRequest) WithWaitTime

func (p *PageRequest) WithWaitTime(waitTime time.Duration) *PageRequest

WithWaitTime Add specific wait time to PageRequest, WaitTime is the time it will wait for the page to load before performing further actions