headless

Documentation

Index

• func IsTransientHeadlessError(err error) bool
• type NetworkHeaderCapture
  • func NewNetworkHeaderCapture() *NetworkHeaderCapture
  • func (n *NetworkHeaderCapture) GetHeaders() map[string][]string
  • func (n *NetworkHeaderCapture) SetHeaders(headers map[string][]string)
• type PageMetadata
• type Requester
  • func NewRequester(timeout int, config *common.HeadlessRequestConfig) *Requester
  • func NewRequesterWithClient(client *cdp.Client, timeout int, minDOMStabalizeTime int) (*Requester, error)
  • func NewRequesterwithBrowser(timeout int, config *common.HeadlessRequestConfig) *Requester
  • func (b *Requester) InitializeBrowser(ctx context.Context) error
  • func (b *Requester) ResetOwnedBrowser(ctx context.Context, reason string)
  • func (b *Requester) SendRequest(ctx context.Context, config common.SendHttpRequestConfig) (common.HttpRequestResponse, error)
  • func (b *Requester) SendRequestWithMetadata(ctx context.Context, config common.SendHttpRequestConfig) (common.HttpRequestResponse, PageMetadata, error)
  • func (b *Requester) SendRequestWithScreenshot(ctx context.Context, config common.SendHttpRequestConfig) (common.HttpRequestResponse, []byte, PageMetadata, error)
  • func (b *Requester) SetProxyConfig(httpProxy, socksProxy string)
  • func (b *Requester) SetProxyConfigFromRequest(config common.SendHttpRequestConfig)

Functions

func IsTransientHeadlessError

func IsTransientHeadlessError(err error) bool

IsTransientHeadlessError returns true for browser/network failures worth retrying with a fresh page or, when owned by this requester, a fresh browser.

Types

type NetworkHeaderCapture

type NetworkHeaderCapture struct {
	Headers map[string][]string
	// contains filtered or unexported fields
}

NetworkHeaderCapture stores captured response headers

func NewNetworkHeaderCapture

func NewNetworkHeaderCapture() *NetworkHeaderCapture

NewNetworkHeaderCapture creates a new header capture instance

func (*NetworkHeaderCapture) GetHeaders

func (n *NetworkHeaderCapture) GetHeaders() map[string][]string

GetHeaders safely gets headers

func (*NetworkHeaderCapture) SetHeaders

func (n *NetworkHeaderCapture) SetHeaders(headers map[string][]string)

SetHeaders safely sets headers

type PageMetadata

type PageMetadata struct {
	HtmlTitle string
}

PageMetadata carries headless-capture metadata that is not part of the wire HttpResponse but is useful to downstream callers (e.g. the <title> tag).

type Requester

type Requester struct {
	Browser                    *rod.Browser
	PathToBrowser              *string
	TimeoutSeconds             int
	MinDOMStabalizeTimeSeconds int
	HttpProxy                  string
	SocksProxy                 string
	// contains filtered or unexported fields
}

Requester manages a headless browser instance and configuration for making requests.

func NewRequester

func NewRequester(timeout int, config *common.HeadlessRequestConfig) *Requester

NewRequester creates a new Requester with the given timeout and headless configuration.

func NewRequesterWithClient

func NewRequesterWithClient(client *cdp.Client, timeout int, minDOMStabalizeTime int) (*Requester, error)

NewRequesterWithClient creates a new Requester using an existing rod cdp.Client.

func NewRequesterwithBrowser

func NewRequesterwithBrowser(timeout int, config *common.HeadlessRequestConfig) *Requester

NewRequesterwithBrowser creates a new Requester with the given timeout and headless configuration.

func (*Requester) InitializeBrowser

func (b *Requester) InitializeBrowser(ctx context.Context) error

InitializeBrowser starts a headless browser instance and establishes connection. If a browser path is specified and exists, it uses that binary. Otherwise it falls back to rod's auto-download, which fetches a compatible Chromium to ~/.cache/rod/browser/.

func (*Requester) ResetOwnedBrowser

func (b *Requester) ResetOwnedBrowser(ctx context.Context, reason string)

ResetOwnedBrowser closes a browser launched by this requester so the next operation can start from a fresh browser process. Caller-owned browsers and externally supplied CDP clients are left untouched.

func (*Requester) SendRequest

func (b *Requester) SendRequest(ctx context.Context, config common.SendHttpRequestConfig) (common.HttpRequestResponse, error)

SendRequest navigates to a URL using the headless browser and captures the response

func (*Requester) SendRequestWithMetadata

func (b *Requester) SendRequestWithMetadata(ctx context.Context, config common.SendHttpRequestConfig) (common.HttpRequestResponse, PageMetadata, error)

SendRequestWithMetadata performs one headless navigation and returns page metadata without paying the screenshot capture cost.

func (*Requester) SendRequestWithScreenshot

func (b *Requester) SendRequestWithScreenshot(ctx context.Context, config common.SendHttpRequestConfig) (common.HttpRequestResponse, []byte, PageMetadata, error)

SendRequestWithScreenshot performs one headless navigation and captures both the HTTP response artifact and a screenshot from the same stabilized page. The returned PageMetadata carries fields harvested from the live DOM (e.g. <title>) that are not part of the HttpResponse contract.

func (*Requester) SetProxyConfig

func (b *Requester) SetProxyConfig(httpProxy, socksProxy string)

SetProxyConfig stores launch-scoped proxy configuration for browser instances created by this requester. HTTP takes precedence when both proxies are set.

func (*Requester) SetProxyConfigFromRequest

func (b *Requester) SetProxyConfigFromRequest(config common.SendHttpRequestConfig)

SetProxyConfigFromRequest stores proxy configuration from a request config.