browser

package
v1.12.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 16, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package browser is octo's owned, in-binary browser automation backend. It drives a real Chrome over the Chrome DevTools Protocol (CDP) — websocket + JSON — with no Node and no external proxy. CDP is the only native surface; the Chrome itself is the sole runtime dependency.

cdp.go holds the transport: a minimal JSON-RPC client over a single websocket, multiplexing commands and events across flattened target sessions. Higher-level actions (navigate, click, …) live in browser.go.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ChromeAvailable 

func ChromeAvailable(execPath string) bool

ChromeAvailable reports whether a Chrome executable can be located, so the browser tool is only advertised on setups where it could work.

func MarshalSkill 

func MarshalSkill(s Skill) ([]byte, error)

MarshalSkill renders the skill to YAML.

func MissingRequiredParams added in v1.10.0 

func MissingRequiredParams(skill *Skill, params map[string]string) []string

MissingRequiredParams reports which {{name}} placeholders in skill's steps remain unresolved after overlaying params on the skill's declared defaults — the same check ReplaySkill performs internally before running any step, exposed so a caller can surface a clear error to the model (which then decides whether to re-invoke with values or ask the user) instead of letting ReplaySkill fail outright.

func Replay 

func Replay(ctx context.Context, page *Page, events []RecordedEvent) error

Replay re-executes recorded events directly (no skill file, no params, no healer) — a convenience over the skill path for raw record→replay.

func SaveSkill 

func SaveSkill(path string, s Skill) error

SaveSkill writes a skill to a YAML file.

Types

type Browser 

type Browser struct {
	// contains filtered or unexported fields
}

Browser is a CDP connection to a Chrome instance, optionally one this process launched.

func Connect 

func Connect(ctx context.Context, wsURL string) (*Browser, error)

Connect attaches to an already-running Chrome via its browser-level CDP websocket URL (e.g. one the user started with --remote-debugging-port so their logged-in session is reused).

func ConnectByPort 

func ConnectByPort(ctx context.Context, port int) (*Browser, error)

ConnectByPort attaches to a Chrome already running with --remote-debugging-port=<port>, resolving the browser-level CDP endpoint from the debug HTTP server. This is how a user's existing, logged-in Chrome is reused.

func ConnectViaProfile 

func ConnectViaProfile(ctx context.Context, userDataDir string) (*Browser, error)

ConnectViaProfile attaches to a Chrome already running with remote debugging by reading its profile's DevToolsActivePort — reusing the user's logged-in session without relaunching, and without the /json HTTP endpoint.

func DiscoverRunningChrome 

func DiscoverRunningChrome(ctx context.Context) (*Browser, error)

DiscoverRunningChrome attaches to the first default-profile Chrome found running with remote debugging.

func Launch 

func Launch(ctx context.Context, opts LaunchOptions) (*Browser, error)

Launch starts a Chrome and connects to it.

func (*Browser) AttachPage 

func (b *Browser) AttachPage(ctx context.Context, targetID string) (*Page, error)

AttachPage attaches to an existing page target by id (e.g. the user's already- open, logged-in tab discovered via Pages).

func (*Browser) CaptureDownload 

func (b *Browser) CaptureDownload(ctx context.Context, dir string, trigger func() error) (string, error)

CaptureDownload directs downloads to dir, runs trigger (which should click the thing that starts a download), and waits for the download to finish — returning the saved file's path.

This is required for systems where the file is produced client-side: the raw HTTP response is not the file, so the only way to obtain it is to let the page generate it and capture what lands on disk.

func (*Browser) ClickFollow 

func (b *Browser) ClickFollow(ctx context.Context, page *Page, target string) (*Page, error)

ClickFollow clicks target on page and, if the click opened a new tab (a target=_blank link or window.open — common on SPAs whose results open in a new tab, e.g. Zhihu hot items), switches to it. Returns the page to continue on: the new tab when one opened, otherwise the same page. Without this a click that spawns a tab looks like it "did nothing" because the original page is unchanged.

func (*Browser) Close 

func (b *Browser) Close() error

Close tears down the connection and, if this process launched Chrome, the Chrome process too.

func (*Browser) ClosePage 

func (b *Browser) ClosePage(ctx context.Context, targetID string) error

ClosePage closes a page target by id.

func (*Browser) NewPage 

func (b *Browser) NewPage(ctx context.Context, url string) (*Page, error)

NewPage opens a fresh tab at url and attaches to it.

func (*Browser) OwnsProcess 

func (b *Browser) OwnsProcess() bool

OwnsProcess reports whether this process launched the Chrome (true) or merely attached to an already-running one (false). Callers use it to decide whether teardown may kill the browser or must only close the tabs it opened.

func (*Browser) Pages 

func (b *Browser) Pages(ctx context.Context) ([]TargetInfo, error)

Pages lists the open page targets — used to attach to an existing logged-in tab rather than opening a fresh one. 

type Cookie struct {
	Name     string  `json:"name"`
	Value    string  `json:"value"`
	Domain   string  `json:"domain"`
	Path     string  `json:"path"`
	Expires  float64 `json:"expires"`
	HTTPOnly bool    `json:"httpOnly"`
	Secure   bool    `json:"secure"`
	SameSite string  `json:"sameSite,omitempty"`
}

Cookie is one browser cookie, including HttpOnly ones JS can't read via document.cookie — which is the point: a logged-in session token usually is HttpOnly, so reuse/extraction needs CDP, not Eval.

type DialError added in v1.12.7 

type DialError struct {
	URL        string
	StatusCode int
	Err        error
}

DialError reports a failed WebSocket dial to a CDP endpoint, carrying the HTTP response status when the server rejected the upgrade (e.g., Chrome's remote- debugging origin check returns 403). Callers can inspect StatusCode to give actionable setup guidance.

func (*DialError) Error added in v1.12.7 

func (e *DialError) Error() string

func (*DialError) IsForbidden added in v1.12.7 

func (e *DialError) IsForbidden() bool

IsForbidden reports whether the dial was rejected by Chrome's remote debugging authorization / origin check (HTTP 403). This typically means the user needs to approve the Chrome authorization prompt or reconfigure remote debugging.

func (*DialError) Unwrap added in v1.12.7 

func (e *DialError) Unwrap() error

type DigestElement 

type DigestElement struct {
	Text     string `json:"text"`
	Selector string `json:"selector"`
}

DigestElement is one interactive element: its visible text and a generated selector. Used to give a healer (or live loop) a textual view of the page so it can repair a drifted selector by intent.

func InteractiveDigest 

func InteractiveDigest(ctx context.Context, page *Page, frame string, max int) ([]DigestElement, error)

InteractiveDigest lists interactive elements (with generated selectors) in the document or a same-origin iframe, capped to max. Text-only, so it feeds any model — no vision needed when the DOM/AX is reachable.

type Healer 

type Healer func(ctx context.Context, page *Page, step *Step, cause error) error

Healer is called when a step fails. It may inspect the page and mutate *step to repair it (e.g. fix a drifted selector); returning nil means "retry now". A non-nil return aborts replay. Provided by the caller (the tool layer wires an LLM-backed healer); the engine itself stays LLM-free.

type LaunchOptions 

type LaunchOptions struct {
	// ExecPath overrides Chrome auto-detection.
	ExecPath string
	// UserDataDir is the Chrome profile directory. Empty launches a throwaway
	// temp profile (no login state) — fine for tests. Real workflows point this
	// at the user's profile so logged-in sessions are reused.
	UserDataDir string
	// Port is the remote-debugging port. 0 lets Chrome pick one, which is then
	// read back from the DevToolsActivePort file in the profile dir.
	Port int
	// Headless runs Chrome with --headless=new. Real interactive workflows want
	// this false so the user can watch and intervene.
	Headless bool
	// DownloadDir, when set, is where the page's downloads are directed.
	DownloadDir string
	// ExtraArgs are appended verbatim.
	ExtraArgs []string
}

LaunchOptions controls how a Chrome instance is started or located.

type Output added in v1.6.0 

type Output struct {
	Name string `yaml:"name"`
	Type string `yaml:"type,omitempty"` // file | file[] | string
}

Output is a value replay exposes to its caller — the handoff surface that lets a recording feed a downstream step. Steps bind a produced value to an Output by name via Step.Bind (a download binds the captured file path; an extract binds the evaluated string). Type controls aggregation: "file[]" collects every bound value in order; "file"/"string" (and anything else) take the last.

type Page 

type Page struct {
	// contains filtered or unexported fields
}

Page is one attached page target (a flattened CDP session).

func ReplaySkill 

func ReplaySkill(ctx context.Context, page *Page, skill *Skill, params map[string]string, opts ReplayOptions) (modified bool, finalPage *Page, outputs map[string]any, err error)

ReplaySkill runs a skill deterministically (no LLM), substituting params. Each step implicitly waits for its target to appear (handling slow loads) and checks any explicit Verify. On a step failure it calls the healer; if the healer repairs the step, replay continues and reports modified=true so the caller can write the corrected skill back. A click that opens a new tab swaps the active page for subsequent steps; finalPage is the page replay ended on (so the caller can keep its session pointed at the right tab). outputs holds the skill's declared Outputs as bound during replay (download paths, extracted values) — the handoff surface for composing this recording with later steps.

func (*Page) AXTree 

func (p *Page) AXTree(ctx context.Context) (json.RawMessage, error)

AXTree returns the full accessibility tree as raw CDP node JSON — the Tier-2 semantic layer used for target resolution and verification.

func (*Page) Back 

func (p *Page) Back(ctx context.Context) error

Back navigates one entry back in history.

func (*Page) Click 

func (p *Page) Click(ctx context.Context, selector string) error

Click performs a trusted browser-level click at the element's center. Trusted input (vs a JS .click()) counts as a real user gesture, which some flows — file dialogs, download buttons — require, and is less detectable.

func (*Page) Cookies 

func (p *Page) Cookies(ctx context.Context) ([]Cookie, error)

Cookies returns the cookies the current page can see (its URL + frames), including HttpOnly. Network.enable is idempotent and a no-op once on.

func (*Page) DismissOverlay added in v1.6.0 

func (p *Page) DismissOverlay(ctx context.Context) (bool, error)

DismissOverlay makes one deterministic attempt to clear a blocking overlay — a cookie/consent banner or a modal whose backdrop intercepts clicks, the most common non-selector reason a replay step suddenly fails. It clicks the first visible control whose trimmed text or aria-label matches a small allowlist of dismiss/accept affordances (never anything else, so it can't perform an unintended destructive action), and reports whether it clicked something. No LLM involved; safe to call speculatively before falling back to the healer.

func (*Page) Eval 

func (p *Page) Eval(ctx context.Context, expr string, out any) error

Eval runs a JS expression in the page and unmarshals its return value into out (pass nil to ignore the value). The expression may be a Promise.

func (*Page) Hover 

func (p *Page) Hover(ctx context.Context, selector string) error

Hover moves the pointer over an element with a real (trusted) mouse move. Synthetic JS events can't trigger CSS :hover reveals — only a genuine pointer move does, which is what this dispatches.

func (*Page) Key 

func (p *Page) Key(ctx context.Context, combo string) error

Key presses a single key or modifier+key combo, e.g. "enter", "escape", "ctrl+a", "cmd+shift+s".

func (*Page) Navigate 

func (p *Page) Navigate(ctx context.Context, url string) error

Navigate loads url and waits for the page load event. When the page is still on the initial about:blank (the tab octo opened), it replaces that history entry instead of pushing one — otherwise a later Back() lands the model back on the blank tab. Navigation between real pages keeps normal history.

func (*Page) Screenshot 

func (p *Page) Screenshot(ctx context.Context) ([]byte, error)

Screenshot captures the current viewport as PNG bytes.

func (*Page) Scroll 

func (p *Page) Scroll(ctx context.Context, selector string, dx, dy float64) error

Scroll dispatches a wheel event. If selector is empty it scrolls at the viewport origin.

func (*Page) SelectOption 

func (p *Page) SelectOption(ctx context.Context, selector, value string) error

SelectOption picks an <option> of a native <select> by its value, visible text, or label, then fires input+change so framework bindings react. Native select popups are browser-drawn (not DOM), so this is the reliable path.

func (*Page) TargetID 

func (p *Page) TargetID() string

TargetID returns the page's target id (for close/switch).

func (*Page) TypeText 

func (p *Page) TypeText(ctx context.Context, selector, text string) error

TypeText focuses the element matched by selector and types text into it.

func (*Page) Upload 

func (p *Page) Upload(ctx context.Context, selector string, files []string) error

Upload sets the files on a file <input> matched by selector, without the OS file-picker dialog (CDP DOM.setFileInputFiles). Paths should be absolute. Resolving the input by JS object (rather than a DOM nodeId) lets it pierce a same-origin iframe via the same frame convention as the other actions.

func (*Page) UploadViaChooser 

func (p *Page) UploadViaChooser(ctx context.Context, selector string, files []string) error

UploadViaChooser sets files by clicking the control that opens a file chooser (a button or a file input) and intercepting the chooser — handling the common case where there is no persistent <input type=file> to target directly. The real Klook SD upload works this way.

func (*Page) WaitFor 

func (p *Page) WaitFor(ctx context.Context, selector string, timeout time.Duration) error

WaitFor polls until a CSS selector matches an element, or the timeout. This is the verify primitive for "the download button appeared after search".

func (*Page) WaitForNetworkIdle added in v1.6.0 

func (p *Page) WaitForNetworkIdle(ctx context.Context, quiet, timeout time.Duration) error

WaitForNetworkIdle is a best-effort settle: it returns once no fetch/XHR has been in flight for `quiet`, or when `timeout` elapses — whichever comes first. It never reports a failure (real pages with polling/keep-alive connections may never go fully idle, and a settle helper must not turn that into a step error); only ctx cancellation returns an error. If the monitor isn't present it returns immediately.

type Param 

type Param struct {
	Name        string `yaml:"name"`
	Description string `yaml:"description,omitempty"`
	Default     string `yaml:"default,omitempty"`
}

Param is a replay-time input; {{name}} placeholders in step values/urls are substituted from it (falling back to Default).

type RecordedEvent 

type RecordedEvent struct {
	Type     string `json:"type"`     // click | change | upload | navigate
	Selector string `json:"selector"` // best-effort CSS selector within its document
	Frame    string `json:"frame"`    // iframe selector when the target is in a child frame
	Tag      string `json:"tag"`      // target tagName (SELECT/INPUT/…), so replay picks the right action
	Value    string `json:"value"`    // input/select value (change events)
	Text     string `json:"text"`     // element text, for context
	Field    string `json:"field"`    // input's placeholder/name/aria-label/id — names the auto-param
	Secret   bool   `json:"secret"`   // password input: don't persist the value as a param default
	URL      string `json:"url"`      // document URL at capture time
}

RecordedEvent is one captured user action during a demonstration. Each event carries enough to replay it: the action kind plus a best-effort selector (and the value for inputs). Frame is set when the element lives in a same-origin iframe, so replay can pierce it via the " >>> " convention.

type Recorder 

type Recorder struct {
	// contains filtered or unexported fields
}

Recorder captures a user's actions on a page by injecting a DOM listener that reports through a CDP binding. It records the user's real demonstration — it does not drive the page itself.

func NewRecorder 

func NewRecorder(page *Page) *Recorder

NewRecorder creates a recorder bound to a page.

func (*Recorder) Events 

func (r *Recorder) Events() []RecordedEvent

Events returns the captured actions so far.

func (*Recorder) Start 

func (r *Recorder) Start(ctx context.Context) error

Start begins capturing. The binding and listeners are installed on the live document and on every future document (so post-navigation pages and same-origin child frames are covered too).

func (*Recorder) Stop 

func (r *Recorder) Stop()

Stop ends capture (all subscriptions are dropped; the page-side listeners are harmless and go away on navigation/close).

type ReplayOptions 

type ReplayOptions struct {
	StepTimeout time.Duration
	Healer      Healer
	Browser     *Browser
	DownloadDir string
}

ReplayOptions tunes a replay. StepTimeout bounds the per-step wait for a target to appear (default 15s — generous for slow back-ends). Healer, when set, is consulted on a step failure. Browser, when set, lets a click follow a new tab it opens (target=_blank / window.open). DownloadDir is where download steps land their files (required only if the skill has download steps).

type Skill 

type Skill struct {
	Name        string   `yaml:"name"`
	Description string   `yaml:"description,omitempty"`
	Params      []Param  `yaml:"params,omitempty"`
	Outputs     []Output `yaml:"outputs,omitempty"`
	Steps       []Step   `yaml:"steps"`
}

Skill is a recorded browser workflow in its editable, replayable form. It serializes to YAML — human-readable, hand-editable, git-versionable — which is the "editable steps" surface. Replay reads it back; self-heal writes it back.

func CompileSkill 

func CompileSkill(name, description, startURL string, events []RecordedEvent) Skill

CompileSkill turns a recording into an editable Skill. It seeds a leading navigate from the start URL (when it's a real page, not the about:blank tab octo opens) and then walks the captured events in order — including the top-level navigations the recorder captured, so a multi-page demonstration replays from the right pages.

func GenerateSkill 

func GenerateSkill(ctx context.Context, name, startURL string, events []RecordedEvent, gen SkillGenerator) Skill

GenerateSkill turns a recording into a skill. The deterministic CompileSkill is always the baseline (its selectors are ground truth). When gen is set, the LLM refines that baseline — dropping detours/retries, parameterizing variable inputs, labeling — but is constrained to the captured selectors; any output that fails to parse or invents a selector falls back to the baseline. So the LLM only ever cleans up real events, never hallucinates targets.

func ListSkills added in v1.6.0 

func ListSkills(dir string) []Skill

ListSkills reads every *.yaml recording in dir and returns them sorted by name. A missing dir yields nil; an unreadable or unparseable file (a half-written or hand-broken skill) is skipped rather than sinking the whole list — this feeds the system-prompt manifest, which must stay robust.

func LoadSkill 

func LoadSkill(path string) (Skill, error)

LoadSkill reads a skill from a YAML file.

func ParseSkill 

func ParseSkill(data []byte) (Skill, error)

ParseSkill parses a skill from YAML.

func (Skill) StepDigest added in v1.6.1 

func (s Skill) StepDigest() string

StepDigest renders a compact one-line summary of what a replay does, e.g. `navigate www.zhihu.com → click "热榜" → click "日元跌破 1 美元兑 162 日元…"`. The skills manifest uses it when a recording has no description, so the model can still see the recording's full path — including its final step, which is what tells it the replay ends somewhere specific.

type SkillGenerator 

type SkillGenerator func(ctx context.Context, system, user string) (string, error)

SkillGenerator asks an LLM to refine a recording into a clean skill. It is a plain string→string call so this package needn't import the agent/provider layers; the app wires a sender-backed implementation.

type Step 

type Step struct {
	Action   string `yaml:"action"` // navigate | click | type | select | upload | wait | download | extract
	URL      string `yaml:"url,omitempty"`
	Frame    string `yaml:"frame,omitempty"`
	Selector string `yaml:"selector,omitempty"`
	Value    string `yaml:"value,omitempty"`
	Label    string `yaml:"label,omitempty"`
	// Hint is a form field's accessible name (placeholder/name/aria-label/id or
	// its <label> text). It's the deterministic fallback for type/select/upload:
	// when the positional Selector drifts, replay re-locates the field by Hint
	// before giving up to the healer — the field-input analogue of Label steering
	// a click by visible text.
	Hint      string  `yaml:"hint,omitempty"`
	Bind      string  `yaml:"bind,omitempty"`       // download/extract: write the produced value into this named Output
	JS        string  `yaml:"js,omitempty"`         // extract: expression whose value is bound
	Network   bool    `yaml:"network,omitempty"`    // wait: settle for network (fetch/XHR) idle instead of a fixed delay
	TimeoutMS int     `yaml:"timeout_ms,omitempty"` // wait: fixed delay (ms) when no selector; also caps the network-idle settle
	Verify    *Verify `yaml:"verify,omitempty"`
}

Step is one action. Selector is within its document; Frame (a same-origin iframe selector) scopes it via the " >>> " convention. Label is a human note; replay ignores it.

type TargetInfo 

type TargetInfo struct {
	TargetID string `json:"targetId"`
	Type     string `json:"type"`
	Title    string `json:"title"`
	URL      string `json:"url"`
	// OpenerID is the target that spawned this one (window.open / a
	// target=_blank click), when Chrome reports it. Used by ClickFollow to
	// confirm a newly-seen tab was actually opened by the click it's
	// following, not an unrelated tab that happened to appear in the same
	// window (the user's own browsing, a notification/extension popup).
	OpenerID string `json:"openerId,omitempty"`
}

TargetInfo describes an open browser target (tab).

type Verify 

type Verify struct {
	Exists string `yaml:"exists,omitempty"`
	Text   string `yaml:"text,omitempty"`
	URL    string `yaml:"url,omitempty"`
}

Verify is an optional post-step assertion. Exists waits for a selector; Text waits for body text to contain a string; URL waits for location.href to contain a substring (auto-set to the destination host on navigate steps, to catch a redirect to a different host — a login/error page — whose DOM often mirrors the target). Empty means the implicit check only (the step's own target became present).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.