godoll

README

Godoll: Go Port of Pydoll

Evasion-first web automation framework extending go-rod with anti-bot detection features.

Status: ✅ Production Ready (90% readiness) - Implementation Status

Godoll successfully ports all core features from Pydoll to Go with 97% test coverage and comprehensive documentation.

Features

• Human-like Interactions - Typing with typos, physics-based scrolling, bezier curve mouse movement
• Retry Logic - Production-ready retry decorator with exponential backoff
• Network Control - Request monitoring, interception, and blocking
• Enhanced Finding - Intuitive element selection with XPath, text matching, and wait functions
• Browser Preferences - Anti-bot evasion with stealth presets
• Remote Connection - Connect to remote Chrome/Chromium instances

Installation

go get github.com/agenthands/godoll

Quick Start

package main

import (
    "github.com/go-rod/rod"
    "github.com/agenthands/godoll/find"
    "github.com/agenthands/godoll/humanize"
)

func main() {
    browser := rod.New().MustConnect()
    defer browser.MustClose()

    page, _ := browser.Page(&rod.PageOptions{})
    page.MustNavigate("https://www.google.com")

    // Find element
    finder := find.New(page)
    searchBox, _ := finder.Find(find.Selector{
        TagName: "textarea",
        Name:    "q",
    })

    // Type with humanization
    humanize.TypeWithHumanize(searchBox, "godoll go")
    searchBox.MustPress("Enter")
}

Recent Updates

Critical Bug Fixes (January 2025):

• ✅ Cookie jar population - HybridClient now properly shares browser cookies with HTTP API calls
• ✅ Mouse movement CDP calls - Mouse cursor now actually moves via Chrome DevTools Protocol
• ✅ Scrolling physics math - Cubic bezier easing curve preserved for natural scrolling motion
• ✅ Production readiness - Improved from 65% to 90% with all critical bugs resolved

See IMPLEMENTATION_STATUS.md for complete feature parity details and test coverage report.

Retry Decorator

Production-ready retry logic with exponential backoff:

import "github.com/agenthands/godoll/retry"

err := retry.Fetch(
    func() error {
        // Your scraping logic here
        return scrapeProduct(url)
    },
    retry.WithMaxRetries(3),
    retry.WithExponentialBackoff(true),
    retry.WithDelay(time.Second),
    retry.WithOnRetry(func(attempt int, err error) {
        log.Printf("Attempt %d failed: %v", attempt, err)
    }),
)

Features:

• Configurable max retries
• Exponential backoff (delay * 2^attempt)
• Custom delay duration
• OnRetry callback for logging
• Exception filtering

See examples/retry/ for more examples.

Request Interception

Block, modify, or mock network requests in real-time:

import "github.com/agenthands/godoll/network"

interceptor := network.NewInterceptor(page)
interceptor.AddRule(network.InterceptRule{
    URLPattern: "*.jpg",
    Action:     network.Block,
})
interceptor.Enable()

page.MustNavigate("https://example.com")
// Images blocked automatically

Features:

• Block requests (ads, trackers, images)
• Modify headers before sending
• Mock responses for faster testing
• URL pattern matching
• Resource type filtering

See examples/interception/ for more examples.

Browser Preferences

Configure browser settings for anti-bot evasion:

import "github.com/agenthands/godoll/browser"

opts := &browser.BrowserOptions{}
stealthPrefs := opts.StealthPreset()

launcher := launcher.New().
    Set("prefs", stealthPrefs).
    MustLaunch()

browser := rod.New().ControlURL(launcher)

Stealth presets include:

• Disable notifications
• Block geolocation requests
• Disable password manager
• Set Do Not Track header
• Custom Accept-Language

See examples/preferences/ for more examples.

Mouse Movement

Human-like mouse movement with cubic bezier curves:

import "github.com/agenthands/godoll/humanize"

// Move to element with natural path
err := humanize.MoveTo(page, element)

// Click with human-like movement
err := humanize.ClickWithMouse(page, element)

// Hover with movement
err := humanize.Hover(element)

Features:

• Cubic bezier curve paths with random control points
• Natural speed variation
• Configurable timing
• Hesitation before clicks
• Configurable path deviation (0.0 = straight, 1.0 = high curves)

See examples/mouse/ for more examples.

Advanced Element Finding

Powerful element selection with XPath and wait functions:

import "github.com/agenthands/godoll/find"

finder := find.New(page)

// XPath support
element, _ := finder.Find(find.Selector{
    XPath: "//div[@id='content']//p[@class='description']",
})

// Wait for element
err := finder.WaitForElement(find.Selector{
    ID: "dynamic-content",
}, 10*time.Second)

// Wait until visible
err := finder.WaitUntilVisible(element, 5*time.Second)

// Check element state
state := find.NewElementState(element)
isVisible, _ := state.IsVisible()
isClickable, _ := state.IsClickable()

Features:

• CSS selectors
• XPath support
• Text matching
• Wait functions (element, text, visible, clickable)
• Element state checks

See find/ package documentation for more details.

Remote Browser Connection

Connect to remote Chrome/Chromium instances:

import "github.com/agenthands/godoll/browser"

// Start Chrome with: chrome --remote-debugging-port=9222

browser, err := browser.ConnectToRemoteBrowser("http://localhost:9222")
if err != nil {
    log.Fatal(err)
}
defer browser.MustClose()

// Or connect and navigate in one call
browser, page, err := browser.ConnectToRemoteBrowserWithURL(
    "http://localhost:9222",
    "https://example.com",
)

Features:

• Connect to running Chrome instances
• Support for custom ports
• Automatic cleanup
• Connection error handling

See examples/remote/ for more examples.

Examples

See examples/ for more usage examples:

• Quickstart - Basic usage
• Retry decorator - Retry logic with exponential backoff
• Request interception - Block and modify network requests
• Browser preferences - Anti-bot evasion settings
• Mouse movement - Human-like bezier curve movement
• Remote connection - Connect to remote Chrome instances

The following features are demonstrated in the main package tests:

• Humanized typing with typos (see humanize/typing_test.go)
• Physics-based scrolling (see humanize/scroll_test.go)
• Advanced element finding (see find/ package)
• Hybrid automation (see hybrid/ package)
• Network monitoring (see network/ package)

Architecture

Godoll extends go-rod using the functional extension pattern:

• Accept go-rod types as parameters
• Apply enhancements via JS injection and event handlers
• Return go-rod types for seamless compatibility

Development

# Run tests
make test

# Run with coverage
make test-cover

# Lint
make lint

# Install development tools
make install-deps

Test-Driven Development

Godoll follows strict TDD:

1. Write tests FIRST
2. Run tests (they fail)
3. Implement minimal code to pass
4. Refactor
5. Commit

See CONTRIBUTING.md for details.

CLI

Godoll includes a command-line interface for browser automation:

# Install
go install github.com/agenthands/godoll/cmd/godoll@latest

# Interactive mode
godoll
godoll> page open https://example.com
godoll> element click "#button"

# Script mode
godoll page open https://example.com
godoll element click "#button" --output json

See cmd/godoll/README.md for complete CLI documentation.

License

MIT License - see LICENSE for details.

Related Projects

• Pydoll - Python version
• go-rod - Underlying DevTools Protocol library
• go-rod/stealth - Anti-bot detection inspiration