proxy

README

Proxy Management Documentation

This directory contains the proxy rotation and management functionality.

Proxy Manager (manager.go)

The proxy manager handles rotating through multiple proxy servers to avoid IP bans and distribute load.

Features

• Proxy Rotation: Automatically cycles through available proxies
• Health Monitoring: Tracks proxy availability and response times
• Failure Handling: Automatically disables failing proxies
• Load Balancing: Distributes requests across healthy proxies

Configuration

proxy:
  enabled: false
  rotation: true                           # Enable proxy rotation
  timeout: 30s                            # Proxy timeout
  
  # Static proxy list
  urls:
    - "http://proxy1:8080"
    - "http://proxy2:8080"
    - "http://proxy3:8080"
  
  # Authentication (if required)
  username: ""
  password: ""
  
  # Health check settings
  healthCheckInterval: 60s                 # How often to check proxy health
  maxFailures: 3                           # Failures before disabling proxy

Usage

// Create proxy manager
proxyManager := proxy.NewManager([]string{
    "http://proxy1:8080",
    "http://proxy2:8080",
}, 30*time.Second)

// Get next available proxy
proxyURL := proxyManager.GetNextProxy()

// Use proxy with HTTP client
transport := &http.Transport{
    Proxy: http.ProxyURL(proxyURL),
}
client := &http.Client{
    Transport: transport,
    Timeout:   30 * time.Second,
}

// Make request through proxy
resp, err := client.Get("https://example.com")

Proxy Types Supported

1. HTTP Proxies: Basic HTTP proxy support
2. SOCKS5 Proxies: Full SOCKS5 protocol support
3. Authenticated Proxies: Username/password authentication
4. Rotating Proxies: Automatic rotation through proxy list

Best Practices

1. Diverse Proxy Pool: Use proxies from different providers/locations
2. Health Monitoring: Regularly check proxy availability
3. Rate Limiting: Don't overwhelm individual proxies
4. Failover: Have backup proxies ready
5. Geographic Distribution: Use proxies from relevant regions

Security Considerations

• Use reputable proxy providers
• Avoid free/public proxies for sensitive operations
• Implement proper authentication
• Monitor for proxy hijacking
• Use HTTPS when possible

Documentation

Index

• type Manager
  • func NewManager(cfg config.ProxyConfig) (*Manager, error)
  • func (m *Manager) Close()
  • func (m *Manager) GetTransport() *http.Transport
  • func (m *Manager) RecordFailure(proxyURL string)
  • func (m *Manager) RecordSuccess(proxyURL string)
• type ProxyServer

Types

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager handles proxy rotation and health checking

func NewManager

func NewManager(cfg config.ProxyConfig) (*Manager, error)

NewManager creates a new proxy rotation manager

func (*Manager) Close

func (m *Manager) Close()

Close stops the refresh timer

func (*Manager) GetTransport

func (m *Manager) GetTransport() *http.Transport

GetTransport returns an http.Transport that uses proxies

func (*Manager) RecordFailure

func (m *Manager) RecordFailure(proxyURL string)

RecordFailure records a failed request through a proxy

func (*Manager) RecordSuccess

func (m *Manager) RecordSuccess(proxyURL string)

RecordSuccess records a successful request through a proxy

type ProxyServer

type ProxyServer struct {
	URL       string
	LastCheck time.Time
	Healthy   bool
	ErrorRate float64
	Failures  int
	Successes int
}

ProxyServer represents a proxy server with health status