prox5

package module
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 25, 2022 License: MIT Imports: 23 Imported by: 0

README

Prox5

GoDoc Go Report Card IRC

SOCKS5/4/4a validating proxy pool + server

Prox5 is a golang library for managing, validating, and accessing thousands upon thousands of arbitrary SOCKS proxies.

Notably it features interface compatible dialer functions that dial out from different proxies for every connection, and a SOCKS5 server that utilizes those functions.

Initial Validation Sequence
  • TCP Dial to the endpoint
  • HTTPS GET request to a list of IP echo endpoints

Prox5 will then store the endpoint's outward appearing IP address and mark it as valid for use.

The Secret Sauce

What makes Prox5 special is largely the Mystery Dialer. This dialer satisfies the net.Dialer interface. Upon using the dialer to connect to and endpoint, Prox5:

  • Loads up a previously verified proxy
  • Attempts to make connection with the dial endpoint using said proxy
  • Upon failure, prox5:
    • repeats this process mid-dial
    • does not drop connection to the client
  • Once a proxy has been successfully used to connect to the target endpoint, prox5 passes the same net.Conn onto the client
Accessing Validated Proxies
  • Retrieve validated 4/4a/5 proxies as simple strings for generic use
  • Use one of the dialer functions with any golang code that calls for a net.Dialer
  • Spin up a SOCKS5 server that will then make rotating use of your validated proxies

The way you choose to use this lib is yours. The API is fairly extensive for you to be able to customize runtime configuration without having to do any surgery.

Things like the amount of validation workers that are concurrently operating, timeouts, and proxy re-use policies may be tuned in real-time. please read the docs.

This project is in development.

It "works" and has been used in "production", but still needs some love.

Please break it and let me know what broke.

See the docs and the example for more details.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Proxy 

type Proxy struct {
	// Endpoint is the address:port of the proxy that we connect to
	Endpoint string
	// ProxiedIP is the address that we end up having when making proxied requests through this proxy
	ProxiedIP string
	// Proto is the version/Protocol (currently SOCKS* only) of the proxy
	Proto atomic.Value
	// contains filtered or unexported fields
}

Proxy represents an individual proxy

func (*Proxy) GetProto 

func (sock *Proxy) GetProto() string

GetProto safely retrieves the protocol value of the Proxy.

func (*Proxy) String added in v0.6.0 

func (sock *Proxy) String() string

GetProto safely retrieves the protocol value of the Proxy.

func (*Proxy) UniqueKey 

func (sock *Proxy) UniqueKey() string

UniqueKey is an implementation of the Identity interface from Rate5. See: https://pkg.go.dev/github.com/yunginnanet/Rate5#Identity

type Statistics 

type Statistics struct {
	// Valid4 is the amount of SOCKS4 proxies validated
	Valid4 int
	// Valid4a is the amount of SOCKS4a proxies validated
	Valid4a int
	// Valid5 is the amount of SOCKS5 proxies validated
	Valid5 int
	// ValidHTTP is the amount of HTTP proxies validated
	ValidHTTP int
	// Dispensed is a simple ticker to keep track of proxies dispensed via our getters
	Dispensed int
	// Stale is the amount of proxies that failed our stale policy upon dispensing
	Stale int
	// Checked is the amount of proxies we've checked.
	Checked int
	// contains filtered or unexported fields
}

Statistics is used to encapsulate various swampy stats

func (*Statistics) GetUptime 

func (stats *Statistics) GetUptime() time.Duration

GetUptime returns the total lifetime duration of our pool.

type Swamp 

type Swamp struct {
	// ValidSocks5 is a constant stream of verified ValidSocks5 proxies
	ValidSocks5 chan *Proxy
	// ValidSocks4 is a constant stream of verified ValidSocks4 proxies
	ValidSocks4 chan *Proxy
	// ValidSocks4a is a constant stream of verified ValidSocks5 proxies
	ValidSocks4a chan *Proxy
	// ValidHTTP is a constant stream of verified ValidSocks5 proxies
	ValidHTTP chan *Proxy

	// Stats holds the Statistics for our swamp
	Stats *Statistics

	Status atomic.Value

	// Pending is a constant stream of proxy strings to be verified
	Pending chan *Proxy
	// contains filtered or unexported fields
}

Swamp represents a proxy pool

func NewDefaultSwamp 

func NewDefaultSwamp() *Swamp

NewDefaultSwamp returns a Swamp with basic options. After calling this you can use the various "setters" to change the options before calling Swamp.Start().

func (*Swamp) AddCheckEndpoints 

func (s *Swamp) AddCheckEndpoints(endpoints []string)

AddCheckEndpoints appends entries to the running list of whatismyip style endpoitns for validation. (must return only the WAN IP)

func (*Swamp) AddUserAgents 

func (s *Swamp) AddUserAgents(uagents []string)

AddUserAgents appends to the list of useragents we randomly choose from during proxied requests

func (*Swamp) ClearSOCKSList 

func (s *Swamp) ClearSOCKSList()

ClearSOCKSList clears the map of proxies that we have on record. Other operations (proxies that are still in buffered channels) will continue.

func (*Swamp) DebugChannel 

func (s *Swamp) DebugChannel() chan string

DebugChannel will return a channel which will receive debug messages once debug is enabled. This will alter the flow of debug messages, they will no longer print to console, they will be pushed into this channel. Make sure you pull from the channel eventually to avoid build up of blocked goroutines.

func (*Swamp) DebugEnabled 

func (s *Swamp) DebugEnabled() bool

DebugEnabled returns the current state of our debug switch.

func (*Swamp) Dial added in v0.6.0 

func (s *Swamp) Dial(network, addr string) (net.Conn, error)

Dial is a simple stub adapter to implement a net.Dialer.

func (*Swamp) DialContext 

func (s *Swamp) DialContext(ctx context.Context, network, addr string) (net.Conn, error)

DialContext is a simple stub adapter to implement a net.Dialer with context.

func (*Swamp) DialTimeout added in v0.6.0 

func (s *Swamp) DialTimeout(network, addr string, timeout time.Duration) (net.Conn, error)

DialTimeout is a simple stub adapter to implement a net.Dialer with a timeout.

func (*Swamp) DisableDebug 

func (s *Swamp) DisableDebug()

DisableDebug enables printing of verbose messages during operation. WARNING: if you are using a DebugChannel, you must read all of the messages in the channel's cache or this will block.

func (*Swamp) DisableDebugChannel 

func (s *Swamp) DisableDebugChannel()

DisableDebugChannel redirects debug messages back to the console. DisableProxyChannel does not disable debug, use DisableDebug().

func (*Swamp) DisableRecycling 

func (s *Swamp) DisableRecycling()

DisableRecycling disables recycling used proxies back into the pending channel for revalidation after dispensed.

func (*Swamp) EnableDebug 

func (s *Swamp) EnableDebug()

EnableDebug enables printing of verbose messages during operation

func (*Swamp) EnableRecycling 

func (s *Swamp) EnableRecycling()

EnableRecycling enables recycling used proxies back into the pending channel for revalidation after dispensed.

func (*Swamp) GetAnySOCKS 

func (s *Swamp) GetAnySOCKS() *Proxy

GetAnySOCKS retrieves any version SOCKS proxy as a Proxy type Will block if one is not available!

func (*Swamp) GetDialerBailout 

func (s *Swamp) GetDialerBailout() int

GetDialerBailout retrieves the dialer bailout policy. See SetDialerBailout for more info.

func (*Swamp) GetHTTPClient 

func (s *Swamp) GetHTTPClient() *http.Client

GetHTTPClient retrieves a pointer to an http.Client powered by MysteryDialer.

func (*Swamp) GetMaxWorkers 

func (s *Swamp) GetMaxWorkers() int

GetMaxWorkers returns maximum amount of workers that validate proxies concurrently. Note this is read-only during runtime.

func (*Swamp) GetRandomEndpoint 

func (s *Swamp) GetRandomEndpoint() string

GetRandomEndpoint returns a random whatismyip style endpoint from our Swamp's options

func (*Swamp) GetRecyclingStatus 

func (s *Swamp) GetRecyclingStatus() bool

GetRecyclingStatus retrieves the current recycling status, see EnableRecycling.

func (*Swamp) GetRemoveAfter 

func (s *Swamp) GetRemoveAfter() int

GetRemoveAfter retrieves the removeafter policy, the amount of times a recycled proxy is marked as bad until it is removed entirely.

  • returns -1 if recycling is disabled.

func (*Swamp) GetServerTimeout 

func (s *Swamp) GetServerTimeout() time.Duration

GetServerTimeout returns the current value of serverTimeout.

func (*Swamp) GetServerTimeoutStr 

func (s *Swamp) GetServerTimeoutStr() string

GetServerTimeoutStr returns the current value of serverTimeout (in seconds string).

func (*Swamp) GetStaleTime 

func (s *Swamp) GetStaleTime() time.Duration

GetStaleTime returns the duration of time after which a proxy will be considered "stale".

func (*Swamp) GetStatistics 

func (s *Swamp) GetStatistics() *Statistics

GetStatistics returns all current statistics. * This is a pointer, do not modify it!

func (*Swamp) GetTotalValidated 

func (s *Swamp) GetTotalValidated() int

GetTotalValidated retrieves our grand total validated proxy count.

func (*Swamp) GetValidationTimeout 

func (s *Swamp) GetValidationTimeout() time.Duration

GetValidationTimeout returns the current value of validationTimeout.

func (*Swamp) GetValidationTimeoutStr 

func (s *Swamp) GetValidationTimeoutStr() string

GetValidationTimeoutStr returns the current value of validationTimeout (in seconds string).

func (*Swamp) GetWorkers 

func (s *Swamp) GetWorkers() (maxWorkers, runningWorkers, idleWorkers int)

GetWorkers retrieves pond worker statistics:

  • return MaxWorkers, RunningWorkers, IdleWorkers

func (*Swamp) IsRunning 

func (s *Swamp) IsRunning() bool

IsRunning returns true if our background goroutines defined in daemons.go are currently operational

func (*Swamp) LoadMultiLineString 

func (s *Swamp) LoadMultiLineString(socks string) int

LoadMultiLineString loads a multiine string object with one (host:port) SOCKS proxy per line.

func (*Swamp) LoadProxyTXT 

func (s *Swamp) LoadProxyTXT(seedFile string) int

LoadProxyTXT loads proxies from a given seed file and feeds them to the mapBuilder to be later queued automatically for validation. Expects the following formats: * 127.0.0.1:1080 * 127.0.0.1:1080:user:pass * yeet.com:1080 * yeet.com:1080:user:pass * [fe80::2ef0:5dff:fe7f:c299]:1080 * [fe80::2ef0:5dff:fe7f:c299]:1080:user:pass

func (*Swamp) LoadSingleProxy 

func (s *Swamp) LoadSingleProxy(sock string) (ok bool)

LoadSingleProxy loads a SOCKS proxy into our map. Uses the format: 127.0.0.1:1080 (host:port).

func (*Swamp) MysteryDialer 

func (s *Swamp) MysteryDialer(ctx context.Context, network, addr string) (net.Conn, error)

MysteryDialer is a dialer function that will use a different proxy for every request.

func (*Swamp) Pause 

func (s *Swamp) Pause() error

Pause will cease the creation of any new proxy validation operations.

  • You will be able to start the proxy pool again with Swamp.Resume(), it will have the same Statistics, options, and ratelimits.
  • During pause you are still able to dispense proxies.
  • Options may be changed and proxy lists may be loaded when paused.
  • Pausing an already paused Swamp is a nonop.

func (*Swamp) RandomUserAgent 

func (s *Swamp) RandomUserAgent() string

RandomUserAgent retrieves a random user agent from our list in string form.

func (*Swamp) Resume 

func (s *Swamp) Resume() error

Resume will resume pause proxy pool operations, attempting to resume a running Swamp is returns an error.

func (*Swamp) RoundTrip 

func (s *Swamp) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip is Mr. WorldWide. Obviously. See: https://pkg.go.dev/net/http#RoundTripper

func (*Swamp) SetCheckEndpoints 

func (s *Swamp) SetCheckEndpoints(newendpoints []string)

SetCheckEndpoints replaces the running list of whatismyip style endpoitns for validation. (must return only the WAN IP)

func (*Swamp) SetDialerBailout 

func (s *Swamp) SetDialerBailout(dialattempts int)

SetDialerBailout sets the amount of times the MysteryDialer will dial out and fail before it bails out.

  • The dialer will attempt to redial a destination with a different proxy a specified amount of times before it gives up

func (*Swamp) SetMaxWorkers 

func (s *Swamp) SetMaxWorkers(num int)

SetMaxWorkers set the maximum workers for proxy checking and clears the current proxy map and worker pool jobs.

func (*Swamp) SetRemoveAfter 

func (s *Swamp) SetRemoveAfter(timesfailed int)

SetRemoveAfter sets the removeafter policy, the amount of times a recycled proxy is marked as bad before it is removed entirely.

  • Default is 5
  • To disable deleting entirely, set this value to -1
  • Only applies when recycling is enabled

func (*Swamp) SetServerTimeout 

func (s *Swamp) SetServerTimeout(timeout time.Duration)

SetServerTimeout sets the serverTimeout option. * serverTimeout defines the timeout for outgoing connections made with the MysteryDialer. * To disable timeout on outgoing MysteryDialer connections, set this to time.Duration(0).

func (*Swamp) SetStaleTime 

func (s *Swamp) SetStaleTime(newtime time.Duration)

SetStaleTime replaces the duration of time after which a proxy will be considered "stale". stale proxies will be skipped upon retrieval.

func (*Swamp) SetUserAgents 

func (s *Swamp) SetUserAgents(uagents []string)

SetUserAgents sets the list of useragents we randomly choose from during proxied requests

func (*Swamp) SetValidationTimeout 

func (s *Swamp) SetValidationTimeout(timeout time.Duration)

SetValidationTimeout sets the validationTimeout option.

func (*Swamp) Socks4Str 

func (s *Swamp) Socks4Str() string

Socks4Str gets a SOCKS4 proxy that we have fully verified. Will block if one is not available!

func (*Swamp) Socks4aStr 

func (s *Swamp) Socks4aStr() string

Socks4aStr gets a SOCKS4 proxy that we have fully verified. Will block if one is not available!

func (*Swamp) Socks5Str 

func (s *Swamp) Socks5Str() string

Socks5Str gets a SOCKS5 proxy that we have fully verified (dialed and then retrieved our IP address from a what-is-my-ip endpoint. Will block if one is not available!

func (*Swamp) Start 

func (s *Swamp) Start() error

Start starts our proxy pool operations. Trying to start a running Swamp will return an error.

func (*Swamp) StartSOCKS5Server 

func (s *Swamp) StartSOCKS5Server(listen, username, password string) error

StartSOCKS5Server starts our rotating proxy SOCKS5 server. listen is standard Go listen string, e.g: "127.0.0.1:1080". username and password are used for authenticatig to the SOCKS5 server.

type SwampStatus 

type SwampStatus uint32

SwampStatus represents the current state of our Swamp. 

const (
	// Running means the proxy pool is currently taking in proxys and validating them, and is available to dispense proxies.
	Running SwampStatus = iota
	// Paused means the proxy pool has been with Swamp.Pause() and may be resumed with Swamp.Resume()
	Paused
	// New means the proxy pool has never been started.
	New
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.