Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var ( // HostsOfBatchHTTPSScorer contains 10 host with https scheme which are often crawled. HostsOfBatchHTTPSScorer = []string{ "https://www.liepin.com", "https://www.lagou.com", "https://www.zhipin.com", "https://nj.lianjia.com", "https://nanjing.anjuke.com", "https://nj.58.com", "https://www.ctrip.com", "https://www.qunar.com", "https://music.163.com", "https://maoyan.com", } )
Functions ¶
This section is empty.
Types ¶
type BatchHTTPSScorer ¶
type BatchHTTPSScorer struct {
// contains filtered or unexported fields
}
BatchHTTPSScorer tryRequest visiting a batch of HTTPS websites and grade the proxy by response time.
type Scorer ¶
Scorer is the interface used to score a proxy.
func NewBatchHTTPSScorer ¶
NewBatchHTTPSScorer returns a new scorer, the hosts can't be empty or bigger than maximum score. The timeout is calculated by the length of hosts. If response time smaller than timeout/2, score increments by (timeout/2 - RT) else score decrements by (RT - timeout/2). So, if all host tryRequest failed, the proxy score will be reduced to 0
Click to show internal directories.
Click to hide internal directories.