resource

Documentation

Overview

Author: Aosen Data: 2016-01-14 QQ: 316052486 Desc: 管理爬虫资源, 一个爬虫对应一个协程

Index

• func CreateTask(method func(*robot.Request), req *robot.Request) task
• type SpidersPool
  • func NewSpidersPool(total int, callback func()) *SpidersPool
  • func (self *SpidersPool) AddTask(method func(req *robot.Request), req *robot.Request)
  • func (self *SpidersPool) Free()
  • func (self *SpidersPool) Has() int
  • func (self *SpidersPool) Start()

Functions

func CreateTask

func CreateTask(method func(*robot.Request), req *robot.Request) task

Types

type SpidersPool

type SpidersPool struct {
	// contains filtered or unexported fields
}

func NewSpidersPool

func NewSpidersPool(total int, callback func()) *SpidersPool

新建爬虫池

func (*SpidersPool) AddTask

func (self *SpidersPool) AddTask(method func(req *robot.Request), req *robot.Request)

添加任务

func (*SpidersPool) Free

func (self *SpidersPool) Free()

释放爬虫池

func (*SpidersPool) Has

func (self *SpidersPool) Has() int

获取当前任务数

func (*SpidersPool) Start

func (self *SpidersPool) Start()

爬虫池开始接任务