gospider

command module

v0.0.0-...-b9337ff Latest Latest Go to latest Published: Sep 30, 2021 License: MIT Imports: 2 Imported by: 0

Details

http://www.zhenai.com/zhenghun ---获取---> Item[city: ,cityUrl:,cityListPares:]

http://www.zhenai.com/zhenghun/ ---获取---> Item[user: ,userUrl:,cityPares:]

http://www.zhenai.com/zhenghun/ ---获取---> Item[username: ,userUrl:,userPares:]

----> 用户信息

队列里面从队列里面拿到一个parserResult进行解析每个parserResult都包含request,然后从result分别获取如下内容: request ---> url --- fetch - content -- > 解析器

城市用户表通过每个城市的页面,获取每个城市下的用户的url

并发版爬虫

There is no documentation for this package.

Path	Synopsis
chromedp
1.testcase001 command
2.testcase002 command
3.cookies command
3.cookies4 command Command cookie is a chromedp example demonstrating how to set a HTTP cookie on requests.	Command cookie is a chromedp example demonstrating how to set a HTTP cookie on requests.
4.wps command
engine
es
fetcher
guitest
model
parser
pkg
httputils
regexptest
4.regexptest001 command
testcase
1.fetchcitylist command
2.fetchuserlist command
3.fetchuserinfo command
zhenai
parse