Directories

Path Synopsis
common
com_interfaces
Package com_interfaces contains some common interface of GO_SPIDER project.
Package com_interfaces contains some common interface of GO_SPIDER project.
config
Package config provides for parse config file.
Package config provides for parse config file.
etc_config
Package etc_config implements config initialization of one spider.
Package etc_config implements config initialization of one spider.
mlog
Package mlog implements log operations.
Package mlog implements log operations.
page
Package page contains result catched by Downloader.
Package page contains result catched by Downloader.
page_items
Package page_items contains parsed result by PageProcesser.
Package page_items contains parsed result by PageProcesser.
request
Package request implements request entity contains url and other relevant informaion.
Package request implements request entity contains url and other relevant informaion.
resource_manage
Package resource_manage implements a resource management.
Package resource_manage implements a resource management.
util
Package util contains some common functions of GO_SPIDER project.
Package util contains some common functions of GO_SPIDER project.
Package downloader is the main module of GO_SPIDER for download page.
Package downloader is the main module of GO_SPIDER for download page.
Package pipeline is the persistent and offline process part of crawler.
Package pipeline is the persistent and offline process part of crawler.
The package is useless
The package is useless
craw master module
craw master module