Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetURL ¶
GetURL sends all links found in page back http.Client misbehaves and might create too many goroutines
Types ¶
type BasicBodyParser ¶
type BasicBodyParser struct {
MaxLinks int
}
BasicBodyParser basic implementation for a href
type BodyParser ¶
BodyParser interface for new ways to parse body
type NewRequestHandler ¶
NewRequestHandler Parses and sends requests to Crawler
func (NewRequestHandler) Post ¶
func (nrh NewRequestHandler) Post(w http.ResponseWriter, r *http.Request)
Post Parses Post and sends to Crawler queue
Click to show internal directories.
Click to hide internal directories.