Imports github.com/PuerkitoBio/goquery github.com/garyburd/redigo/redis github.com/temoto/robotstxt-go Standard library imports errors fmt io io/ioutil log math net/http net/url strings time