spider

Documentation

Overview

Package spider is the web crawler at the core of the project. It deals with identifying potential mailing-list or registration forms that take an email field.

Index

• func Crawl(domains []string, w *csv.Writer) error
• func Leak(contacts []Contact) error
• func MassSign(workers int, target, contact *os.File)
• func SignUp(f Form, c Contact) error
• type Contact
  • func Unpack(arg []string) Contact
• type Form

Functions

func Crawl

func Crawl(domains []string, w *csv.Writer) error

Crawl traverses a pre-defined list of malicious websites and attempts to identify URLs containing Form input for email contact inFormation such as mailing lists or registrations. This is expected to be a time-consuming process, so results are saved and POST-related functionality is its own function. Failure is expected to occur frequently, due to rate limiting or dead links, so errors are passed over.

spider.Crawl([]string{"google.com","yahoo.com"})

func Leak

func Leak(contacts []Contact) error

Leak posts a contact to locations on the internet likely to be scraped by others. Failure is expected to occur frequently, so errors are logged and then passed over. This is the low-hanging fruit of pirhana.

Posts to: 1. Craigslist.org 2. Pastebin.org

func MassSign

func MassSign(workers int, target, contact *os.File)

MassSign takes csv files created using Crawl

func SignUp

func SignUp(f Form, c Contact) error

SignUp submits POST data to contact registration Forms. Cookies or other contextual variables are not attempted. Secondary input validation, like captchas, are not attempted.

Types

type Contact

type Contact struct {
	First       string
	Last        string
	Street      string
	City        string
	State       string
	Email       string
	Zip         string
	HomePhone   string
	MobilePhone string
}

Contact is a collection of identifying information for an individual. Fields are self-explanatory. All fields, including 'numeric' fields are represented as strings for simplicity.

func Unpack

func Unpack(arg []string) Contact

Unpack assigns elements in a string slice to a new Contact

type Form

type Form struct {
	URL    *url.URL
	Action string
	Fields []string
}

Form is a container for HTML form inputs. Input tag names are saved in the Fields slice.