Version: v0.0.0-...-da42865 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Feb 25, 2017 License: MIT Imports: 11 Imported by: 261




This section is empty.


This section is empty.


This section is empty.


type CrawlConfig

type CrawlConfig struct {
	Onion         string         `json:"onion"`
	Base          string         `json:"base"`
	Exclude       []string       `json:"exclude"`
	Relationships []Relationship `json:"relationships"`

CrawlConfig defines user-specified options to tweak the current crawl.

func LoadCrawlConfig

func LoadCrawlConfig(filename string) (CrawlConfig, error)

LoadCrawlConfig creates a CrawlConfig object by loading a given filename.

func (*CrawlConfig) GetRelationship

func (cc *CrawlConfig) GetRelationship(name string) (Relationship, error)

GetRelationship provides a Relationship by its name.

type ExtraRelationship

type ExtraRelationship struct {
	Name   string `json:"name"`
	Regex  string `json:"regex"`
	Rollup bool   `json:"rollup"`

ExtraRelationship defines additional information which can be extracted after an initial relationship is detected.

type OnionScanConfig

type OnionScanConfig struct {
	TorProxyAddress string
	Depth           int
	Fingerprint     bool
	Timeout         time.Duration
	Verbose         bool
	Database        *crawldb.CrawlDB
	RescanDuration  time.Duration
	Scans           []string
	CrawlConfigs    map[string]CrawlConfig
	Cookies         []*http.Cookie

OnionScanConfig defines options to tweak the overall OnionScan system.

func Configure

func Configure(torProxyAddress string, directoryDepth int, fingerprint bool, timeout int, database string, scans []string, crawlconfigdir string, cookie string, verbose bool) *OnionScanConfig

Configure creates a new OnionScanConfig object with a set of options. FIXME: We can make this a decorate and make it much nicer.

func (*OnionScanConfig) LogError

func (os *OnionScanConfig) LogError(err error)

LogError logs an error message to the log, always.

func (*OnionScanConfig) LogInfo

func (os *OnionScanConfig) LogInfo(message string)

LogInfo logs an informational message to the log, assuming that the log level is set low enough.

type Relationship

type Relationship struct {
	Name                   string              `json:"name"`
	TriggerIdentifierRegex string              `json:"triggeridentifierregex"`
	ExtraRelationships     []ExtraRelationship `json:"extrarelationships"`

Relationship defines a section of a page that can be extract to provide a unique identifier relationship.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL