mapper

Documentation

Overview

Package mapper contains the data structures for a site map, a simple directed graph, and the tooling to create one by crawling a website.

Index

• type SiteMap
  • func NewSiteMap(startPage string, workerCount uint) (*SiteMap, error)
  • func (sm *SiteMap) MarshalJSON() ([]byte, error)
  • func (sm *SiteMap) ServeHTTP(w http.ResponseWriter, r *http.Request)
  • func (sm *SiteMap) Start() error

Types

type SiteMap

type SiteMap struct {
	URL *url.URL
	// contains filtered or unexported fields
}

SiteMap is the data structure in which a mapping of a website is built.

func NewSiteMap

func NewSiteMap(startPage string, workerCount uint) (*SiteMap, error)

NewSiteMap returns a SiteMap initialized with the starting URL, path and the number of workers used when crawling the site. It also sets up signal handling which stops crawling for SIGTERM or SIGINT.

func (*SiteMap) MarshalJSON

func (sm *SiteMap) MarshalJSON() ([]byte, error)

MarshalJSON outputs the JSON representaion of sm needed for use by sigmajs to display a site map. It implements the json.Marshaller interface.

func (*SiteMap) ServeHTTP

func (sm *SiteMap) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implments the http.Handler interface responding with sm marshaled as JSON.

func (*SiteMap) Start

func (sm *SiteMap) Start() error

Start begins crawling a website with the starting URL using the assigned number of workers, exiting when the process is completed or when a signal is received on the SiteMap shutdown channel.