mongo

package

v0.0.0-...-68eaa9b Latest Latest Go to latest Published: Jul 26, 2024 License: MIT Imports: 8 Imported by: 0

Details

This section is empty.

This section is empty.

This section is empty.

type CookiesDocument struct {
	Host    string `json:"host"`
	Cookies string `json:"cookies"`
}

type Storage struct {
	Database string
	URI      string
	// contains filtered or unexported fields
}

Storage implements a MongoDB storage backend for colly

func (s *Storage) Cookies(u *url.URL) string

Cookies implements colly/storage.Cookies()

func (s *Storage) Init() error

Init initializes the MongoDB storage

func (s *Storage) IsVisited(requestID uint64) (bool, error)

IsVisited implements colly/storage.IsVisited()

func (s *Storage) SetCookies(u *url.URL, cookies string)

SetCookies implements colly/storage.SetCookies()

func (s *Storage) Visited(requestID uint64) error

Visited implements colly/storage.VisitedDocument()

type VisitedDocument struct {
	RequestID string `json:"requestID"`
	Visited   bool   `json:"visited"`
}

VisitedDocument represents the document we want to insert into MongoDB when we have visited a page.