Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ContainsCookie

func ContainsCookie(cookies []*http.Cookie, name string) bool

    ContainsCookie checks if a cookie name is represented in cookies

    func StringifyCookies

    func StringifyCookies(cookies []*http.Cookie) string

      StringifyCookies serializes list of http.Cookies to string

      func UnstringifyCookies

      func UnstringifyCookies(s string) []*http.Cookie

        UnstringifyCookies deserializes a cookie string to http.Cookies

        Types

        type InMemoryStorage

        type InMemoryStorage struct {
        	// contains filtered or unexported fields
        }

          InMemoryStorage is the default storage backend of colly. InMemoryStorage keeps cookies and visited urls in memory without persisting data on the disk.

          func (*InMemoryStorage) Close

          func (s *InMemoryStorage) Close() error

            Close implements Storage.Close()

            func (*InMemoryStorage) Cookies

            func (s *InMemoryStorage) Cookies(u *url.URL) string

              Cookies implements Storage.Cookies()

              func (*InMemoryStorage) Init

              func (s *InMemoryStorage) Init() error

                Init initializes InMemoryStorage

                func (*InMemoryStorage) IsVisited

                func (s *InMemoryStorage) IsVisited(requestID uint64) (bool, error)

                  IsVisited implements Storage.IsVisited()

                  func (*InMemoryStorage) SetCookies

                  func (s *InMemoryStorage) SetCookies(u *url.URL, cookies string)

                    SetCookies implements Storage.SetCookies()

                    func (*InMemoryStorage) Visited

                    func (s *InMemoryStorage) Visited(requestID uint64) error

                      Visited implements Storage.Visited()

                      type Storage

                      type Storage interface {
                      	// Init initializes the storage
                      	Init() error
                      	// Visited receives and stores a request ID that is visited by the Collector
                      	Visited(requestID uint64) error
                      	// IsVisited returns true if the request was visited before IsVisited
                      	// is called
                      	IsVisited(requestID uint64) (bool, error)
                      	// Cookies retrieves stored cookies for a given host
                      	Cookies(u *url.URL) string
                      	// SetCookies stores cookies for a given host
                      	SetCookies(u *url.URL, cookies string)
                      }

                        Storage is an interface which handles Collector's internal data, like visited urls and cookies. The default Storage of the Collector is the InMemoryStorage. Collector's storage can be changed by calling Collector.SetStorage() function.

                        Source Files