app

package
v0.0.0-...-87eabf6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 16, 2020 License: NCSA Imports: 19 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrNoSuchBucket = errors.NewError("no such bucket")

Functions

This section is empty.

Types

type CrawlOptions

type CrawlOptions struct {
	Start     int
	Stop      int
	MaxErrors int
}

type Crawler

type Crawler struct {
	// contains filtered or unexported fields
}

Crawler is a high-level representation of the Gab client and controls access to resources, managing some state. In particular, responses returned by the client will be written by the Crawler to permanent storage or elsewhere (depending on configuration).

func NewCrawler

func NewCrawler(configuration *api.Configuration) (*Crawler, error)

NewCrawler configures and returns a Crawler instance.

func (*Crawler) Groups

func (c *Crawler) Groups(options *CrawlOptions) error

Groups instructs the crawler to crawl groups in a manner consistent with the current configuration.

func (*Crawler) Init

func (c *Crawler) Init() error

func (*Crawler) Login

func (c *Crawler) Login(login *api.Login)

Login sets the local Login instance for clients' use.

func (*Crawler) TestLogin

func (c *Crawler) TestLogin() error

TestLogin creates a new client, which attempts to log in with the credentials configured for this crawler, and then returns an error on failure.

func (*Crawler) Token

func (c *Crawler) Token() string

Token returns the current authorization token if available.

type CrawlerState

type CrawlerState struct {
	Token     string `yaml:"token"`
	LastCrawl string `yaml:"last_crawl"`
	LastID    int    `yaml:"last_id"`
	// contains filtered or unexported fields
}

func CrawlerStateFromPath

func CrawlerStateFromPath(path string) (*CrawlerState, error)

func (*CrawlerState) Save

func (c *CrawlerState) Save() error

type CredentialsFile

type CredentialsFile struct {
	AuthorizationToken string `yaml:"authorization_token"`
	EmailAddress       string `yaml:"email_address"`
	Password           string `yaml:"password"`
}

CredentialsFile defines the layout of the YAML configuration in which the user's authentication is expected to be found. If this file is empty or doesn't exist, the user will be prompted for their credentials.

The credentials file isn't necessary but may be useful for automating the client via a token.

func CredentialsFileFromPath

func CredentialsFileFromPath(path string) *CredentialsFile

CredentialsFileFromPath returns a CredentialsFile as returned from the specified path, or an empty CredentialsFile struct if it does not exist.

type Daemon

type Daemon struct {
	// contains filtered or unexported fields
}

func NewDaemon

func NewDaemon(configuration *api.Configuration) (*Daemon, error)

func (*Daemon) Init

func (d *Daemon) Init() error

func (*Daemon) Run

func (d *Daemon) Run()

func (*Daemon) Stop

func (d *Daemon) Stop()

func (*Daemon) TestLogin

func (d *Daemon) TestLogin() error

TestLogin creates a new client, which attempts to log in with the credentials configured for this crawler, and then returns an error on failure.

type GroupCSVWriter

type GroupCSVWriter struct {
	File io.WriteCloser
	CSV  *csv.Writer
	// contains filtered or unexported fields
}

func (*GroupCSVWriter) Close

func (g *GroupCSVWriter) Close() error

Close the CSV output and file interface.

Does not wrap errors.

func (*GroupCSVWriter) Write

func (g *GroupCSVWriter) Write(group *ct.Group) error

Write a group as CSV-formatted data to the file writer.

Does not wrap errors.

func (*GroupCSVWriter) WriteAll

func (g *GroupCSVWriter) WriteAll(groups []*ct.Group) error

WriteAll groups as CSV-formatted data to the file writer.

Does not wrap errors.

type GroupJSONWriter

type GroupJSONWriter struct {
	File io.WriteCloser
	// contains filtered or unexported fields
}

func (*GroupJSONWriter) Close

func (g *GroupJSONWriter) Close() error

Close the JSON output and file interface.

Does not wrap errors.

func (*GroupJSONWriter) Write

func (g *GroupJSONWriter) Write(group *ct.Group) error

Write a group as JSON-formatted data to the file writer.

Does not wrap errors.

func (*GroupJSONWriter) WriteAll

func (g *GroupJSONWriter) WriteAll(groups []*ct.Group) error

WriteAll groups as JSON-formatted data to the file writer.

Does not wrap errors.

type GroupWriter

type GroupWriter interface {
	Close() error
	Write(*ct.Group) error
	WriteAll([]*ct.Group) error
}

func NewGroupWriter

func NewGroupWriter(output io.WriteCloser, format string) (GroupWriter, error)

NewGroupWriter creates a new serializer for group data based on the specified format. If an invalid format is requested, ErrInvalidFormatRequested will be returned as the error.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL