fbbotscan

README

fbbotscan

An experimental approach to detecting Facebook bots (actually, just a facebook data pipeline)

Goals

To provide a public API which can identify Facebook users as bots/trolls.

This project is nowhere near there.

Status

This is not some fancy ML approach to detecting bots, it is actually simply a data pipeline to stream public facebook posts, comments, and sub-comments.

There is a classifier, but it simply uses an ElasticSearch MLT Query to find similar comments. As of now, the classifier doesn't do anything with that information yet.

Architecture

The data pipeline consists of four major components:

• scheduler: Queries postgres for objects which need to be scheduled to be queried for new posts or comments
• fetcher: Goes off to Facebook and pulls down new posts and comments using the Graph API.
• storer: Stores/Indexes new Facebook comments in ElasticSearch
• classifier: Attempts to classify each new comment to determine whether the author should be marked as a bot or not

The postgres database simply holds metadata (object IDs and time stamp of last check) for the purpose of telling the scheduler when to issue a re-check.

All these components communicate by means of AMQP queues (tested against RabbitMQ 3.x). In an actual setup, you would run one scheduler, and then many fetchers, storers (I should rename this to "indexer"...) and classifiers.

The "storer" indexes new comments in ElasticSearch.

Usage

TODO

Documentation

Index

• Constants
• type FBApp
  • func New(configPath string) *FBApp
  • func (a *FBApp) Initialize() error
  • func (a *FBApp) LoadComments(objectId string, since int64) ([]FBComment, error)
  • func (a *FBApp) LoadConfig() error
  • func (a *FBApp) LoadFeed(pageId string, maxEntries int, since int64) ([]FBPost, error)
• type FBComment
• type FBCommentList
• type FBComments
• type FBFeed
• type FBPage
• type FBPageList
• type FBPaging
• type FBPost
• type FBPostList
• type FBUser
• type FBUserList
• type QueueEntry

Constants

const CommentMapping = `` /* 2002-byte string literal not displayed */

Types

type FBApp

type FBApp struct {
	AppToken   string
	App        *fb.App
	Session    *fb.Session
	ConfigPath string
	Config     *config.Config
}

func New

func New(configPath string) *FBApp

func (*FBApp) Initialize

func (a *FBApp) Initialize() error

func (*FBApp) LoadComments

func (a *FBApp) LoadComments(objectId string, since int64) ([]FBComment, error)

func (*FBApp) LoadConfig

func (a *FBApp) LoadConfig() error

func (*FBApp) LoadFeed

func (a *FBApp) LoadFeed(pageId string, maxEntries int, since int64) ([]FBPost, error)

type FBComment

type FBComment struct {
	CreatedTime string `json:"created_time"`
	From        FBUser `json:"from"`
	ID          string `json:"id"`
	Message     string `json:"message"`
	Parent      struct {
		CreatedTime string `json:"created_time"`
		From        FBUser `json:"from"`
		ID          string `json:"id"`
		Message     string `json:"message"`
	} `json:"parent"`
	PermalinkURL string `json:"permalink_url"`
	CommentCount int32  `json:"comment_count"`
	LikeCount    int32  `json:"like_count"`
}

type FBCommentList

type FBCommentList struct {
	Entries []FBComment `json:"data"`
}

type FBComments

type FBComments struct {
	Comments FBCommentList `json:"data"`
	Paging   FBPaging      `json:"paging"`
}

type FBFeed

type FBFeed struct {
	Posts  FBPostList `json:"data"`
	Paging FBPaging   `json:"paging"`
}

type FBPage

type FBPage struct {
	ID   string `json:"id"`
	Link string `json:"link"`
	Name string `json:"name"`
}

type FBPageList

type FBPageList struct {
	Entries []FBPage `json:"data"`
}

type FBPaging

type FBPaging struct {
	Cursors struct {
		After  string `json:"after"`
		Before string `json:"before"`
	} `json:"cursors"`
	Next     string `json:"next"`
	Previous string `json:"previous"`
}

type FBPost

type FBPost struct {
	CreatedTime  string `json:"created_time"`
	ID           string `json:"id"`
	Link         string `json:"link"`
	Message      string `json:"message"`
	Story        string `json:"story"`
	PermalinkURL string `json:"permalink_url"`
}

type FBPostList

type FBPostList struct {
	Entries []FBPost `json:"data"`
}

type FBUser

type FBUser struct {
	FirstName  string `json:"first_name"`
	ID         string `json:"id"`
	IsVerified bool   `json:"is_verified"`
	LastName   string `json:"last_name"`
	Name       string `json:"name"`
	NameFormat string `json:"name_format"`
	ShortName  string `json:"short_name"`
}

type FBUserList

type FBUserList struct {
	Entries []FBUser `json:"data"`
}

type QueueEntry

type QueueEntry struct {
	ObjectID    string `json:"object_id"`
	LastChecked int64  `json:"last_checked"`
	ObjectType  string `json:"type"`
}