parser

Documentation

Overview

Package parser provide html email parsers this package contains different email parsers for each newsletter mail each parser should be ablt to parse a specfic newsletter email's html Contents and return a list of articles, which can be conposed into rss feeds

Index

• Constants
• type Analyzer
  • func NewAylienAnalyzer(appid, appkey string) Analyzer
• type Article
• type AylienAnalyzer
  • func (ay *AylienAnalyzer) Analyze(articles []Article)
• type DiscoverdevParser
  • func (dp *DiscoverdevParser) Parse(item *gofeed.Item) ([]Article, error)
• type FeedItemParser
  • func FindFeedParser(url, title string) (FeedItemParser, error)
• type HDCFeedParser
  • func (p *HDCFeedParser) Parse(item *gofeed.Item) ([]Article, error)
• type HackerNewsDigestParser
  • func (hn *HackerNewsDigestParser) Parse(html []byte) ([]Article, error)
• type MicroserviceParser
  • func (mp *MicroserviceParser) Parse(html []byte) ([]Article, error)
• type Parser
  • func FindParser(fromMail string, subject string, html string) (Parser, error)

Constants

const (
	// AylienAPIMaxConcurrentCount maximum concurrent api request count for aylien
	AylienAPIMaxConcurrentCount = 5
)

Types

type Analyzer

type Analyzer interface {
	Analyze([]Article)
}

Analyzer interface to provider analyze function for articles

func NewAylienAnalyzer

func NewAylienAnalyzer(appid, appkey string) Analyzer

NewAylienAnalyzer create an aylien analyzer

type Article

type Article struct {
	Title       string
	Link        string
	Summary     string
	Author      string
	PublishDate time.Time
	Image       string
	Videos      []string
	Domain      string
	Content     string
}

Article one article

type AylienAnalyzer

type AylienAnalyzer struct {
	AppID      string
	AppKey     string
	Client     *textapi.Client
	APIChannel chan struct{}
}

AylienAnalyzer analyzer using aylien api

func (*AylienAnalyzer) Analyze

func (ay *AylienAnalyzer) Analyze(articles []Article)

Analyze use aylien api to extract each article and get summary of them

type DiscoverdevParser

type DiscoverdevParser struct {
}

DiscoverdevParser mail parser for microservice weekly newsletter

func (*DiscoverdevParser) Parse

func (dp *DiscoverdevParser) Parse(item *gofeed.Item) ([]Article, error)

type FeedItemParser

type FeedItemParser interface {
	Parse(item *gofeed.Item) ([]Article, error)
}

func FindFeedParser

func FindFeedParser(url, title string) (FeedItemParser, error)

FindFeedParser find proper html parser for the specified feed

type HDCFeedParser

type HDCFeedParser struct{}

HDCFeedParser parse hdc feed

func (*HDCFeedParser) Parse

func (p *HDCFeedParser) Parse(item *gofeed.Item) ([]Article, error)

type HackerNewsDigestParser

type HackerNewsDigestParser struct {
}

MicroserviceParser mail parser for microservice weekly newsletter

func (*HackerNewsDigestParser) Parse

func (hn *HackerNewsDigestParser) Parse(html []byte) ([]Article, error)

Parse parst html contents and return an article list

type MicroserviceParser

type MicroserviceParser struct {
}

MicroserviceParser mail parser for microservice weekly newsletter

func (*MicroserviceParser) Parse

func (mp *MicroserviceParser) Parse(html []byte) ([]Article, error)

Parse parst html contents and return an article list

type Parser

type Parser interface {
	Parse(html []byte) ([]Article, error)
}

Parser the common parser interface

func FindParser

func FindParser(fromMail string, subject string, html string) (Parser, error)

FindParser find proper parser by the newsletter's information