extractor

Documentation

Index

• func FormatArticle(article *Article, includeImages bool) (string, error)
• func StripImages(markdown string) string
• func ToMarkdown(html string) (string, error)
• type Article
  • func ExtractArticle(html string, pageURL string) (*Article, error)
• type PageResult
  • func FetchPage(url string, timeout time.Duration) (*PageResult, error)
  • func FetchPageHTTP(url string, timeout time.Duration) (*PageResult, error)

Functions

func FormatArticle

func FormatArticle(article *Article, includeImages bool) (string, error)

FormatArticle formats the extracted article as markdown with metadata

func StripImages

func StripImages(markdown string) string

StripImages removes all markdown image references from content

func ToMarkdown

func ToMarkdown(html string) (string, error)

ToMarkdown converts HTML content to GitHub Flavored Markdown

Types

type Article

type Article struct {
	Title   string
	Byline  string
	Content string // HTML content
	URL     string
}

Article contains extracted article content

func ExtractArticle

func ExtractArticle(html string, pageURL string) (*Article, error)

ExtractArticle uses Readability to extract the main content from HTML

type PageResult

type PageResult struct {
	HTML string
	URL  string // Final URL after redirects
}

PageResult contains the fetched page data

func FetchPage

func FetchPage(url string, timeout time.Duration) (*PageResult, error)

FetchPage loads a URL using Chrome with the user's profile for authentication

func FetchPageHTTP

func FetchPageHTTP(url string, timeout time.Duration) (*PageResult, error)

FetchPageHTTP fetches a page using simple HTTP (no JavaScript execution)