cafef

Documentation

Overview

Package cafef is the library behind the cafef command line: the HTTP client, HTML scraping, and typed data models for CafeF (cafef.vn), Vietnam's leading financial news and stock-market portal.

CafeF does not publish public RSS feeds. Articles are scraped from category listing HTML pages. Stock prices are scraped from the market data table at /du-lieu-chung-khoan/gia-co-phieu.chn. Article URLs embed a numeric ID: /{category}/{slug}-{id}.chn.

Index

• Constants
• Variables
• type Article
• type Category
• type Client
  • func NewClient() *Client
  • func NewClientWithConfig(cfg Config) *Client
  • func (c *Client) CategoryArticles(ctx context.Context, slug string, limit int) ([]*Article, error)
  • func (c *Client) Get(ctx context.Context, rawURL string) ([]byte, error)
  • func (c *Client) GetCompany(ctx context.Context, ticker string) (*Company, error)
  • func (c *Client) GetStock(ctx context.Context, ticker string) (*Stock, error)
  • func (c *Client) LatestArticles(ctx context.Context, limit int) ([]*Article, error)
  • func (c *Client) ListCategories() []*Category
  • func (c *Client) ListStocks(ctx context.Context, exchange string, limit int) ([]*Stock, error)
  • func (c *Client) SearchArticles(ctx context.Context, query string, limit int) ([]*Article, error)
• type Company
• type Config
  • func DefaultConfig() Config
• type Domain
  • func (Domain) Classify(input string) (uriType, id string, err error)
  • func (Domain) Info() kit.DomainInfo
  • func (Domain) Locate(uriType, id string) (string, error)
  • func (Domain) Register(app *kit.App)
• type Stock

Constants

const DefaultUserAgent = "cafef-cli/0.1.0 (+https://github.com/tamnd/cafef-cli)"

DefaultUserAgent identifies this client to CafeF.

const Host = "cafef.vn"

Host is the canonical site hostname.

Variables

var Categories = []string{
	"thi-truong-chung-khoan",
	"bat-dong-san",
	"ngan-hang",
	"doanh-nghiep",
	"vi-mo-dau-tu",
	"tai-chinh-ca-nhan",
	"the-gioi",
}

Categories lists CafeF's financial news category slugs.

Types

type Article

type Article struct {
	ID          string `json:"id"                   kit:"id" table:"id"`
	Title       string `json:"title"                          table:"title"`
	URL         string `json:"url,omitempty"                  table:"url,url"`
	Category    string `json:"category,omitempty"             table:"category"`
	Description string `json:"description,omitempty"          table:"-"`
	Thumbnail   string `json:"thumbnail,omitempty"            table:"-"`
	PublishedAt string `json:"published_at,omitempty"         table:"published_at"`
}

Article is one CafeF news article scraped from a category listing page.

type Category

type Category struct {
	Slug string `json:"slug" kit:"id" table:"slug"`
	Name string `json:"name"          table:"name"`
	URL  string `json:"url"           table:"url,url"`
}

Category represents one CafeF news category.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client talks to CafeF over HTTP.

func NewClient

func NewClient() *Client

NewClient returns a Client from DefaultConfig.

func NewClientWithConfig

func NewClientWithConfig(cfg Config) *Client

NewClientWithConfig returns a Client built from cfg.

func (*Client) CategoryArticles

func (c *Client) CategoryArticles(ctx context.Context, slug string, limit int) ([]*Article, error)

CategoryArticles fetches articles from the given category's listing page.

func (*Client) Get

func (c *Client) Get(ctx context.Context, rawURL string) ([]byte, error)

Get fetches rawURL and returns the body bytes, pacing and retrying on transient errors.

func (*Client) GetCompany

func (c *Client) GetCompany(ctx context.Context, ticker string) (*Company, error)

GetCompany fetches the company profile for a single ticker symbol.

func (*Client) GetStock

func (c *Client) GetStock(ctx context.Context, ticker string) (*Stock, error)

GetStock fetches price details for a single ticker symbol.

func (*Client) LatestArticles

func (c *Client) LatestArticles(ctx context.Context, limit int) ([]*Article, error)

LatestArticles fetches recent articles from the stock market news feed.

func (*Client) ListCategories

func (c *Client) ListCategories() []*Category

ListCategories returns all known CafeF news categories.

func (*Client) ListStocks

func (c *Client) ListStocks(ctx context.Context, exchange string, limit int) ([]*Stock, error)

ListStocks fetches the stock price table for the given exchange (HOSE, HNX, UPCOM). An empty exchange returns all rows available on the default page.

func (*Client) SearchArticles

func (c *Client) SearchArticles(ctx context.Context, query string, limit int) ([]*Article, error)

SearchArticles keyword-searches recent articles across all category feeds.

type Company

type Company struct {
	Ticker      string `json:"ticker"                    kit:"id" table:"ticker"`
	CompanyName string `json:"company_name"                       table:"company_name"`
	FullName    string `json:"full_name,omitempty"                table:"full_name"`
	Exchange    string `json:"exchange"                           table:"exchange"`
	Sector      string `json:"sector,omitempty"                   table:"sector"`
	Industry    string `json:"industry,omitempty"                 table:"industry"`
	Website     string `json:"website,omitempty"                  table:"website,url"`
	Address     string `json:"address,omitempty"                  table:"-"`
	FetchedAt   string `json:"fetched_at,omitempty"               table:"fetched_at"`
}

Company is a company profile scraped from CafeF's stock data section.

type Config

type Config struct {
	BaseURL   string
	Rate      time.Duration
	Retries   int
	Timeout   time.Duration
	UserAgent string
}

Config holds the tunable knobs for the HTTP client.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns sensible production defaults.

type Domain

type Domain struct{}

Domain is the CafeF driver.

func (Domain) Classify

func (Domain) Classify(input string) (uriType, id string, err error)

Classify turns a CafeF URL or numeric article ID into (type, id).

func (Domain) Info

func (Domain) Info() kit.DomainInfo

func (Domain) Locate

func (Domain) Locate(uriType, id string) (string, error)

Locate returns the live https URL for a (type, id).

func (Domain) Register

func (Domain) Register(app *kit.App)

type Stock

type Stock struct {
	Ticker        string  `json:"ticker"                    kit:"id" table:"ticker"`
	CompanyName   string  `json:"company_name"                       table:"company_name"`
	Exchange      string  `json:"exchange"                           table:"exchange"`
	Price         float64 `json:"price"                              table:"price"`
	ChangeAmount  float64 `json:"change_amount"                      table:"change_amount"`
	ChangePercent float64 `json:"change_percent"                     table:"change_percent"`
	Volume        int64   `json:"volume"                             table:"volume"`
	FetchedAt     string  `json:"fetched_at,omitempty"               table:"fetched_at"`
}

Stock is a stock price record scraped from the CafeF market data table.