nld

Documentation

Overview

Package nld is the library behind the nld command line: the HTTP client, RSS feed parsing, and typed data models for Người Lao Động (nld.com.vn), Vietnam's independent labor and general news daily.

NLD publishes per-category RSS 2.0 feeds. Article URLs embed a numeric ID: /{category}/{slug}-{id}.htm.

Index

• Constants
• Variables
• type Article
• type Category
• type Client
  • func NewClient() *Client
  • func NewClientWithConfig(cfg Config) *Client
  • func (c *Client) CategoryArticles(ctx context.Context, slug string, limit int) ([]*Article, error)
  • func (c *Client) Get(ctx context.Context, rawURL string) ([]byte, error)
  • func (c *Client) LatestArticles(ctx context.Context, limit int) ([]*Article, error)
  • func (c *Client) ListCategories() []*Category
  • func (c *Client) SearchArticles(ctx context.Context, query string, limit int) ([]*Article, error)
• type Config
  • func DefaultConfig() Config
• type Domain
  • func (Domain) Classify(input string) (uriType, id string, err error)
  • func (Domain) Info() kit.DomainInfo
  • func (Domain) Locate(uriType, id string) (string, error)
  • func (Domain) Register(app *kit.App)

Constants

const DefaultUserAgent = "nld-cli/0.1.0 (+https://github.com/tamnd/nld-cli)"

DefaultUserAgent identifies this client to NLD.

const Host = "nld.com.vn"

Host is the canonical site hostname.

Variables

var Categories = []string{
	"home",
	"thoi-su",
	"ban-doc",
	"kinh-te",
	"the-gioi",
	"van-hoa-giai-tri",
	"the-thao",
	"phap-luat",
	"giao-duc",
	"suc-khoe",
	"khoa-hoc-cong-nghe",
	"nha-dep-doi-song",
}

Categories lists NLD's RSS feed category slugs.

Types

type Article

type Article struct {
	ID          string `json:"id"                   kit:"id" table:"id"`
	Title       string `json:"title"                          table:"title"`
	URL         string `json:"url,omitempty"                  table:"url,url"`
	Category    string `json:"category,omitempty"             table:"category"`
	Description string `json:"description,omitempty"          table:"-"`
	Author      string `json:"author,omitempty"               table:"author"`
	Thumbnail   string `json:"thumbnail,omitempty"            table:"-"`
	PublishedAt string `json:"published_at,omitempty"         table:"published_at"`
}

Article is one NLD article extracted from an RSS feed.

type Category

type Category struct {
	Slug string `json:"slug" kit:"id" table:"slug"`
	Name string `json:"name"          table:"name"`
	URL  string `json:"url"           table:"url,url"`
	RSS  string `json:"rss"           table:"-"`
}

Category represents one NLD RSS feed category.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client talks to NLD RSS feeds over HTTP.

func NewClient

func NewClient() *Client

NewClient returns a Client from DefaultConfig.

func NewClientWithConfig

func NewClientWithConfig(cfg Config) *Client

NewClientWithConfig returns a Client built from cfg.

func (*Client) CategoryArticles

func (c *Client) CategoryArticles(ctx context.Context, slug string, limit int) ([]*Article, error)

CategoryArticles fetches articles for the given category slug from its RSS feed.

func (*Client) Get

func (c *Client) Get(ctx context.Context, rawURL string) ([]byte, error)

Get fetches rawURL and returns the body bytes, pacing and retrying on transient errors.

func (*Client) LatestArticles

func (c *Client) LatestArticles(ctx context.Context, limit int) ([]*Article, error)

LatestArticles fetches the most recent articles from the home RSS feed.

func (*Client) ListCategories

func (c *Client) ListCategories() []*Category

ListCategories returns all known NLD RSS feed categories.

func (*Client) SearchArticles

func (c *Client) SearchArticles(ctx context.Context, query string, limit int) ([]*Article, error)

SearchArticles keyword-searches across several category feeds.

type Config

type Config struct {
	BaseURL   string
	Rate      time.Duration
	Retries   int
	Timeout   time.Duration
	UserAgent string
}

Config holds the tunable knobs for the HTTP client.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns sensible production defaults.

type Domain

type Domain struct{}

Domain is the NLD driver.

func (Domain) Classify

func (Domain) Classify(input string) (uriType, id string, err error)

Classify turns an NLD URL or numeric ID into (type, id).

func (Domain) Info

func (Domain) Info() kit.DomainInfo

func (Domain) Locate

func (Domain) Locate(uriType, id string) (string, error)

Locate returns the live https URL for a (type, id).

func (Domain) Register

func (Domain) Register(app *kit.App)