parser

README

habr-auto

habr parser for w_popularity.

Status: working. Uses Habr's public JSON endpoints (no auth) with HTML

• RSS fallback.

Strategy

• Channel: GET /kek/v2/users/<handle>/card — the same JSON that habr.com itself fetches to render the profile card. Returns followers, post count, karma (scoreStats.score), rating and counters. Falls back to scraping the public profile HTML at /ru/users/<handle>/ if the JSON endpoint changes shape or 404s.
• Posts: GET /kek/v2/articles/?user=<handle>&perPage=20&order=date returns the latest 20 publications with full engagement stats (votesCountPlus, readingCount, commentsCount, reach). Falls back to the RSS feed at /ru/rss/users/<handle>/publications/articles/ when the JSON path fails (RSS has no engagement counts).
• Aggregates: ChannelSnapshot.TotalViews and TotalLikes are computed by summing across the most recent posts when the user card doesn't expose them directly.

Error mapping

HTTP	Sentinel
404 / NOT_FOUND	shared.ErrNotFound
429 / 503	shared.ErrRateLimited
other 5xx	shared.ErrTransient

Usage

import parser "github.com/suenot/habr-auto"

p := parser.New(parser.Config{HTTPTimeout: 15 * time.Second})
snap, err := p.FetchChannel(ctx, "suenot")
posts, err := p.FetchRecentPosts(ctx, "suenot", time.Time{})

A custom HTTPClient (e.g. with a RoundTripper for instrumentation or proxying) and a UserAgent can be supplied via Config. Habr returns 403 to requests with an empty UA, so the default mimics Chrome.

License

MIT

Documentation

Overview

Package parser implements the w_popularity Habr adapter using public HTML, RSS and Habr's own JSON endpoints (the same ones habr.com uses).

Endpoints used (all GET, JSON unless noted):

• /kek/v2/users/<handle>/card user profile counters
• /kek/v2/articles/?user=<h>&perPage=20&order=date author article list
• /ru/users/<handle>/ HTML fallback for card
• /ru/rss/users/<handle>/publications/articles/ RSS fallback for posts

Strategy:

primary:  Habr public JSON API (no auth required)
fallback: HTML scrape + RSS feed

Index

• type Config
• type HabrParser
  • func New(cfg Config) *HabrParser
  • func (p *HabrParser) FetchChannel(ctx context.Context, handle string) (shared.ChannelSnapshot, error)
  • func (p *HabrParser) FetchRecentPosts(ctx context.Context, handle string, since time.Time) ([]shared.PostSnapshot, error)
  • func (p *HabrParser) Platform() shared.Platform

Types

type Config

type Config struct {
	HTTPClient  *http.Client
	HTTPTimeout time.Duration
	// UserAgent overrides the default browser-ish UA. Habr 403s requests
	// with an empty UA.
	UserAgent string
}

Config controls runtime behaviour.

type HabrParser

type HabrParser struct {
	// contains filtered or unexported fields
}

func New

func New(cfg Config) *HabrParser

func (*HabrParser) FetchChannel

func (p *HabrParser) FetchChannel(ctx context.Context, handle string) (shared.ChannelSnapshot, error)

func (*HabrParser) FetchRecentPosts

func (p *HabrParser) FetchRecentPosts(ctx context.Context, handle string, since time.Time) ([]shared.PostSnapshot, error)

func (*HabrParser) Platform

func (p *HabrParser) Platform() shared.Platform