README
¶
github-auto
GitHub parser for w_popularity.
Strategy
REST API v3 over https://api.github.com. No HTML scraping, no auth required
for the public-data path. Anonymous quota is 60 req/hour per IP; setting
Config.Token (a personal access token, fine-grained or classic — no scopes
needed for read-only public access) bumps it to 5000 req/hour.
Endpoints used:
| Purpose | URL |
|---|---|
| Profile counts | GET /users/<handle> |
| Owned repos (per_page=100, sort=updated) | GET /users/<handle>/repos?type=owner |
| Recent activity (push/create/release/…) | GET /users/<handle>/events/public?per_page=50 |
Field mapping
| Source | ChannelSnapshot |
|---|---|
followers |
Followers |
public_repos |
PostsCount |
Σ stargazers_count across owned repos |
TotalLikes |
Σ watchers_count |
TotalViews |
top_repos, languages, topics_top, name, bio, company, location, blog, twitter_username, avatar_url, created_at, total_forks, archived_repo_count, forked_repo_count |
Raw |
top_repos
Repos owned by the user with stargazers_count >= Config.TopReposMinStars
(default 5), sorted by stars desc, capped at 20. Each entry has:
name, full_name, url, stars, forks, language, description, topics, pushed_at, archived, fork.
languages
map[language] → repo_count across owned repos.
topics_top
Top 15 GitHub topics by frequency across owned repos.
Usage
import parser "github.com/suenot/github-auto"
p := parser.New(parser.Config{
Token: os.Getenv("GITHUB_TOKEN"), // optional
TopReposMinStars: 5,
})
snap, err := p.FetchChannel(ctx, "suenot")
License
MIT
Documentation
¶
Overview ¶
Package parser implements the w_popularity GitHub adapter.
Strategy: GitHub REST API v3 (https://api.github.com).
- GET /users/<handle> → followers, following, public_repos, ...
- GET /users/<handle>/repos?... → owned repos with stargazers_count, forks etc.
- GET /users/<handle>/events/public?per_page=50 → recent activity (pushes, creates, releases) used as "posts" surrogate.
Anonymous quota: 60 req/hour per IP. Authenticated (Config.Token): 5000/hour. We optimise for 1 fetch per channel per day → anonymous is fine for a handful of channels.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// Token is an optional GitHub personal access token (PAT) or fine-grained
// token. When set, bumps the rate limit from 60 → 5000 req/hour. The
// parser only reads public data so `public_repo` scope is plenty (none is
// needed for fully public reads).
Token string
HTTPClient *http.Client
HTTPTimeout time.Duration
UserAgent string
BaseURL string // overrides https://api.github.com (test hook)
// TopReposMinStars is the threshold below which repos are NOT included in
// the per-snapshot `Raw["top_repos"]` summary list. Stars-based filtering
// keeps the snapshot lean for users with hundreds of small repos.
// Default: 5.
TopReposMinStars int
// MaxReposPage caps the per-page repo listing. Default 100 (the GitHub
// max). We fetch a single page; users with >100 repos lose tail data,
// which is fine for the "top repos by stars" use case.
MaxReposPage int
}
Config controls runtime behaviour.
type GitHubParser ¶
type GitHubParser struct {
// contains filtered or unexported fields
}
func New ¶
func New(cfg Config) *GitHubParser
func (*GitHubParser) FetchChannel ¶
func (p *GitHubParser) FetchChannel(ctx context.Context, handle string) (shared.ChannelSnapshot, error)
FetchChannel returns one snapshot per call: profile counts + an aggregate of owned repos. We assemble `Raw["top_repos"]` (repos with stars ≥ TopReposMinStars, sorted desc by stars, capped at 20) and per-language / per-topic histograms.
func (*GitHubParser) FetchRecentPosts ¶
func (p *GitHubParser) FetchRecentPosts(ctx context.Context, handle string, since time.Time) ([]shared.PostSnapshot, error)
func (*GitHubParser) Platform ¶
func (p *GitHubParser) Platform() shared.Platform
Source Files