twitter

README

go-birdsite / twitter

A pure-Go (CGO=0), dependency-free, best-effort read client for public Twitter/X profile timelines. It reads the public syndication timeline endpoint that powers embedded timeline widgets and extracts tweets from the __NEXT_DATA__ JSON blob.

c := twitter.New(twitter.WithHTTPClient(browserhttp.NewClient(30 * time.Second)))
tl, err := c.UserTweets(context.Background(), "jack")
for _, tw := range tl.Tweets {
    o := tw.Original() // the retweeted tweet for a retweet, else tw itself
    fmt.Printf("@%s (%s): %s (%d likes)\n", o.User.ScreenName, o.User.Name, o.ExpandedText(), o.Likes)
    for _, m := range o.Media {
        if v, ok := m.BestVariant(); ok {
            fmt.Printf("  video %dx%d: %s\n", m.Width, m.Height, v.URL)
        }
    }
}

🔑 Use a browser-fingerprinting HTTP client

The endpoint answers 429 to Go's stock net/http even when the account's quota is untouched — it fingerprints the TLS/HTTP2 handshake, not the User-Agent. curl gets a 200 from the same IP with the same User-Agent in the same second. Pass a fingerprinting client such as go-browserhttp via WithHTTPClient for anything beyond a one-shot read. That refusal is reported as ErrFingerprinted, distinct from ErrNotFound (unknown account) and ErrProtected (non-public account), so callers can say what actually happened instead of blaming a missing token.

What a tweet carries

Author (handle, display name, avatar, bio, verified, followers) · full text · ExpandedText() with every t.co resolved · Links (short/expanded/display) · PrimaryLink(), the first destination that is not Twitter/X itself · photos and videos with alt text, pixel size, duration and every encoding, plus BestVariant() for the highest-bitrate progressive MP4 · retweets (Retweeted, Original()) and quotes (Quoted) · like/retweet/reply/quote counts · language and sensitivity flags.

⚠️ Fragility & Terms of Service

This is inherently fragile. Twitter/X changes and locks these endpoints, and some profiles or rate states require a valid auth token (WithAuthToken). Blocked requests surface as errors. Respect Twitter/X's Terms of Service and applicable law when using this library.

License

BSD-3-Clause © the go-birdsite/twitter authors.

Documentation

Overview

Package twitter is a dependency-free, best-effort read client for public Twitter/X profile timelines. It uses the public syndication timeline endpoint that powers embedded timeline widgets, extracting the tweets from the __NEXT_DATA__ JSON blob in the returned HTML.

This is inherently fragile: Twitter/X changes and locks these endpoints, and some profiles or rate states require a valid auth token. Requests that are blocked surface as errors rather than pretending to be reliable.

Client fingerprinting

The endpoint answers 429 to Go's stock net/http even when the account's quota is untouched, because it fingerprints the TLS/HTTP2 handshake rather than the User-Agent. Pass a browser-fingerprinting http.Client via WithHTTPClient (for example github.com/go-browserhttp/browserhttp) for anything beyond a one-shot read. ErrFingerprinted reports that case so callers can say so precisely instead of blaming a missing token.

Index

• Constants
• Variables
• type Client
  • func New(opts ...Option) *Client
  • func (c *Client) Following(ctx context.Context, userID, cursor string) (*FollowingPage, error)
  • func (c *Client) UserTweets(ctx context.Context, screenName string) (*Timeline, error)
• type FollowedUser
• type FollowingPage
• type Link
• type Media
  • func (m Media) BestVariant() (v VideoVariant, ok bool)
• type Option
  • func WithAPIBaseURL(u string) Option
  • func WithAuthToken(t string) Option
  • func WithBaseURL(u string) Option
  • func WithBearer(token string) Option
  • func WithFollowingQueryID(id string) Option
  • func WithHTTPClient(h *http.Client) Option
  • func WithSessionCookies(authToken, csrf string) Option
  • func WithUserAgent(ua string) Option
• type Timeline
• type Tweet
  • func (t Tweet) ExpandedText() string
  • func (t Tweet) Original() Tweet
  • func (t Tweet) PrimaryLink() string
• type User
• type VideoVariant

Constants

const DefaultAPIBaseURL = "https://x.com"

DefaultAPIBaseURL is the origin X serves its private GraphQL API from. Unlike the public syndication host (DefaultBaseURL) this one requires the logged-in auth_token + ct0 cookies (see WithSessionCookies).

const DefaultBaseURL = "https://syndication.twitter.com"

DefaultBaseURL is the public syndication host.

const DefaultFollowingQueryID = "iSicc7LrzWGBgDPL0tM_TQ"

DefaultFollowingQueryID is the GraphQL query id for the Following operation. X rotates these ids; when it does the endpoint 404s and Client.Following reports ErrQueryIDRotated so a caller can say so precisely.

const DefaultWebBearer = "AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs=" +
	"1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA"

DefaultWebBearer is the public web-app bearer token X's own site sends on every GraphQL call. It is not a per-user secret — the same constant is used by every logged-out and logged-in web session; the real authentication is the auth_token + ct0 cookies. X may rotate it, in which case the call is rejected and Client.Following reports it (see ErrNeedsAuth).

Variables

var (
	// ErrNeedsAuth reports that the Following call cannot proceed: the auth_token
	// + ct0 session cookies are missing, or X refused the read as unauthenticated
	// (401/403) — the session expired or the bearer rotated.
	ErrNeedsAuth = errors.New("twitter: the Following list needs a logged-in session (auth_token + ct0)")

	// ErrQueryIDRotated reports that X answered 404: the GraphQL query id is stale
	// (X rotated it). The read cannot succeed until the id is updated.
	ErrQueryIDRotated = errors.New("twitter: the Following GraphQL query id rotated (404)")
)

var (
	// ErrFingerprinted reports a 429 refusal aimed at the HTTP client's TLS
	// fingerprint rather than at the account quota. Retrying with the same client
	// never succeeds; use a browser-fingerprinting http.Client instead.
	ErrFingerprinted = errors.New("twitter: request refused (429): the endpoint fingerprints the TLS client — use a browser-fingerprinting http.Client")

	// ErrNotFound reports an unknown, suspended or renamed screen name.
	ErrNotFound = errors.New("twitter: no such account")

	// ErrProtected reports an account whose tweets are not public.
	ErrProtected = errors.New("twitter: account is protected")
)

Types

type Client

type Client struct {
	// BaseURL is the syndication host; defaults to DefaultBaseURL.
	BaseURL string
	// HTTPClient is used for all requests; defaults to http.DefaultClient.
	HTTPClient *http.Client
	// UserAgent is sent with every request.
	UserAgent string
	// AuthToken, when set, is sent as a bearer token for authenticated reads.
	AuthToken string

	// APIBaseURL is the GraphQL origin; defaults to [DefaultAPIBaseURL].
	APIBaseURL string
	// SessionAuthToken is the logged-in "auth_token" cookie the Following call
	// authenticates with.
	SessionAuthToken string
	// CSRFToken is the logged-in "ct0" cookie, sent as the ct0 cookie and the
	// x-csrf-token header on the Following call.
	CSRFToken string
	// Bearer overrides the web-app bearer token; defaults to [DefaultWebBearer].
	Bearer string
	// FollowingQueryID overrides the Following GraphQL query id; defaults to
	// [DefaultFollowingQueryID].
	FollowingQueryID string
}

Client reads public profile timelines.

func New

func New(opts ...Option) *Client

New returns a Client with sane defaults.

func (*Client) Following

func (c *Client) Following(ctx context.Context, userID, cursor string) (*FollowingPage, error)

Following returns one page of the accounts userID follows, starting at cursor ("" for the first page). userID is the viewer's own numeric id (read, for example, from the twid cookie). It calls X's private GraphQL Following query with the web bearer and the auth_token + ct0 cookies, unwraps the timeline's user entries, and returns the bottom pagination cursor.

It requires a logged-in session: without the auth_token + ct0 cookies (see WithSessionCookies) it returns ErrNeedsAuth without a request. A 401/403 (expired session / rotated bearer) also maps to ErrNeedsAuth, and a 404 (rotated query id) to ErrQueryIDRotated, so a caller can report each case precisely rather than as a generic failure.

func (*Client) UserTweets

func (c *Client) UserTweets(ctx context.Context, screenName string) (*Timeline, error)

UserTweets fetches the public profile timeline for screenName.

type FollowedUser

type FollowedUser struct {
	ID         string // the account's numeric rest id
	ScreenName string // handle, without "@"
	Name       string // display name
}

FollowedUser is one account the viewer follows.

type FollowingPage

type FollowingPage struct {
	Users  []FollowedUser
	Cursor string
}

FollowingPage is one page of the viewer's Following list. Cursor is the bottom pagination cursor to pass on the next call, and is "" when the list is exhausted.

type Link

type Link struct {
	URL      string // the t.co short URL as it appears in Text
	Expanded string // the real destination
	Display  string // the human-readable form Twitter renders ("nasa.gov/live")
}

Link is a URL in a tweet's text: the t.co shortener, plus what it points at.

type Media

type Media struct {
	URL        string // media_url_https: the photo, or a video's preview frame
	Type       string // "photo" | "video" | "animated_gif"
	AltText    string // author-supplied accessibility description ("" if none)
	Width      int    // original pixel width (0 if unknown)
	Height     int    // original pixel height (0 if unknown)
	DurationMS int    // video/GIF duration in milliseconds (0 for photos)
	Variants   []VideoVariant
}

Media is an attachment on a tweet. For a video or animated GIF, URL is the still preview image and Variants carries the playable encodings.

func (Media) BestVariant

func (m Media) BestVariant() (v VideoVariant, ok bool)

BestVariant returns the highest-bitrate progressive MP4 encoding, which is the one a plain video decoder can play. ok is false for photos, and for videos offered only as an HLS playlist.

type Option

type Option func(*Client)

Option configures a Client.

func WithAPIBaseURL

func WithAPIBaseURL(u string) Option

WithAPIBaseURL overrides the GraphQL origin used by Client.Following (used in tests).

func WithAuthToken

func WithAuthToken(t string) Option

WithAuthToken sets an optional bearer token for authenticated reads.

func WithBaseURL

func WithBaseURL(u string) Option

WithBaseURL overrides the syndication host (used in tests).

func WithBearer

func WithBearer(token string) Option

WithBearer overrides the web-app bearer token sent on the Following call.

func WithFollowingQueryID

func WithFollowingQueryID(id string) Option

WithFollowingQueryID overrides the Following GraphQL query id.

func WithHTTPClient

func WithHTTPClient(h *http.Client) Option

WithHTTPClient sets the http.Client used for requests.

func WithSessionCookies

func WithSessionCookies(authToken, csrf string) Option

WithSessionCookies sets the logged-in auth_token + ct0 cookies that Client.Following authenticates with.

func WithUserAgent

func WithUserAgent(ua string) Option

WithUserAgent sets the User-Agent header.

type Timeline

type Timeline struct {
	Tweets []Tweet
}

Timeline is a page of tweets.

type Tweet

type Tweet struct {
	ID        string
	Text      string
	Author    string // screen name; the same value as User.ScreenName
	User      User
	Permalink string
	CreatedAt time.Time
	Likes     int
	Retweets  int
	Replies   int
	Quotes    int
	Lang      string
	Sensitive bool
	Media     []Media
	Links     []Link
	// Retweeted is the original tweet when this entry is a retweet, else nil. Its
	// own Text is the real content; this tweet's Text is the "RT @x: …" stub.
	Retweeted *Tweet
	// Quoted is the tweet this one quotes, else nil.
	Quoted *Tweet
}

Tweet is a single normalized tweet.

func (Tweet) ExpandedText

func (t Tweet) ExpandedText() string

ExpandedText returns Text with every t.co short URL replaced by its real destination, so a reader can display and follow the links it actually shows.

func (Tweet) Original

func (t Tweet) Original() Tweet

Original returns the tweet carrying the actual content: the retweeted tweet for a retweet, otherwise the tweet itself.

func (Tweet) PrimaryLink

func (t Tweet) PrimaryLink() string

PrimaryLink returns the first external destination the tweet points at, or "" when it links nowhere. Links to Twitter/X itself (a quoted tweet's permalink, a photo page) are skipped: they are not "an article this post links out to".

type User

type User struct {
	ID          string
	ScreenName  string // handle, without "@"
	Name        string // display name
	AvatarURL   string // profile picture (https)
	Description string // bio
	Verified    bool   // legacy verified or Blue-verified
	Followers   int
	Protected   bool
}

User is the author of a tweet.

type VideoVariant

type VideoVariant struct {
	URL         string
	ContentType string // "video/mp4", "application/x-mpegURL", …
	Bitrate     int    // bits per second; 0 for playlists
}

VideoVariant is one encoding of a video or animated GIF attachment.