analytics

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 31, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package analytics records and reads click data.

The privacy guarantee here is structural rather than procedural: no IP address is ever written to click_events, because the table has no column for one. Visitor identity is a keyed hash whose key is deleted on a schedule, and that deletion is what makes the day's hashes irreversible.

Index

Constants

View Source 
const PermRead = "analytics.read"

PermRead is the permission analytics reads require.

View Source 
const SaltLength = 32

SaltLength is the size of a daily salt.

View Source 
const SaltRetentionDays = 2

SaltRetentionDays is how long a salt is kept before deletion.

The salt must outlive the events it keyed for as long as unique-visitor counting needs to recompute, but no longer — its deletion is what makes those hashes permanently unlinkable to an address. Two days covers a same-day rollup plus the finalize pass over the previous day.

View Source 
const VisitorHashLength = 16

VisitorHashLength is how much of the HMAC is kept.

16 bytes is far beyond collision range for per-day, per-link counting, and truncating reduces what a database holds without weakening the property that matters: the hash cannot be reversed once the salt is gone.

Variables

View Source 
var ErrIngesterClosed = errors.New("analytics: ingester is closed")

Functions

func AnonymizeIP 

func AnonymizeIP(addr netip.Addr) string

AnonymizeIP reduces an address to a network prefix: /24 for IPv4, /48 for IPv6.

Used for session and audit records, never for click events — those keep no address at all. The distinction is deliberate: "where was my account signed in from" is a question a user legitimately asks about their own data, whereas analytics has no such need.

func FormatHash 

func FormatHash(h []byte) string

FormatHash renders a hash for logs and debugging. Never used for storage.

func NewSalt 

func NewSalt() ([]byte, error)

NewSalt generates a day's salt.

func PrimaryLanguage 

func PrimaryLanguage(header string) string

PrimaryLanguage extracts the first tag from an Accept-Language header.

Only the language subtag is kept ("en" from "en-GB"). Region adds granularity nobody reports on and narrows the anonymity set.

func ReferrerHost 

func ReferrerHost(referrer string) string

ReferrerHost extracts the host from a referrer.

Only the host is kept. Full referrer URLs routinely carry query parameters with session tokens, search terms and personal data, so the rest is discarded at the edge rather than stored and cleaned up later.

func SaltDay 

func SaltDay(at time.Time) time.Time

SaltDay returns the UTC day a timestamp belongs to.

UTC always, never local time. A local-time boundary would rotate the salt at a different instant per deployment and, worse, make the same visitor hash differently on either side of a daylight-saving change.

func VisitorHash 

func VisitorHash(salt []byte, ip netip.Addr, userAgent string, workspaceID uuid.UUID) []byte

VisitorHash derives a per-day, per-workspace visitor identifier.

HMAC rather than a plain hash of salt||data: a plain concatenation is vulnerable to length extension, and HMAC is the construction actually designed for keying a hash.

workspaceID is part of the message, not the key. That is what stops the same person being correlated across two workspaces on the same instance — the salt is shared per day, but the derived hashes differ, so one workspace's analytics cannot be joined against another's.

The inputs are separated by a NUL byte so that ("ab", "c") and ("a", "bc") cannot produce the same hash. Without a separator, a crafted user agent could be made to collide with a different address.

Types

type Classification 

type Classification struct {
	Device  Device
	Browser string
	OS      string
	IsBot   bool
}

Classification is everything derived from a user agent.

func Classify 

func Classify(ua string) Classification

Classify buckets a user agent.

type CountryResolver 

type CountryResolver interface {
	Country(netip.Addr) string
}

CountryResolver turns an address into an ISO 3166-1 alpha-2 country code, or "" when it cannot.

An interface, not the geoip package, for two reasons: analytics should not depend on how geography is looked up, and a test needs to enrich events without a MaxMind database on disk.

type DayPoint 

type DayPoint struct {
	Day            string `json:"day"`
	Clicks         int64  `json:"clicks"`
	UniqueVisitors int64  `json:"unique_visitors"`
	BotClicks      int64  `json:"bot_clicks"`
}

DayPoint is one day in a time series.

type Device 

type Device string
const (
	DeviceDesktop Device = "desktop"
	DeviceMobile  Device = "mobile"
	DeviceTablet  Device = "tablet"
	DeviceBot     Device = "bot"
	DeviceUnknown Device = "unknown"
)

type DimensionValue 

type DimensionValue struct {
	Value          string `json:"value"`
	Clicks         int64  `json:"clicks"`
	UniqueVisitors int64  `json:"unique_visitors"`
}

DimensionValue is one bucket of a breakdown.

type Event 

type Event struct {
	LinkID      uuid.UUID
	WorkspaceID uuid.UUID
	OccurredAt  time.Time
	IP          netip.Addr
	UserAgent   string
	Referrer    string
	Language    string
	LatencyUS   int32
}

Event is a click as the redirect handler observes it.

The raw IP is present here and nowhere else. It is consumed to derive the visitor hash and then dropped; it is never written, never logged and never leaves this package.

type IngestConfig 

type IngestConfig struct {
	QueueSize     int
	BatchSize     int
	FlushInterval time.Duration
	Logger        *slog.Logger

	// Geo is optional. Nil leaves the country column null, which is the default
	// state: the MaxMind database cannot be shipped in the image.
	Geo CountryResolver
}

type Ingester 

type Ingester struct {
	Stats Stats
	// contains filtered or unexported fields
}

Ingester buffers click events and writes them in batches.

The contract that shapes everything else: recording a click must never delay a redirect and must never fail one. So the queue is bounded and Record drops rather than blocks — applying backpressure to the hot path would trade a complete analytics record for a slow site, which is the wrong way round. Drops are counted, and a non-zero counter is an alert rather than a silent gap.

func NewIngester 

func NewIngester(pool *pgxpool.Pool, salts *SaltCache, cfg IngestConfig) *Ingester

func (*Ingester) Close 

func (i *Ingester) Close(ctx context.Context) error

Close stops accepting events and flushes what is buffered.

Called during shutdown after the listener has closed, so no new events can arrive. Without this, every restart loses up to a full batch.

Signals through stop rather than closing ch. Record's closed check and its send cannot be one atomic step, so a redirect that passed the check just before Close ran would send on a closed channel and panic — killing the process during the one window where it is trying to save data. That is not hypothetical during a shutdown that times out with requests still in flight, which is exactly when Close is called. An event that races the drain is lost instead, which is what a full queue already does to it.

func (*Ingester) Counters 

func (i *Ingester) Counters() (enqueued, dropped, flushed, failed, batches int64)

Counters returns the lifetime totals, for the metrics collector.

One method returning all five rather than five accessors, so a scrape reads them in one call and the set cannot be sampled half a flush apart.

func (*Ingester) QueueDepth 

func (i *Ingester) QueueDepth() int

QueueDepth reports buffered events.

The leading indicator for the whole pipeline: depth climbing means the database is falling behind, minutes before drops start.

func (*Ingester) Record 

func (i *Ingester) Record(ev Event)

Record enqueues an event, or drops it.

Never blocks, never returns an error, never panics after Close. The default branch is the entire point: when the queue is full the event is discarded so the redirect returns on time.

func (*Ingester) Start 

func (i *Ingester) Start()

Start launches the batching goroutine.

type LinkStats 

type LinkStats struct {
	LinkID     uuid.UUID                   `json:"link_id"`
	From       string                      `json:"from"`
	To         string                      `json:"to"`
	Totals     Totals                      `json:"totals"`
	Series     []DayPoint                  `json:"series"`
	Dimensions map[string][]DimensionValue `json:"dimensions"`
	// Caveat travels with the data rather than living only in documentation,
	// so a client rendering these numbers can surface it.
	Caveat string `json:"caveat"`
}

LinkStats is a link's analytics over a window.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader serves dashboard queries.

Every query here reads a rollup table, never click_events, except the deliberately bounded recent-activity feed. That is what holds analytics under the 2s target once the raw table reaches tens of millions of rows.

func NewReader 

func NewReader(pool *pgxpool.Pool) *Reader

func (*Reader) LinkStats 

func (r *Reader) LinkStats(ctx context.Context, actor *auth.Identity, linkID uuid.UUID, from, to time.Time) (*LinkStats, error)

func (*Reader) Overview 

func (r *Reader) Overview(ctx context.Context, actor *auth.Identity, from, to time.Time) (*WorkspaceOverview, error)

func (*Reader) RecentClicks 

func (r *Reader) RecentClicks(ctx context.Context, actor *auth.Identity, linkID uuid.UUID, limit int32) ([]RecentClick, error)

RecentClicks reads raw events, bounded and index-backed.

The one query that touches click_events directly. Safe because it is capped and served by the (link_id, occurred_at DESC) index, so cost does not grow with table size.

type RecentClick 

type RecentClick struct {
	OccurredAt time.Time `json:"occurred_at"`
	Device     string    `json:"device,omitempty"`
	Browser    string    `json:"browser,omitempty"`
	OS         string    `json:"os,omitempty"`
	Country    string    `json:"country,omitempty"`
	Referrer   string    `json:"referrer,omitempty"`
	IsBot      bool      `json:"is_bot"`
}

RecentClick is one entry in the live activity feed.

type Roller 

type Roller struct {
	// contains filtered or unexported fields
}

Roller recomputes the pre-aggregated tables the dashboard reads.

func NewRoller 

func NewRoller(pool *pgxpool.Pool, log *slog.Logger) *Roller

func (*Roller) Run 

func (r *Roller) Run(ctx context.Context, from, to time.Time) error

Run recomputes rollups for [from, to).

Recomputation rather than incremental accumulation is deliberate. Each run derives whole days from the raw events and upserts, so a retry after a crash mid-run converges to the same numbers. An "add what arrived since the last watermark" design double-counts on every retry and drifts permanently once it does — and the drift is invisible until someone reconciles by hand.

func (*Roller) RunRecent 

func (r *Roller) RunRecent(ctx context.Context, now time.Time) error

RunRecent recomputes everything not known to be final, and at minimum yesterday and today.

Two days is the floor rather than the window because a click just before midnight UTC can be written just after it, and a run covering only today would miss it until the next pass. The upper end of the reopened window comes from job_state: a fixed two-day window meant that downtime spanning a UTC day left that day permanently unaggregated, because by the time the process came back the day was no longer "yesterday" and no run would ever look at it again. Recomputation makes reopening old days safe — each run derives whole days from raw events and upserts — so the watermark only ever decides how far back to start, never what the numbers are.

type SaltCache 

type SaltCache struct {
	// contains filtered or unexported fields
}

SaltCache resolves the salt for a UTC day, creating it on first use.

Cached in memory because every batch needs it and it changes once a day. The cache is small and bounded by retention, so it is a plain map rather than anything with eviction.

func NewSaltCache 

func NewSaltCache(pool *pgxpool.Pool) *SaltCache

func (*SaltCache) For 

func (c *SaltCache) For(ctx context.Context, day time.Time) ([]byte, error)

For returns the salt for a day, creating it if absent.

func (*SaltCache) Purge 

func (c *SaltCache) Purge(ctx context.Context) (int64, error)

Purge deletes salts past their retention.

This is the de-identification step, not housekeeping: once a salt is gone, the hashes it produced cannot be linked back to an address even by someone holding the original addresses.

type Stats 

type Stats struct {
	Enqueued atomic.Int64
	Dropped  atomic.Int64
	Flushed  atomic.Int64
	Failed   atomic.Int64
	Batches  atomic.Int64
}

Stats are the ingester's counters.

type Totals 

type Totals struct {
	Clicks         int64 `json:"clicks"`
	UniqueVisitors int64 `json:"unique_visitors"`
	BotClicks      int64 `json:"bot_clicks"`
}

type WorkspaceOverview 

type WorkspaceOverview struct {
	From   string     `json:"from"`
	To     string     `json:"to"`
	Totals Totals     `json:"totals"`
	Series []DayPoint `json:"series"`
	Caveat string     `json:"caveat"`
}

WorkspaceOverview is the dashboard summary.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.