parser

Documentation

Overview

Package parser contains the core detection machinery shared by all stages of devicedetector: anchored regex matching (a faithful port of the upstream PHP matching semantics on top of the regexp2 engine), YAML database loading with order-preserving maps, version building and truncation, plus the bot, operating-system and vendor-fragment parsers.

Most applications should use the root devicedetector package instead of this one.

Index

• Constants
• func BuildByMatch(item string, matches []string) string
• func BuildVersion(versionString string, matches []string, truncation int) string
• func CombineRegexes(patterns []string) string
• func FuzzyCompare(a, b string) bool
• func GateStats() (gated, total int)
• func HasDesktopFragment(ua string) bool
• func HasUserAgentClientHintsFragment(ua string) bool
• func IsDesktopOS(osName string) bool
• func Load[T any](fsys fs.FS, name string, out *T) error
• func MatchTimeout() time.Duration
• func MatchUserAgent(ua, pattern string) ([]string, error)
• func OSFamily(osLabel string) (string, bool)
• func OSNameFromID(short, version string) (string, bool)
• func OSShortName(name string) (string, bool)
• func PreMatchEmpty(combined string) bool
• func SetMatchTimeout(d time.Duration)
• func StampTimeout(re *regexp2.Regexp)
• type Bot
  • func NewBot(fsys fs.FS) (*Bot, error)
  • func (b *Bot) IsBot(ua string) (bool, error)
  • func (b *Bot) Parse(ua string) (*BotResult, error)
• type BotProducer
• type BotResult
• type BrandVersion
• type ClientHints
  • func NewClientHintsFromHeaders(h http.Header) *ClientHints
  • func NewClientHintsFromMap(headers map[string]any) *ClientHints
  • func (c *ClientHints) BrandList() []BrandVersion
  • func (c *ClientHints) BrandVersion() string
  • func (c *ClientHints) IsMobile() bool
  • func (c *ClientHints) RestoreUserAgent(ua string) string
• type Compiled
  • func Compile(pattern string) (*Compiled, error)
  • func (c *Compiled) Full() *regexp2.Regexp
• type GateSet
  • func CompileGateSet(patterns []string) *GateSet
  • func (g *GateSet) SkipGated(ua string) ([]int, bool)
• type OS
  • func NewOS(fsys fs.FS) (*OS, error)
  • func (o *OS) Parse(ua string, hints *ClientHints) (*OSResult, error)
  • func (o *OS) SetVersionTruncation(t int)
• type OSResult
• type OrderedEntry
• type OrderedMap
  • func (m *OrderedMap[T]) UnmarshalYAML(node *yaml.Node) error
• type VendorFragment
  • func NewVendorFragment(fsys fs.FS) (*VendorFragment, error)
  • func (v *VendorFragment) Parse(ua string) (brand, matchedRegex string, err error)

Constants

const (
	VersionTruncationMajor = 0
	VersionTruncationMinor = 1
	VersionTruncationPatch = 2
	VersionTruncationBuild = 3
	VersionTruncationNone  = -1
)

Version truncation levels, mirroring AbstractParser::VERSION_TRUNCATION_*.

const DefaultMatchTimeout = time.Second

DefaultMatchTimeout is the per-match backstop stamped onto every compiled pattern. regexp2 is a backtracking engine, and the database mirrors the upstream design of one large alternation per file; without a bound, a crafted user agent can pin a core for tens of seconds. A single legitimate match never approaches this, so the value only ever fires on pathological input. Tune with SetMatchTimeout before constructing detectors.

Functions

func BuildByMatch

func BuildByMatch(item string, matches []string) string

BuildByMatch substitutes $1..$N placeholders in item with capture groups, replicating AbstractParser::buildByMatch() exactly — including its sequential-replacement semantics ($1 is substituted before $10).

func BuildVersion

func BuildVersion(versionString string, matches []string, truncation int) string

BuildVersion expands placeholders in versionString, normalizes underscores to dots and truncates to the requested precision, mirroring AbstractParser::buildVersion().

func CombineRegexes

func CombineRegexes(patterns []string) string

CombineRegexes builds the AbstractParser::preMatchOverall() alternation: all patterns reversed (generic entries last in the file match most UAs) and joined with '|'.

func FuzzyCompare

func FuzzyCompare(a, b string) bool

FuzzyCompare reports whether two strings are equal ignoring case and spaces, mirroring AbstractParser::fuzzyCompare().

func GateStats

func GateStats() (gated, total int)

GateStats reports how many cached patterns carry an RE2 gate. Diagnostic only (used by tests and benchmarks to prove coverage).

func HasDesktopFragment

func HasDesktopFragment(ua string) bool

HasDesktopFragment mirrors AbstractParser::hasDesktopFragment(): the UA carries a desktop OS fragment and none of the known mobile/TV markers.

func HasUserAgentClientHintsFragment

func HasUserAgentClientHintsFragment(ua string) bool

HasUserAgentClientHintsFragment reports whether ua carries a frozen Android client-hints fragment, mirroring AbstractParser::hasUserAgentClientHintsFragment. Telegram-Android reuses the same shape and is excluded.

func IsDesktopOS

func IsDesktopOS(osName string) bool

IsDesktopOS reports whether the OS (given by short code or name) belongs to a desktop-only family, mirroring OperatingSystem::isDesktopOs.

func Load

func Load[T any](fsys fs.FS, name string, out *T) error

Load reads and decodes a YAML file from the regex database.

func MatchTimeout

func MatchTimeout() time.Duration

MatchTimeout returns the current per-match timeout (0 if disabled).

func MatchUserAgent

func MatchUserAgent(ua, pattern string) ([]string, error)

MatchUserAgent matches ua against a database pattern and returns the capture groups PHP-style: index 0 is the full match, unmatched groups are empty strings. Returns nil when there is no match.

An empty pattern matches every user agent, mirroring PHP preg_match on an empty alternation; the database relies on this for catch-all model regexes. Callers that build a preMatchOverall regex must reject the empty-list case themselves via PreMatchEmpty before matching.

func OSFamily

func OSFamily(osLabel string) (string, bool)

OSFamily returns the OS family for the given label, which may be either a short code or a full OS name, mirroring OperatingSystem::getOsFamily. The boolean is false when the OS has no known family ("Unknown" in the PHP).

func OSNameFromID

func OSNameFromID(short, version string) (string, bool)

OSNameFromID returns the full OS name for a short code with an optional version appended, mirroring OperatingSystem::getNameFromId. The boolean is false when the short code is unknown.

func OSShortName

func OSShortName(name string) (string, bool)

OSShortName returns the short code for an OS full name, mirroring array_search over the operatingSystems map (first match, case-sensitive).

func PreMatchEmpty

func PreMatchEmpty(combined string) bool

PreMatchEmpty reports whether a combined preMatchOverall regex is empty and must therefore be treated as no-match. matomo/device-detector 6.5.1 (PR #8271) added this guard so an empty regex list no longer degrades into a bare anchor that matches every user agent. Individual empty patterns keep their catch-all semantics; only the combined-list case is guarded.

func SetMatchTimeout

func SetMatchTimeout(d time.Duration)

SetMatchTimeout sets the per-match timeout applied to patterns compiled afterwards; d <= 0 disables it. The regex cache is process-wide and a pattern keeps the timeout in effect when it was first compiled, so call this once at startup, before constructing detectors, for a uniform bound.

func StampTimeout

func StampTimeout(re *regexp2.Regexp)

StampTimeout applies the current match timeout to re. Call before publishing a regexp for concurrent matching; the field is then read-only and race-free.

Types

type Bot

type Bot struct {
	// contains filtered or unexported fields
}

Bot parses a user agent for bot information, mirroring DeviceDetector's Parser\Bot. It is immutable after construction and safe for concurrent use.

func NewBot

func NewBot(fsys fs.FS) (*Bot, error)

NewBot loads bots.yml from fsys, precompiles every entry regex and builds the combined "overall" regex used to short-circuit non-bot user agents.

func (*Bot) IsBot

func (b *Bot) IsBot(ua string) (bool, error)

IsBot reports whether ua belongs to a bot without collecting its details, mirroring Parser\Bot with discardDetails enabled: only the combined regex is evaluated.

func (*Bot) Parse

func (b *Bot) Parse(ua string) (*BotResult, error)

Parse checks whether ua belongs to a bot and returns its details. It returns (nil, nil) when the user agent is not a known bot.

The detection first tests the combined regex (a fast rejection for the common non-bot case) and only then walks the individual entries in file order.

type BotProducer

type BotProducer struct {
	Name string `yaml:"name"`
	URL  string `yaml:"url"`
}

BotProducer identifies the organisation operating a bot.

type BotResult

type BotResult struct {
	Name     string
	Category string
	URL      string
	Producer BotProducer
}

BotResult is the outcome of a successful bot detection.

type BrandVersion

type BrandVersion struct {
	Brand   string
	Version string
}

BrandVersion is one entry of a Sec-CH-UA brand/version list.

type ClientHints

type ClientHints struct {
	Architecture    string
	Bitness         string
	Mobile          bool
	Model           string
	Platform        string
	PlatformVersion string
	UAFullVersion   string
	FullVersionList []BrandVersion
	App             string
	FormFactors     []string
}

ClientHints holds the parsed HTTP Client Hints for a request, mirroring DeviceDetector\ClientHints. The zero value carries no hints.

func NewClientHintsFromHeaders

func NewClientHintsFromHeaders(h http.Header) *ClientHints

NewClientHintsFromHeaders builds ClientHints from HTTP request headers.

func NewClientHintsFromMap

func NewClientHintsFromMap(headers map[string]any) *ClientHints

NewClientHintsFromMap builds ClientHints from a header/value map. Values may be strings (HTTP headers) or structured values from navigator.userAgentData (bool mobile, []string form factors, a brand/version list). It mirrors ClientHints::factory, including its strict per-field type handling.

func (*ClientHints) BrandList

func (c *ClientHints) BrandList() []BrandVersion

BrandList returns the brand/version pairs, de-duplicated by brand keeping the first position and the last version (mirroring PHP array_combine).

func (*ClientHints) BrandVersion

func (c *ClientHints) BrandVersion() string

BrandVersion returns the Sec-CH-UA-Full-Version hint (getBrandVersion).

func (*ClientHints) IsMobile

func (c *ClientHints) IsMobile() bool

IsMobile reports the Sec-CH-UA-Mobile hint.

func (*ClientHints) RestoreUserAgent

func (c *ClientHints) RestoreUserAgent(ua string) string

RestoreUserAgent rebuilds a frozen user agent from the device model reported via client hints, mirroring AbstractParser::restoreUserAgentFromClientHints. It returns ua unchanged when there is nothing to restore.

type Compiled

type Compiled struct {
	// contains filtered or unexported fields
}

Compiled pairs the authoritative regexp2 pattern with its optional RE2 prefilter gate (nil when the pattern cannot be gated — see prefilter.go).

func Compile

func Compile(pattern string) (*Compiled, error)

Compile compiles a raw regex from the database wrapped with the standard user-agent anchor, case-insensitively. Compiled patterns are cached process-wide and carry the current match timeout.

An empty pattern deliberately matches every user agent: the database uses empty model regexes as catch-alls (e.g. Roku's "Digital Video Player"). The empty-list guard for preMatchOverall lives in its callers, not here — see PreMatchEmpty.

func (*Compiled) Full

func (c *Compiled) Full() *regexp2.Regexp

Full exposes the underlying regexp2 for callers that manage matching themselves (timeout stamping, direct FindStringMatch).

type GateSet

type GateSet struct {
	// contains filtered or unexported fields
}

GateSet narrows an entry walk (device brands, OS rules, browsers — parsers with no upstream preMatchOverall) using required literals. For each entry a set of case-folded literals is extracted from its regex AST such that a match implies at least one literal is present in the user agent. At parse time one lowercased copy of the UA and a substring probe per distinct literal shrink the walk to the entries whose literal actually occurs (plus the few entries no literal set could be proven for). This is what collapses the aggregate walk on junk input — and most of the walk on real traffic — while remaining a strict superset of the regexp2 match semantics.

(A single combined RE2 union was tried first and was slower than the per-pattern walk it replaced: Go's regexp simulates an NFA, so one pass over a ~20k-state union costs more than thousands of memchr-accelerated substring probes.)

func CompileGateSet

func CompileGateSet(patterns []string) *GateSet

CompileGateSet builds a literal gate over patterns; entries whose regex yields no usable literal set are recorded as always-walk.

func (*GateSet) SkipGated

func (g *GateSet) SkipGated(ua string) ([]int, bool)

SkipGated reports whether the walk can be narrowed for ua. When it returns (only, true), entries outside `only` cannot match and the caller walks just those indexes (sorted, possibly empty). (nil, false) means walk everything.

type OS

type OS struct {
	// contains filtered or unexported fields
}

OS parses a user agent for operating system information, mirroring DeviceDetector's Parser\OperatingSystem. Apart from SetVersionTruncation (intended to be called during setup, before any Parse), it is immutable and safe for concurrent use.

func NewOS

func NewOS(fsys fs.FS) (*OS, error)

NewOS loads oss.yml from fsys and precompiles every regex. Version truncation defaults to VersionTruncationMinor, matching the PHP default.

func (*OS) Parse

func (o *OS) Parse(ua string, hints *ClientHints) (*OSResult, error)

Parse detects the operating system from ua, optionally refined by client hints. It returns (nil, nil) when no OS can be determined.

func (*OS) SetVersionTruncation

func (o *OS) SetVersionTruncation(t int)

SetVersionTruncation sets how deep version numbers are reported. It accepts any of the VersionTruncation* constants and ignores anything else, mirroring AbstractParser::setVersionTruncation. Call it during setup, not concurrently with Parse.

type OSResult

type OSResult struct {
	Name      string
	ShortName string
	Version   string
	Platform  string
	Family    string
}

OSResult is the operating system detected from a user agent.

type OrderedEntry

type OrderedEntry[T any] struct {
	Key   string
	Value T
}

OrderedEntry is a key/value pair of an OrderedMap.

type OrderedMap

type OrderedMap[T any] struct {
	Entries []OrderedEntry[T]
}

OrderedMap decodes a YAML mapping while preserving document order. Several database files (notably the device brand map) rely on entry order for correct matching, which a plain Go map would destroy.

func (*OrderedMap[T]) UnmarshalYAML

func (m *OrderedMap[T]) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML implements yaml.Unmarshaler.

type VendorFragment

type VendorFragment struct {
	// contains filtered or unexported fields
}

VendorFragment maps device vendor fragments in a user agent to a brand, mirroring DeviceDetector's Parser\VendorFragment. It is immutable after construction and safe for concurrent use.

func NewVendorFragment

func NewVendorFragment(fsys fs.FS) (*VendorFragment, error)

NewVendorFragment loads vendorfragments.yml from fsys, preserving brand order, and precompiles every fragment regex with the trailing "[^a-z0-9]+" guard the PHP applies at match time.

func (*VendorFragment) Parse

func (v *VendorFragment) Parse(ua string) (brand, matchedRegex string, err error)

Parse returns the brand whose fragment matches ua, plus the raw regex that matched (mirroring getMatchedRegex). Both are empty when nothing matches.