Documentation
¶
Overview ¶
Package suspicion is the rule engine behind the Wanted List.
The design constraint that shapes everything here comes from the specification: every score must be explainable in plain language, and the engine stays rule-based rather than becoming a classifier. A finding nobody can check is a finding nobody should act on, and a tool that cries wolf gets ignored, at which point it is worse than absent, because it also provided false comfort.
Three consequences run through this package:
- A rule reports *facts*, not prose. It emits the numbers and names that make its case, and the dashboard turns them into a sentence in the viewer's language. No English is ever stored.
- A rule must be able to recognise its own earlier work, so that running every few minutes does not turn one event into a hundred findings.
- Scores are small and combine. A single weak signal should not reach the top of the list on its own; several together should.
Index ¶
- Constants
- func Clamp(score float64) float64
- func IsReportable(addr string) bool
- func RareScoreForTest(share float64) float64
- type Beaconing
- type DGADomain
- type Engine
- type FirstContact
- type Input
- type Observation
- type Plaintext
- type PortScan
- type Queryer
- type RareDestination
- type Rows
- type Rule
- type Sink
- type ThreatList
- type VolumeAnomaly
Constants ¶
const DefaultInterval = 5 * time.Minute
DefaultInterval is how often the rules run.
Comfortably shorter than the window, so overlapping coverage is guaranteed and nothing falls between two passes. Rules deduplicate their own findings, so seeing the same behaviour twice costs nothing.
const DefaultWindow = 2 * time.Hour
DefaultWindow is the period each pass examines.
const MinBaseline = 24 * time.Hour
MinBaseline is how much history must exist before rules that reason about "normal for this network" will say anything.
A day. Everything is unusual to a database that started an hour ago, and a Wanted List full of a user's ordinary evening is the fastest way to teach them the feature is noise.
Variables ¶
This section is empty.
Functions ¶
func IsReportable ¶
IsReportable reports whether a destination address is one worth reasoning about.
The endpoints table cannot be trusted for this on its own. Its `is_internal` flag is only set once enrichment has reached an address, so a destination with no row yet reads as external, and on the development database that put `127.0.0.1`, with 187 connections, at the top of the beaconing candidates. A machine talking to itself is not command-and-control.
So the address itself is checked, and the table is treated as an additional signal rather than the authority.
func RareScoreForTest ¶
RareScoreForTest exposes the scoring curve so its shape can be checked independently of any fixture's size, which is what caused two tests to disagree with a correct implementation.
Types ¶
type Beaconing ¶
type Beaconing struct{}
Beaconing notices a device contacting the same destination at a regular interval.
This is the rule that can actually catch command-and-control traffic, and it is also the rule most likely to be wrong in a way that ruins the product. A home network is full of things that talk on a timer: NTP, update checkers, push-notification keepalives, telemetry, a thermostat reporting temperature. A rule that reports all of them is a rule that gets switched off.
Three things separate a beacon from a heartbeat, and all three are required:
- **Regularity far beyond what human-driven traffic produces.** Software on a timer is regular; malware on a timer is *very* regular. The measure is robust to a single missed or delayed connection, because one hiccup should not clear an otherwise perfect rhythm.
- **An interval in a band that matters.** Below twenty seconds is a keepalive on an open session. Above the observation window there are not enough repetitions to call it a rhythm rather than a coincidence.
- **Enough repetitions to mean something.** Three connections at similar spacing is an accident; a dozen is a schedule.
Even then the finding says what it saw, this destination, this interval, this many times, and leaves the judgement legible. Plenty of legitimate software beacons, and the honest thing is to show the rhythm and let the user recognise their own thermostat.
type DGADomain ¶
type DGADomain struct{}
DGADomain notices a device looking up several machine-generated domain names that do not exist.
Malware that cannot hard-code its command server generates candidate names from a seed and tries them until one answers. The give-away is not any single name, it is a burst of failures across unrelated nonsense.
**Entropy alone would be useless here, and worse than useless.** Modern content delivery is full of legitimately random-looking hostnames: `d3n8a8pro7vhmx.cloudfront.net`, hashed bucket names, per-session subdomains. A rule that flagged high entropy would fire on ordinary browsing all day.
Three things are required together:
- **The registrable domain looks generated**, not the subdomain. CDN randomness lives in the labels *below* a recognisable parent; generated domains are random at the registrable level itself.
- **The lookup failed.** A name that resolves is somebody's real service, however odd it looks. A name that does not is a guess, and guessing is the whole technique.
- **Several of them, from one device, close together.** One failed lookup of a strange name is a typo or a dead link.
type Engine ¶
type Engine struct {
Rules []Rule
Sink Sink
DB Queryer
// Window is how far back each pass looks. Longer than the interval between
// passes, so a slow pass cannot leave a gap in what was examined.
Window time.Duration
}
Engine runs the rules.
func (*Engine) Run ¶
Run evaluates every rule once.
A rule that fails is logged and skipped. One broken rule must not stop the others: the engine's job is to notice things, and noticing fewer of them is better than noticing none.
func (*Engine) RunPeriodically ¶
func (e *Engine) RunPeriodically(ctx context.Context, every time.Duration, baselineAt func() time.Time)
RunPeriodically evaluates the rules until ctx is cancelled.
baselineAt reports when this install began observing, so rules that reason about what is normal here can stay silent until there is enough history to have an opinion.
type FirstContact ¶
type FirstContact struct{}
FirstContact notices a device talking to an organization it has never reached before.
The obvious version of this rule is useless. A laptop meets new organizations constantly (every website, every CDN, every advertising network) so "new organization" on its own would fire hundreds of times a day and be correctly ignored.
What makes it worth reporting is *who it happened to*. A thermostat that has spoken to two organizations in its life reaching a third is worth a look. A browser doing the same is Tuesday. So the score is a function of how unusual meeting somebody new is **for that particular device**, which is the network's own baseline rather than an assumption about what is normal.
func (FirstContact) Code ¶
func (FirstContact) Code() string
func (FirstContact) Evaluate ¶
func (r FirstContact) Evaluate(ctx context.Context, in Input) ([]Observation, error)
func (FirstContact) Weight ¶
func (FirstContact) Weight() float64
Weight is low. On its own this is a weak signal by design; it earns its place by combining with others on the same subject.
type Input ¶
type Input struct {
// DB is the read side of the store. Rules query; they never write.
DB Queryer
// Now is the end of the window under examination.
Now time.Time
// Window is how far back this pass looks.
Window time.Duration
// Baseline is how much history exists. A rule that needs to know what is
// normal here must stay silent until there is enough of it to say.
Baseline time.Duration
}
Input is what a rule is given.
type Observation ¶
type Observation struct {
// Subject is a device ID or an endpoint address.
Subject string
SubjectType string
// Score is this observation's contribution, from 0 to 1.
Score float64
// Detail carries the facts the explanation is built from, a count, an
// interval, an organization name. Everything the sentence needs.
Detail map[string]any
// Dedup identifies *this* finding as distinct from other findings by the
// same rule. Two observations sharing a dedup key are the same finding seen
// again, not a new one.
Dedup string
// At is when the behaviour was observed, which is not always now: a rule
// examining a window reports when the thing happened.
At time.Time
}
Observation is one rule's claim about one subject.
Deliberately not a sentence. The rule supplies the evidence; the interface supplies the words, because the server cannot know what language the reader uses and English written into a database is untranslatable afterwards.
type Plaintext ¶
type Plaintext struct{}
Plaintext notices a device sending credentials or mail over the internet without encryption.
**Port 80 is deliberately not in this list**, which is the whole design decision. Ordinary browsing produces plain HTTP constantly, certificate status checks, redirects to HTTPS, captive-portal probes, ad networks, and a rule that flagged it would fire hundreds of times a day on a healthy network. It would also be nearly useless advice, since the user cannot do anything about somebody else's redirect.
What is left is the short list where plaintext genuinely means credentials or private mail crossing the internet in the clear, and where the answer is actionable: stop using that service, or configure it for TLS.
type PortScan ¶
type PortScan struct{}
PortScan notices a device probing many ports on one host, or one port across many hosts.
Two shapes, both worth knowing about:
- **Vertical**: one destination, many ports. Somebody asking what a particular machine runs.
- **Horizontal**: one port, many destinations. Somebody asking which machines run a particular thing, how a worm spreads.
**What separates a scan from ordinary software is that a scan is mostly refused.** A backup client, an FTP client, a stream deck, all touch a surprising number of ports, and all of them *connect*. A scan is a series of questions where most answers are "nothing here". Without that test the rule reports FileZilla.
It also has to exclude this application. LAN Sheriff's own on-demand port check probes thirty-five ports on one host, which is precisely the vertical shape above, on the development machine it appears as 85 flows to a single destination across 85 ports. A monitor that reports itself is not a monitor.
type Queryer ¶
type Queryer interface {
QueryContext(ctx context.Context, query string, args ...any) (Rows, error)
}
Queryer is the narrow slice of the database a rule may use.
Read-only by construction: a rule that could write would be able to influence the evidence the next rule sees, and the order rules ran in would start to matter.
type RareDestination ¶
type RareDestination struct{}
RareDestination notices traffic to a part of the internet this network essentially never uses.
The distinction from first contact matters. First contact asks "has this device met this organization before"; this asks "does this *network* go here at all". A household has a shape, a handful of countries, a few dozen hosting providers, and something reaching outside that shape is worth a glance even when the specific organization is new to nobody.
The comparison is always against this network's own history. There is no list of suspicious countries and there will not be one: a Canadian household and a Japanese one have different normal, and a rule that shipped somebody's idea of a risky region would be both wrong and offensive.
func (RareDestination) Code ¶
func (RareDestination) Code() string
func (RareDestination) Evaluate ¶
func (r RareDestination) Evaluate(ctx context.Context, in Input) ([]Observation, error)
func (RareDestination) Weight ¶
func (RareDestination) Weight() float64
Weight sits between first contact and beaconing. Rare is meaningful but not damning, people travel, buy things abroad, and use services with unusual hosting.
type Rule ¶
type Rule interface {
// Code is the stable identifier stored with every finding and used as the
// translation key. Renaming one rewrites history, so it does not change.
Code() string
// Weight scales this rule's observations when they are combined into a
// subject's overall suspicion. A rule that fires often on ordinary traffic
// carries less weight than one that almost never does.
Weight() float64
// Evaluate looks at the window ending at now and reports what it found.
Evaluate(ctx context.Context, in Input) ([]Observation, error)
}
Rule is one thing worth noticing.
type Sink ¶
type Sink interface {
// RecordObservations writes a rule's findings, updating rather than
// duplicating anything it has raised before.
RecordObservations(ctx context.Context, rule string, weight float64, obs []Observation) error
}
Sink is where findings go. The store implements it.
type ThreatList ¶
type ThreatList struct{}
ThreatList notices a device looking up a domain that appears on a published list of known-malicious names.
This is the most direct rule in the set, and the only one that is not statistical: every other rule reasons about what is normal *on this network* and needs history before it can say anything. A domain that somebody else has already identified as a malware command server does not become more malicious by being unusual here. So this rule needs no baseline at all and can fire on the first day.
**It does need to see DNS lookups, which in practice means Patrol Mode.** Deputy Mode reads socket tables and reports a DNS feed only when this machine is itself serving DNS. On an ordinary Deputy install `dns_events` is empty and this rule is silent, not because nothing matched, but because there is nothing to match against. That is worth stating plainly rather than leaving a user to conclude their network is clean.
**Only the malware category fires.** The same blocklist infrastructure labels advertising, tracking and telemetry domains, and it would be trivial to raise findings for those too. It would also be a mistake. An ordinary browser touches dozens of tracker domains an hour; a Wanted List that reported them would be a list of ordinary web browsing, and the one entry that mattered would be buried in it. Ad and tracker labels are shown inline in Radio Chatter where they are useful context, and are not findings.
**A failed lookup still counts.** If the name did not resolve, because the domain has been taken down, or because something upstream is already blocking it, the device still asked for it. The question this rule answers is "did something here try to reach a known-bad host", and a query is the attempt. Whether it succeeded changes the urgency, not the fact.
**The label is applied when the lookup is recorded, not when the rule runs.** Lookups observed before the lists were first fetched carry no label and are invisible here. That gap is bounded by the first successful fetch after install, and is preferable to re-labelling the entire history on every pass.
func (ThreatList) Code ¶
func (ThreatList) Code() string
func (ThreatList) Evaluate ¶
func (r ThreatList) Evaluate(ctx context.Context, in Input) ([]Observation, error)
func (ThreatList) Weight ¶
func (ThreatList) Weight() float64
Weight is the highest of any rule. This is not an inference from behaviour, it is a name-for-name match against a list of hosts that other people have already caught doing harm.
type VolumeAnomaly ¶
type VolumeAnomaly struct{}
VolumeAnomaly notices a device doing far more than it usually does.
The measure is the device's own history, never a fixed threshold. A media server and a doorbell have nothing in common except that a tenfold change in either is worth a glance.
**Counted in connections, not bytes.** Deputy Mode reads socket tables, which carry no byte counters, so a bytes-based rule would be silent on most installs , the same reason the "busiest device" widget counts connections.
Two things keep it from firing on ordinary life:
- **Robust statistics.** Median and median absolute deviation, not mean and standard deviation. A device's traffic is spiky by nature, and a single large hour in the history would inflate a mean-based threshold enough to hide everything afterwards.
- **Enough history to know the rhythm.** A laptop is quiet at four in the morning and busy at nine. Comparing one hour against a baseline that has not yet seen a full daily cycle would report every morning as an anomaly.
func (VolumeAnomaly) Code ¶
func (VolumeAnomaly) Code() string
func (VolumeAnomaly) Evaluate ¶
func (r VolumeAnomaly) Evaluate(ctx context.Context, in Input) ([]Observation, error)
func (VolumeAnomaly) Weight ¶
func (VolumeAnomaly) Weight() float64
Weight is moderate. A busy hour is often a large download, and the finding is most useful in combination with something else about the same device.
Source Files