store

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2026 License: AGPL-3.0 Imports: 22 Imported by: 0

Documentation

Overview

Package store is the bounded, self-pruning embedded database every view reads from. It uses SQLite through a pure-Go driver so the default build stays cgo-free.

Index

Constants

View Source 
const (
	FindingOpen    = "open"
	FindingCleared = "cleared"
	FindingTrusted = "trusted"
)

Finding statuses.

View Source 
const (
	// PeerTrusted merges and displays the peer's data.
	PeerTrusted = "trusted"
	// PeerSuspended keeps the pairing and the connection, and stops believing
	// anything the peer says. Unpairing would mean losing the ability to watch a
	// machine at exactly the moment it became interesting.
	PeerSuspended = "suspended"
)

Peer trust states.

View Source 
const (
	RollupEndpoint = "endpoint"
	RollupOrg      = "org"
	RollupDevice   = "device"
	RollupProcess  = "process"
	RollupCountry  = "country"
)

RollupKind identifies what a rollup row counts.

View Source 
const ActiveWithin = 2 * time.Minute

ActiveWithin is how recently a flow must have been seen to count as live.

**The stored `active` flag is set and never cleared.** A capture source marks a flow active while it can see it, and when the connection closes the flow simply stops being reported: nothing goes back and writes active = 0. So the flag means "was open at some point", and reading it as "is open now" produced a headline that said 176,838 live connections on a laptop where 238 flows had been seen in the last minute, and drew 91% of the world map as live traffic including destinations last touched three days earlier.

Recency is the honest signal, because ts_last is updated every time a source sees the flow. Two minutes against a two-second poll is sixty chances to be counted, so an open connection is not going to be missed, and a closed one drops off quickly enough that the number means what a reader thinks it means.

Deliberately shorter than OfflineAfter, which is five minutes for a device. A device that has said nothing for four minutes is very likely still on the network; a connection that has carried nothing for four minutes is very likely finished.

The alternative was a background pass writing active = 0, as MarkStaleDevicesOffline does for devices. Deriving it at read time cannot drift, needs no migration, and corrects every existing database on the next query rather than only after a pass has run.

View Source 
const DefaultLimit = 500

DefaultLimit caps a query that did not ask for one, so a filter that matches everything cannot pull the whole database into memory.

View Source 
const ErrFindingNotFound = findingError("finding not found")

ErrFindingNotFound is returned when a status change names a finding that does not exist.

View Source 
const ExportCeiling = 50000

ExportCeiling is the most rows any export returns. Large enough that a normal day of a normal network comes out whole, bounded so that one request cannot read an entire year into memory. When a result reaches it the file says so in its own name, because a truncated export that looks complete is the kind of wrong answer nobody checks.

View Source 
const FeedCeiling = 2000

FeedCeiling is the most rows the live feed will return however large a limit is asked for. It protects the endpoint from a caller asking for a million.

View Source 
const FindingTTL = 7 * 24 * time.Hour

FindingTTL is how long a finding stays open without being seen again.

View Source 
const GlanceWindow = 24 * time.Hour

GlanceWindow is the period the tally covers.

A day rather than an hour: the point is "since I last looked", and most people do not look hourly. It also spans a night, which is when a household network is most revealing about what runs without anybody present.

View Source 
const OfflineAfter = 5 * time.Minute

OfflineAfter is how long a device may go unseen before the Roster stops calling it online.

Comfortably longer than the neighbour poll and than the interval at which devices re-announce themselves, so a device is not flickering between states because one poll happened to miss it. A phone that genuinely leaves the house shows as offline within a few minutes.

View Source 
const ReasonManual = "manual"

ReasonManual marks a device type the user chose, as opposed to one inferred.

View Source 
const (
	// RuleNewDevice fires when a device appears that this network has not seen
	// before.
	RuleNewDevice = "new_device"
)

Rule codes. Stable identifiers, the dashboard translates them, and they are stored, so renaming one changes historical data.

View Source 
const ScreenCeiling = 5000

ScreenCeiling is the most rows a screen will be given however large a limit is asked for, so one request cannot read the whole database into memory.

Variables

View Source 
var ErrDeviceNotFound = errors.New("device not found")

ErrDeviceNotFound is returned when an edit names a device that does not exist.

View Source 
var ErrPeerSuspended = errors.New("store: peer is suspended")

ErrPeerSuspended is returned when a suspended peer reports data.

View Source 
var ErrUnknownPeer = errors.New("store: peer is not paired")

ErrUnknownPeer is returned when a write names a peer that is not paired.

Functions

This section is empty.

Types

type Count 

type Count struct {
	Key   string `json:"key"`
	Label string `json:"label,omitempty"`
	N     int64  `json:"n"`
}

Count is a label with a tally.

type DNSOptions 

type DNSOptions struct {
	Since  time.Time
	Until  time.Time
	Device string
	// Domain matches the queried name, as a substring.
	Domain string
	// FlaggedOnly restricts results to names that matched a labelling list.
	FlaggedOnly bool
	Limit       int

	// Export lifts the feed ceiling. Set only by the export handler.
	Export bool
}

DNSOptions filters the DNS feed.

type DNSStats 

type DNSStats struct {
	Lookups    int64 `json:"lookups"`
	Domains    int64 `json:"domains"`
	NewDomains int64 `json:"new_domains"`
	Flagged    int64 `json:"flagged"`
	Devices    int64 `json:"devices"`
}

DNSStats is the header summary for Radio Chatter.

type DeviceEdit 

type DeviceEdit struct {
	Trust      *string
	Label      *string
	Notes      *string
	DeviceType *string
}

DeviceEdit is what a user may change about a device.

Every field is a pointer so that "not supplied" is distinct from "set to empty". Clearing a label is a real instruction and must not be confused with a request that simply did not mention it.

type DomainSummary 

type DomainSummary struct {
	Domain    string `json:"domain"`
	Lookups   int64  `json:"lookups"`
	Devices   int64  `json:"devices"`
	FirstSeen int64  `json:"first_seen"`
	LastSeen  int64  `json:"last_seen"`
	Flagged   string `json:"flagged,omitempty"`
	// New marks a domain first seen inside the window being viewed, which is
	// what makes "something started talking to somewhere new" visible.
	New bool `json:"new"`
}

DomainSummary is one row of the top-domains aggregate.

type EgressRow 

type EgressRow struct {
	types.Endpoint
	Conns     int      `json:"conns"`
	BytesOut  uint64   `json:"bytes_out"`
	BytesIn   uint64   `json:"bytes_in"`
	Processes []string `json:"processes,omitempty"`
	Devices   []string `json:"devices,omitempty"`
	Ports     []int    `json:"ports,omitempty"`
	Active    bool     `json:"active"`
	LastFlow  int64    `json:"last_flow"`
}

EgressRow is one external destination, aggregated across its flows: what the Watchtower draws.

type EndpointSighting 

type EndpointSighting struct {
	Internal bool
	Seen     time.Time
}

EndpointSighting is one observation of an address.

func Sighting 

func Sighting(internal bool, seen time.Time) EndpointSighting

Sighting builds an endpoint sighting.

type Filter 

type Filter struct {
	Since time.Time
	Until time.Time

	Device  string
	Process string
	Country string
	Org     string
	Proto   types.Proto
	Port    int

	// Direction filters by who opened the connection. Empty means any.
	Direction types.Direction

	// ActiveOnly restricts results to flows still open.
	ActiveOnly bool

	// Search is free text matched against address, hostname, organization and
	// owning application.
	Search string

	Limit int

	// Export lifts the screen ceiling. Set only by the export handler.
	Export bool
}

Filter is the one filter model every view shares.

Having a single struct rather than per-endpoint options means a filter set in the Watchtower carries into Radio Chatter and the Roster unchanged, which is the whole point of "one engine, many views": the user narrows once.

type Finding 

type Finding struct {
	ID      int64     `json:"id"`
	TS      time.Time `json:"ts"`
	Subject string    `json:"subject"`
	// SubjectType is "device" or "endpoint".
	SubjectType string  `json:"subject_type"`
	Rule        string  `json:"rule"`
	Score       float64 `json:"score"`
	// Detail carries rule-specific facts as JSON, so the UI can explain the
	// finding in the viewer's language rather than storing English prose.
	Detail map[string]any `json:"detail,omitempty"`
	Status string         `json:"status"`
	// Label is the subject's display name, resolved at read time so a device
	// renamed after the finding was raised shows its current name.
	Label string `json:"label,omitempty"`
}

Finding is one recorded observation worth surfacing.

type Glance 

type Glance struct {
	// NewOrgs is organizations contacted for the first time within the window.
	NewOrgs int64 `json:"new_orgs"`
	// NewDevices is devices seen for the first time within the window.
	NewDevices int64 `json:"new_devices"`

	// Loudest names the device accounting for the most connections in the
	// window, which on most networks is not the one people expect.
	LoudestDevice string `json:"loudest_device,omitempty"`
	LoudestID     string `json:"loudest_id,omitempty"`
	LoudestConns  int64  `json:"loudest_conns,omitempty"`

	// QuietestHour is the hour of the local day with the least traffic, over a
	// longer window than the rest, a single day says nothing about habit.
	// Negative means not enough history to say.
	QuietestHour  int   `json:"quietest_hour"`
	QuietestConns int64 `json:"quietest_conns"`

	// DevicesOnline and DevicesKnown put the roster in one line.
	DevicesOnline int64 `json:"devices_online"`
	DevicesKnown  int64 `json:"devices_known"`

	// Window is how far back the "new" and "loudest" figures look.
	Window string `json:"window"`
}

Glance is the sidebar summary.

type KeyKind 

type KeyKind string

KeyKind names an identity key's type, which is also its confidence ranking. 

const (
	// KeyMAC is a manufacturer-assigned hardware address: the strongest key,
	// unchanged by DHCP, reboots or reinstalls.
	KeyMAC KeyKind = "mac"
	// KeyRandomMAC is a randomized hardware address. Stable on this network, but
	// it rotates if the user toggles the setting or the OS re-derives it.
	KeyRandomMAC KeyKind = "rmac"
	// KeyHostname is a device's claimed name. Weaker than an address because it
	// is user-editable and can collide, but it is what re-identifies a device
	// that has rotated its randomized address.
	KeyHostname KeyKind = "host"
)

type PairingEvent 

type PairingEvent struct {
	At    time.Time `json:"at"`
	Peer  string    `json:"peer_id"`
	Label string    `json:"label,omitempty"`
	Event string    `json:"event"`
	Addr  string    `json:"addr,omitempty"`
}

PairingEvent is one entry in the ledger.

type Peer 

type Peer struct {
	PeerID    string    `json:"peer_id"`
	PublicKey []byte    `json:"-"` // never leaves the process
	Label     string    `json:"label,omitempty"`
	Trust     string    `json:"trust"`
	PairedAt  time.Time `json:"paired_at"`
	LastSeen  time.Time `json:"last_seen"`
	LastAddr  string    `json:"last_addr,omitempty"`
	ClockSkew int       `json:"clock_skew_secs"`
}

Peer is a paired instance.

type PeerDestination 

type PeerDestination struct {
	PeerID  string  `json:"peer_id"`
	Label   string  `json:"label,omitempty"`
	Device  string  `json:"device"`
	Org     string  `json:"org,omitempty"`
	Country string  `json:"country,omitempty"`
	ASN     int     `json:"asn,omitempty"`
	App     string  `json:"app,omitempty"`
	Flows   int64   `json:"flows"`
	Bytes   int64   `json:"bytes"`
	Lat     float64 `json:"lat,omitempty"`
	Lon     float64 `json:"lon,omitempty"`

	// LastHour is the most recent hour this peer reported the destination in.
	//
	// Peer data is aggregated into hourly buckets, so this is an hour rather
	// than a moment, and it is the honest resolution: claiming a timestamp
	// would imply a precision the protocol deliberately does not carry.
	//
	// It was missing entirely, and its absence was the whole problem: a
	// destination from a peer sat in the list beside local ones with no way to
	// tell whether it happened minutes ago or was the last thing that peer
	// managed to send before it went offline yesterday.
	LastHour time.Time `json:"last_hour,omitempty"`
}

PeerDestination is one organization a peer has reached, aggregated for display.

Deliberately not a `types.Endpoint`: an endpoint is an address, and peer data has none. Giving this the same shape would invite code that treats the two alike and then reaches for a field that is always empty.

type PeerDevice 

type PeerDevice struct {
	PeerID string `json:"peer_id"`
	Label  string `json:"label"`  // the peer's name, not the device's
	Device string `json:"device"` // the peer's own identifier for it
	ID     string `json:"id"`     // namespaced, matches the topology node

	Orgs     int64  `json:"orgs"`
	Flows    int64  `json:"flows"`
	Bytes    int64  `json:"bytes"`
	LastHour int64  `json:"last_hour"`
	TopOrg   string `json:"top_org,omitempty"`
	TopApp   string `json:"top_app,omitempty"`
}

PeerDevice is one device belonging to a peer, as far as this machine can know it: a name, who reported it, and what it has been talking to.

Deliberately not a types.Device. That type carries a hardware address, a vendor, and the services a device advertises, none of which a peer sends. A shared type would produce rows of empty columns, which reads as a lookup that failed rather than as detail that was never transmitted.

type PeerSummary 

type PeerSummary struct {
	PeerID  string
	Device  string
	Hour    int64
	Org     string
	Country string
	ASN     int
	App     string
	Proto   string
	Port    uint16

	Flows    int64
	BytesOut int64
	BytesIn  int64
}

PeerSummary is one merged bucket, as stored.

type PruneStats 

type PruneStats struct {
	Flows     int64
	DNSEvents int64
	Endpoints int64
	Rollups   int64
	Findings  int64
	Bytes     int64
	OverCap   bool
}

PruneStats reports what one pass removed.

type RecordShape 

type RecordShape struct {
	Flows      bool
	Bytes      bool
	Processes  bool
	DNS        bool
	Devices    bool
	OtherHosts bool
}

RecordShape reports what a stored database contains.

For --offline, where nothing is capturing and the capability model would otherwise describe an absence. Advertising nothing was the first attempt and it was wrong in a way that mattered: the dashboard hid traffic volumes behind "needs Patrol mode" while the record it was reading held byte counts for every flow. Capabilities drive what the views are willing to render, so offline has to answer "what is in here", not "what is running".

type Retention 

type Retention struct {
	Raw      time.Duration // full-resolution flows and DNS events
	Rollup   time.Duration // hourly aggregates
	MaxBytes int64         // hard cap on the database file
	Interval time.Duration // how often the pruner runs
}

Retention bounds how much history is kept. The defaults are chosen to be safe on a Raspberry Pi left running for weeks; both limits are enforced, size first.

func DefaultRetention 

func DefaultRetention() Retention

DefaultRetention is the Pi-safe default.

type SearchResult 

type SearchResult struct {
	Kind   string `json:"kind"` // endpoint | org | process | country
	Key    string `json:"key"`
	Label  string `json:"label"`
	Detail string `json:"detail,omitempty"`
	Count  int64  `json:"count"`
	// Peer names the machine that reported this hit, and is empty for anything
	// observed here. Searching found only this machine's traffic, so a paired
	// household could watch a peer's connections on the map and then fail to
	// find the same organization by typing its name.
	Peer string `json:"peer,omitempty"`
}

SearchResult is one hit from the global search.

type Store 

type Store struct {

	// OnFinding is called once for each genuinely new finding, never for one
	// that was merely seen again. Optional; nil means nothing is announced.
	//
	// Set by the caller so the store does not need to know that notifications
	// exist.
	OnFinding func(rule, subject string, score float64)
	// contains filtered or unexported fields
}

Store is the database handle.

func Open 

func Open(path string) (*Store, error)

Open opens (creating if needed) the database at path.

func (*Store) AddDispatchPeer 

func (s *Store) AddDispatchPeer(ctx context.Context, p dispatch.PairedPeer) error

AddDispatchPeer implements dispatch.Store, recording a completed pairing.

func (*Store) AddPeer 

func (s *Store) AddPeer(ctx context.Context, p Peer) error

AddPeer records a pairing.

func (*Store) BaselineAt 

func (s *Store) BaselineAt(ctx context.Context) time.Time

BaselineAt reports when this install began observing.

Exposed so the suspicion engine can tell how much history it is reasoning about. A rule that decides what is normal here has to know whether "here" has been watched for a day or a month.

func (*Store) Close 

func (s *Store) Close() error

Close releases the database.

func (*Store) DB 

func (s *Store) DB() *sql.DB

DB exposes the handle for callers that need one-off queries.

func (*Store) DNSEvents 

func (s *Store) DNSEvents(ctx context.Context, o DNSOptions) ([]types.DNSEvent, error)

DNSEvents returns the feed, newest first.

func (*Store) DNSSummary 

func (s *Store) DNSSummary(ctx context.Context, since, until time.Time) (DNSStats, error)

DNSSummary counts the DNS activity in a window.

func (*Store) DeviceAddresses 

func (s *Store) DeviceAddresses(ctx context.Context, id string) ([]types.DeviceAddress, error)

DeviceAddresses returns every address a device has held, most recent first.

func (*Store) DeviceServices 

func (s *Store) DeviceServices(ctx context.Context, id string) ([]types.DeviceService, error)

DeviceServices returns the services a device advertises.

func (*Store) Devices 

func (s *Store) Devices(ctx context.Context) ([]types.Device, error)

Devices returns the roster.

func (*Store) DispatchPeers 

func (s *Store) DispatchPeers(ctx context.Context) ([]dispatch.PeerRecord, error)

DispatchPeers implements dispatch.Store.

func (*Store) EditDevice 

func (s *Store) EditDevice(ctx context.Context, id string, e DeviceEdit) error

EditDevice applies a user's changes to a device.

Setting a device type by hand also locks it, because a person correcting a wrong guess should not have to keep correcting it every thirty seconds when inference runs again. Clearing the type unlocks it and hands the decision back to inference.

func (*Store) Egress 

func (s *Store) Egress(ctx context.Context, f Filter) ([]EgressRow, error)

func (*Store) EgressOmitted 

func (s *Store) EgressOmitted(ctx context.Context, f Filter, shown int) (int, error)

Egress returns external destinations with their aggregated traffic: what the Watchtower draws. EgressOmitted reports how many destinations matched the filter beyond the rows Egress returned.

Egress applies its limit in SQL, so a full result is indistinguishable from a truncated one: the caller receives exactly `limit` rows either way. The Watchtower then stated that number as "N seen in this period", which stops being true the moment somebody has more destinations than the cap, and says so most confidently to the people watching the busiest networks. The Precinct Map has always reported what it folded away; this is the same honesty for the list beside it.

Returns without querying unless the result came back full, so the ordinary case where nothing was cut pays nothing for the check.

func (*Store) Endpoint 

func (s *Store) Endpoint(ctx context.Context, ip string) (types.Endpoint, error)

Endpoint reads one endpoint with its enrichment.

func (*Store) ExpireDispatchSummaries 

func (s *Store) ExpireDispatchSummaries(ctx context.Context, ttl time.Duration, now time.Time) (int64, error)

ExpireDispatchSummaries implements dispatch.Store.

func (*Store) ExpireFindings 

func (s *Store) ExpireFindings(ctx context.Context, now time.Time, after time.Duration) (int64, error)

ExpireFindings closes findings that have not been seen for a while.

A finding is a claim about current behaviour. One that stopped happening a week ago is history, and leaving it open would mean the Wanted List slowly becomes a list of everything that ever happened, which is the same as a list of nothing.

func (*Store) ExpirePeerSummaries 

func (s *Store) ExpirePeerSummaries(ctx context.Context, ttl time.Duration, now time.Time) (int64, error)

ExpirePeerSummaries drops peer data past its time to live.

Peer data is a cache, not a record. This instance's own observations are the only thing it treats as durable truth.

func (*Store) Findings 

func (s *Store) Findings(ctx context.Context, status string, limit int) ([]Finding, error)

Findings returns recorded findings, newest first.

func (*Store) Flows 

func (s *Store) Flows(ctx context.Context, f Filter) ([]types.Flow, error)

Flows returns individual flow records, for export and for drilling into a single endpoint rather than the aggregate.

func (*Store) Glance 

func (s *Store) Glance(ctx context.Context, now time.Time) (Glance, error)

Glance computes the sidebar summary.

func (*Store) LabelDNS 

func (s *Store) LabelDNS(ctx context.Context, domain, category string) (int64, error)

LabelDNS applies a category to every stored lookup of a domain and its subdomains.

Labelling happens after the fact rather than at ingest, so that adding or updating a list re-labels history rather than only affecting what arrives next. Nothing is ever blocked: the label is information, not enforcement.

func (*Store) LocalSummaries 

func (s *Store) LocalSummaries(ctx context.Context, since time.Time, limit int) ([]dispatch.SummaryBucket, error)

LocalSummaries builds the aggregates this instance offers to its peers.

**This is the sending half of the exchange, and it was missing for a while:** the receive path, the wire format and the merge were all built and tested while nothing ever produced a bucket, so two paired instances connected and shared nothing. The lesson is the one this project keeps relearning, a feature is not the sum of its halves until something calls both.

Only outbound flows, aggregated to the hour. Nothing that identifies a destination beyond its organization and country leaves this machine: no address, no hostname, no process path. See docs/DISPATCH-PROTOCOL.md §D-5.

func (*Store) LogPeeringChange 

func (s *Store) LogPeeringChange(ctx context.Context, on bool) error

LogPeeringChange records that sharing was switched on or off.

In the same ledger as pairings, because the question somebody asks is "was this machine ever sharing", and an answer that lists pairings but not the times sharing itself was turned on is only part of it.

func (*Store) MarkStaleDevicesOffline 

func (s *Store) MarkStaleDevicesOffline(ctx context.Context, now time.Time) (int64, error)

MarkStaleDevicesOffline clears the online flag on devices nothing has seen recently, and reports how many changed.

Without this every device ever seen stays online forever, because a sighting can only ever prove presence: nothing announces its own absence.

func (*Store) MergeDispatchSummaries 

func (s *Store) MergeDispatchSummaries(ctx context.Context, peerID string,
	buckets []dispatch.SummaryBucket, now time.Time) (int, error)

MergeDispatchSummaries implements dispatch.Store.

peerID comes from the authenticated connection, and is passed through to MergePeerSummaries, which refuses any bucket claiming to belong to somebody else. The buckets arrive already sanitized by the wire layer; this converts them and nothing more.

func (*Store) MergePeerSummaries 

func (s *Store) MergePeerSummaries(ctx context.Context, peerID string, buckets []PeerSummary, now time.Time) (int, error)

MergePeerSummaries stores buckets reported by one authenticated peer.

peerID comes from the transport, which learned it from the pinned key that completed the TLS handshake. It is never taken from the message body: a peer that could name itself could name somebody else.

Counts are **replaced, not accumulated**. A peer reporting the same hour twice is restating a total it recomputed, not adding a second observation, and summing would let a peer inflate its own numbers without limit simply by resending.

func (*Store) NewDomains 

func (s *Store) NewDomains(ctx context.Context, since, until time.Time, limit int) ([]DomainSummary, error)

NewDomains returns domains whose first-ever lookup falls inside the window, most recent first.

This is the query that answers "what has this network started talking to that it never did before", which is the DNS half of first-contact detection.

func (*Store) ObserveDevice 

func (s *Store) ObserveDevice(ctx context.Context, o types.Sighting) (string, error)

ObserveDevice records a sighting, creating, updating or merging device records as the evidence requires. It returns the device's stable ID.

An observation carrying no identity key at all is not an error: a flow seen from an address nobody has named yet is still worth attributing, so the address alone is used to find an existing device, and otherwise nothing is created. Inventing a device per unnamed address would fill the Roster with rows that can never be merged.

func (*Store) PairingHistory 

func (s *Store) PairingHistory(ctx context.Context, limit int) ([]PairingEvent, error)

PairingHistory returns every pairing and unpairing this install has seen, most recent first.

Deliberately not filtered by whether the peer still exists: the entries for peers that were removed are the ones worth reading.

func (*Store) Path 

func (s *Store) Path() string

Path is the database file location.

func (*Store) PeerDestinations 

func (s *Store) PeerDestinations(ctx context.Context, since time.Time, limit int) ([]PeerDestination, error)

PeerDestinations aggregates what trusted peers have reported since a time, grouped for the map and the destination list.

Suspended peers are excluded by the join, as everywhere else: a read path that forgot the trust filter would silently reintroduce data the operator chose to stop believing.

func (*Store) PeerDevices 

func (s *Store) PeerDevices(ctx context.Context, since time.Time, peerID string) ([]PeerDevice, error)

PeerDevices lists the devices trusted peers have reported since a time.

func (*Store) PeerSearch 

func (s *Store) PeerSearch(ctx context.Context, term string, since time.Time, limit int) ([]SearchResult, error)

PeerSearch finds organizations and applications a peer has reported.

Addresses and countries are absent by design: a peer sends no address, and a country alone is not something anybody searches for by name here.

func (*Store) PeerSummariesSince 

func (s *Store) PeerSummariesSince(ctx context.Context, since time.Time) ([]PeerSummary, error)

PeerSummariesSince returns merged buckets from trusted peers only.

The trust filter lives in the query rather than in the caller. A suspended peer's rows stay on disk, suspension is reversible, and re-fetching a day of history on un-suspending would be worse, so every read path must exclude them, and the reliable way to guarantee that is to make the join do it.

func (*Store) PeerTopology 

func (s *Store) PeerTopology(ctx context.Context, since time.Time, peerID string) (Topology, error)

PeerTopology builds the Precinct Map's graph from what peers have reported.

peerID empty means every trusted peer; otherwise just that one. The shape is the same Topology the local graph uses, so the view merges the two by appending rather than by special-casing.

func (*Store) Peers 

func (s *Store) Peers(ctx context.Context) ([]Peer, error)

Peers lists paired instances.

func (*Store) PendingEnrichment 

func (s *Store) PendingEnrichment(ctx context.Context, limit int) ([]string, error)

PendingEnrichment returns external addresses that have never been enriched, most recently seen first so the map fills in with what the user is looking at.

func (*Store) Prune 

func (s *Store) Prune(ctx context.Context, r Retention) (PruneStats, error)

Prune enforces the retention policy. Age limits are applied first; if the database is still over its size cap, the oldest raw data is dropped in batches until it fits, and only then are rollups touched, losing detail before losing the long trend line.

func (*Store) QueryContext 

func (s *Store) QueryContext(ctx context.Context, query string, args ...any) (suspicion.Rows, error)

QueryContext satisfies suspicion.Queryer, giving rules a read-only view.

func (*Store) RecordObservations 

func (s *Store) RecordObservations(
	ctx context.Context, rule string, weight float64, obs []suspicion.Observation,
) error

RecordObservations writes a rule's findings.

A finding already raised is refreshed rather than repeated: its last_seen moves and its score takes the higher of the two, so behaviour that gets worse climbs the list while behaviour that merely continues does not accumulate.

func (*Store) RecordScannedServices 

func (s *Store) RecordScannedServices(ctx context.Context, id string, open []discover.OpenPort) error

RecordScannedServices stores the result of a user-requested port check.

Recorded with source "scan" so the Roster can say how a service was learned, a device that advertised SSH and a device that merely answered on 22 are different claims, and the second is the weaker one.

func (*Store) RecordShape 

func (s *Store) RecordShape(ctx context.Context) (RecordShape, error)

func (*Store) RefreshDeviceTypes 

func (s *Store) RefreshDeviceTypes(ctx context.Context, gateway netip.Addr) (int, error)

RefreshDeviceTypes re-infers what each device is, and returns how many changed.

Runs over the store rather than over individual sightings because the evidence arrives from different sources at different times: the neighbour table supplies the vendor, mDNS supplies the services, and neither alone is enough. Inference needs the merged view, which only exists here.

A type the user set by hand is never overwritten. Discovery is allowed to make a first guess, not to keep correcting a person who knows better.

func (*Store) RefreshObservedServices 

func (s *Store) RefreshObservedServices(ctx context.Context, since time.Time) (int, error)

RefreshObservedServices records services inferred from the ports that internal devices were seen answering on.

This is the passive half of service detection: no scanning, no packets. A connection *to* an address on this network reveals that something was listening there, and the port conventionally names it.

In Deputy Mode the evidence is limited to what this machine connected to, which is real but partial. Patrol Mode sees every device's inbound connections. Both feed the same table, distinguished by source, so the UI can say how a service was learned.

func (*Store) RemovePeer 

func (s *Store) RemovePeer(ctx context.Context, peerID string) error

RemovePeer unpairs and deletes everything that peer ever reported.

Both, together, in one transaction. An unpairing that left the data behind would mean the operator's decision to stop trusting a machine had no visible effect, which is the opposite of what the action means.

func (*Store) RenamePeer 

func (s *Store) RenamePeer(ctx context.Context, peerID, label string) error

RenamePeer sets the name shown for a peer on this machine only.

Unconditional, unlike SetDispatchPeerLabelIfEmpty: this is the operator speaking, and they outrank whatever the far end calls itself. It is also the answer to a machine that cannot describe itself, which is commoner than it sounds. A container's hostname is its id, and a machine nobody has named is "localhost"; both arrive here as no name at all, and without a way to set one the peer would be a fingerprint forever.

An empty name is allowed and means "go back to whatever it calls itself".

func (*Store) RequeueUnresolved 

func (s *Store) RequeueUnresolved(ctx context.Context) (int64, error)

RequeueUnresolved clears the enrichment stamp on endpoints that came back with nothing useful, so they are tried again.

This exists because the location databases are fetched in the background: endpoints observed in the first seconds of a first run get resolved against a set that has not landed yet. Without this they would stay blank forever, and the map would be permanently half-empty for every new install.

func (*Store) Rollup 

func (s *Store) Rollup(ctx context.Context, now time.Time) (int, error)

Rollup aggregates every complete hour that has not been aggregated yet. It returns the number of buckets processed.

func (*Store) RunDeviceTyping 

func (s *Store) RunDeviceTyping(ctx context.Context, every time.Duration)

RunDeviceTyping keeps device types and observed services current until ctx is cancelled.

func (*Store) RunPresence 

func (s *Store) RunPresence(ctx context.Context, every time.Duration)

RunPresence keeps the online flags honest until ctx is cancelled.

func (*Store) RunPruner 

func (s *Store) RunPruner(ctx context.Context, policy func() Retention)

RunPruner prunes on a schedule until the context is cancelled.

The policy is fetched fresh on every pass rather than captured once, so changing retention in settings takes effect without a restart.

func (*Store) RunRollups 

func (s *Store) RunRollups(ctx context.Context, every time.Duration)

RunRollups aggregates on a schedule until the context is cancelled.

func (*Store) SaveEnrichment 

func (s *Store) SaveEnrichment(ctx context.Context, e types.Endpoint) error

SaveEnrichment stores the enrichment for one endpoint.

func (*Store) Search 

func (s *Store) Search(ctx context.Context, term string, limit int) ([]SearchResult, error)

Search looks across endpoints, organizations, applications and countries at once, so the user can type "netflix" or "1.1.1.1" without choosing a category first.

func (*Store) SetDispatchPeerAddr 

func (s *Store) SetDispatchPeerAddr(ctx context.Context, peerID, addr string) error

SetDispatchPeerAddr implements dispatch.Store.

func (*Store) SetDispatchPeerLabelIfEmpty 

func (s *Store) SetDispatchPeerLabelIfEmpty(ctx context.Context, peerID, label string) error

SetDispatchPeerLabelIfEmpty implements dispatch.Store.

The WHERE clause carries the whole rule: a peer this machine has already named keeps that name, however often the far end announces its own.

func (*Store) SetFindingStatus 

func (s *Store) SetFindingStatus(ctx context.Context, id int64, status string) error

SetFindingStatus clears or trusts a finding.

func (*Store) SetPeerTrust 

func (s *Store) SetPeerTrust(ctx context.Context, peerID, trust string) error

SetPeerTrust moves a peer between trusted and suspended.

func (*Store) SetSetting 

func (s *Store) SetSetting(ctx context.Context, key, value string) error

SetSetting writes a persisted setting.

func (*Store) Setting 

func (s *Store) Setting(ctx context.Context, key string) (string, bool, error)

Setting reads a persisted setting.

func (*Store) Summary 

func (s *Store) Summary(ctx context.Context, since time.Time) (Summary, error)

Summary computes the dashboard headline numbers over a time window.

func (*Store) Timeline 

func (s *Store) Timeline(ctx context.Context, since, until time.Time) ([]TimePoint, error)

Timeline returns hourly activity between two times, for the scrub control.

Recent hours are computed from raw flows, which is exact; older hours come from the rollups, which is what lets the range extend far past the raw retention window without keeping the flows themselves.

func (*Store) TopDomains 

func (s *Store) TopDomains(ctx context.Context, since, until time.Time, limit int) ([]DomainSummary, error)

TopDomains returns the most-queried domains in a window, marking any whose first-ever sighting falls inside it.

The "new" flag is computed against the whole history rather than the window, because a domain queried every day for a month is not news, and one that appeared an hour ago is.

func (*Store) Topology 

func (s *Store) Topology(ctx context.Context, f Filter) (Topology, error)

Topology derives the graph from observed flows.

func (*Store) TouchEndpoints 

func (s *Store) TouchEndpoints(ctx context.Context, seen map[string]EndpointSighting) error

TouchEndpoints records that these addresses were seen, creating rows for any that are new so the enricher has something to work on.

func (*Store) UpsertDevice 

func (s *Store) UpsertDevice(ctx context.Context, d types.Device) error

UpsertDevice inserts or refreshes a device.

func (*Store) Wanted 

func (s *Store) Wanted(ctx context.Context, limit int) ([]SubjectSuspicion, error)

Wanted returns subjects ranked by how much is open against them.

Scores combine additively and are capped at 1. Several weak signals about one device is the case worth surfacing, a single weak signal is not, and this is what stops any one rule from dominating the list on its own.

func (*Store) Wipe 

func (s *Store) Wipe(ctx context.Context) error

Wipe deletes every observation, leaving settings and the schema intact. It backs the one-click wipe the privacy posture promises.

func (*Store) WriteDNS 

func (s *Store) WriteDNS(ctx context.Context, events []types.DNSEvent) error

WriteDNS records DNS observations.

func (*Store) WriteFlows 

func (s *Store) WriteFlows(ctx context.Context, flows []types.Flow) error

WriteFlows upserts a batch of flows in a single transaction.

type SubjectSuspicion 

type SubjectSuspicion struct {
	Subject     string  `json:"subject"`
	SubjectType string  `json:"subject_type"`
	Label       string  `json:"label,omitempty"`
	Score       float64 `json:"score"`
	Findings    int     `json:"findings"`
}

SubjectSuspicion is a subject's total score across its open findings.

type Summary 

type Summary struct {
	Flows        int64   `json:"flows"`
	ActiveFlows  int64   `json:"active_flows"`
	Endpoints    int64   `json:"endpoints"`
	Countries    int64   `json:"countries"`
	Devices      int64   `json:"devices"`
	Inbound      int64   `json:"inbound"`
	DNSEvents    int64   `json:"dns_events"`
	TopOrgs      []Count `json:"top_orgs"`
	TopCountries []Count `json:"top_countries"`
	TopProcesses []Count `json:"top_processes"`
	DBBytes      int64   `json:"db_bytes"`
}

Summary is the headline count set.

type TimePoint 

type TimePoint struct {
	TS        int64  `json:"ts"`
	Conns     int64  `json:"conns"`
	BytesOut  uint64 `json:"bytes_out"`
	BytesIn   uint64 `json:"bytes_in"`
	Endpoints int64  `json:"endpoints"`
}

TimePoint is one bucket of the activity timeline.

type TopoEdge 

type TopoEdge struct {
	Source string `json:"source"`
	Target string `json:"target"`
	Conns  int64  `json:"conns"`
	Bytes  int64  `json:"bytes"`
}

TopoEdge is a line between two nodes.

type TopoNode 

type TopoNode struct {
	ID string `json:"id"`
	// Kind is "device" for something on this network, "org" for an external
	// destination, and "gateway" for the router.
	Kind  string `json:"kind"`
	Label string `json:"label"`
	// Type is a device type code for internal nodes; empty for external ones.
	Type string `json:"type,omitempty"`
	// Country is the ISO code for external nodes, for colouring and tooltips.
	Country string `json:"country,omitempty"`
	Conns   int64  `json:"conns"`
	Bytes   int64  `json:"bytes"`
	// Online and Trust apply to internal nodes.
	Online bool   `json:"online,omitempty"`
	Trust  string `json:"trust,omitempty"`
	// New marks an organization this network had not contacted before the
	// window, which is the whole point of watching a map rather than a list.
	New bool `json:"new,omitempty"`
}

TopoNode is one circle on the map.

type Topology 

type Topology struct {
	Nodes []TopoNode `json:"nodes"`
	Edges []TopoEdge `json:"edges"`
	// Truncated reports that quieter organizations were folded away to keep the
	// graph readable, so the UI can say so rather than appearing to lie.
	Truncated int `json:"truncated"`
}

Topology is the whole graph.

func MergeTopology 

func MergeTopology(local, peer Topology) Topology

MergeTopology folds a peer graph into a local one.

**Not an append.** Organizations are keyed by name, so an organization both this machine and a peer have contacted arrives twice with the same node ID. Two nodes sharing an ID is not a cosmetic duplicate: the force layout keys on it, so edges resolve to whichever copy was indexed last and the two circles sit on top of each other. It looks like a rendering fault and is a data one.

Merging is also the honest answer. "Your laptop and your peer's television both talk to this company" is precisely what the Precinct Map exists to show, and it can only show it if the company is one circle with lines from both.

Devices never merge: their identifiers are namespaced by peer, because two households can each own a laptop called "macbook" and they are not the same laptop.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.