config

package
v0.0.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2026 License: AGPL-3.0 Imports: 15 Imported by: 0

Documentation

Overview

Package config loads and validates the agent's configuration files.

The files are YAML, one format only, and hand-written config is the primary interface (SPEC §5.9): flat keys, a default for everything that has a sensible one, compact scalar forms where they read naturally. Loading is atomic — a configuration is fully valid or refused whole, with every error reported at once, each carrying its file and line.

Index

Constants

View Source 
const (
	Warning = "warning"
	Notice  = "notice"
)
View Source 
const CLIExemptions = "exemptions-cli.yaml"

CLIExemptions is the file `shield exempt` writes into: shieldlist's own, beside the operator's, so that neither rewrites the other's.

View Source 
const DefaultLevel = "standard"

DefaultLevel is the level a machine runs at when its agent.yaml names none: the numbers as written.

View Source 
const DefaultReport = "default"

ReportPresetName is the report a rule uses when it names none.

View Source 
const FallbackMemoryLimit int64 = 192 << 20

FallbackMemoryLimit applies when the machine will not say how much memory it has, so a share of it cannot be worked out. Modest on purpose: a program that cannot measure its host should err small.

View Source 
const InstantLevel = "instant"

InstantLevel is the level where the first hit convicts.

Variables

View Source 
var DefaultMemoryLimit = Size{Percent: 50}

DefaultMemoryLimit is what the agent allows itself when nobody says: half the machine. That is a lot for a guest program, and it is deliberate — this one is what stops the attack, and evidence it cannot hold is an attacker it cannot convict.

It is a ceiling and not a reservation: the agent takes only what its rules actually need, and on a quiet machine that is a few tens of megabytes. But it is a FIXED ceiling — the agent does not watch how much memory the machine has free and does not shrink when something else grows. Set it to what this program may have on this machine; dividing the rest is the administrator's job, not ours.

Functions

func FormatDuration 

func FormatDuration(d Duration) string

FormatDuration writes a duration the way the files do: whole days as "30d", whole weeks are left as days (a reader adds up days faster than weeks), anything else in Go's units with the zero parts dropped.

func InlineParserName 

func InlineParserName(rule string, n int) string

InlineParserName is how the n-th regex written inside a rule's parsers is named (1-based): the rule, a hash, the position.

func TimeLayout 

func TimeLayout(format string) (string, error)

TimeLayout resolves a declared format to a layout, or reports that it is not one this build understands.

Types

type Agent 

type Agent struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`

	// Mode is "local" (the default: configuration comes from these files)
	// or "enrolled" (rules, exemptions, trust and policies come from the
	// controller, activity reports back).
	Mode       string     `yaml:"mode"`
	Controller string     `yaml:"controller"` // base URL, required when enrolled
	LogInputs  []LogInput `yaml:"logs"`
	Limits     Limits     `yaml:"limits"`

	// State is the whole agent's: "on" (the default) or "detect-only",
	// which puts every rule in detect-only and leaves the data plane
	// alone entirely — no table, no restore, nothing added or removed.
	//
	// It exists because of how this program is adopted. Nobody points a
	// new blocker at a production machine and hopes; they run it beside
	// whatever already guards the box, read a day of what it *would*
	// have done, and only then let it act. Doing that by editing the
	// state of every rule is an invitation to miss one, and to forget
	// which ones were switched when the day comes to switch them back.
	State string `yaml:"state"`

	// Level scales every rule's threshold on this machine (SPEC §5.3):
	// instant (the first hit convicts), strict (half the hits), standard
	// (as written, the default), lenient (twice the hits). One knob a
	// person can predict: rules never carry several sets of numbers.
	Level string `yaml:"level"`

	// LogFile is where the agent writes its own log, as well as to its
	// standard error — which under systemd is the journal, and is the
	// right default. Empty means the journal alone.
	//
	// It exists because the first place an administrator looks after
	// starting a security daemon is /var/log, because that is where the
	// other ones write. Finding nothing there reads as "it did not
	// start". The agent rotates out of its own way: the file is
	// identified by (device, inode), so logrotate needs no cooperation
	// and no configuration ships with it.
	LogFile string `yaml:"log_file"`

	// Reporting is how this machine submits offenders to abuse
	// providers. Absent means it does not: reporting is an egress of
	// personal data and an act in the operator's name, and it never
	// happens because a default said so.
	Reporting Reporting `yaml:"reporting"`

	// GeoIP names MaxMind-DB files of the operator's own — a country
	// database, an ASN database, either — for the rules and exemptions
	// that look at where an address is from. An enrolled machine needs
	// none of this: it pulls the controller's (the free DB-IP lite
	// files) under its state directory and keeps them current.
	GeoIP GeoIPFiles `yaml:"geoip"`

	// Hooks are the operator's own endpoints, each POSTed one JSON
	// document per enforced ban. They are LOCAL configuration and a
	// controller can never push one (SPEC §5.5) — a hook is an egress
	// the operator alone decides, or whoever controlled the controller
	// would control where every machine's decisions are sent.
	Hooks []Hook `yaml:"hooks"`

	// ReplayOnStart makes a starting agent read this much of its logs'
	// past before joining the present, and enforce what it finds that is
	// still worth enforcing — a crossing counts only if the sanction it
	// earns would still be in force now.
	//
	// Off unless asked for. A live tail joins the present, and reading a
	// log's past is a deliberate act; but an operator who turns up after
	// an attack began wants the agent to catch up on it, and the tidiest
	// way to ask is to restart it.
	ReplayOnStart Duration `yaml:"replay_on_start"`
}

Agent is agent.yaml: how this machine runs. Everything is optional; an absent file means a standalone agent with defaults.

type Config 

type Config struct {
	Agent      Agent
	Parsers    []*Parser
	Rules      []*Rule
	Policies   []*Policy
	Reports    []*ReportPreset
	Packs      []*Pack
	Trust      []*TrustEntry
	Exemptions []*Exemption
	Secrets    Secrets

	// Notes are what loading found that is worth saying and not worth
	// refusing over. Printed by `shieldlist-agent -t` and logged at
	// startup.
	Notes []Note
}

Config is a fully loaded, not yet validated configuration tree.

func Load 

func Load(root string) (*Config, error)

Load reads the configuration tree under root: 

agent.yaml          (optional)
parsers/**.yaml     one parser per file (sub-directories: by source, for people)
rules/**.yaml       one rule per file (sub-directories: by category)
policies/**.yaml    one policy per file
reports/**.yaml     (optional) one report preset per file
packs/*.yaml        (optional) sets of rules, by name
secrets.yaml        (optional) provider credentials and allowances
trust.yaml          (optional) list of trust entries
exemptions.yaml     (optional) list of exemptions, yours
exemptions.d/*.yaml (optional) more of them, one list per file
exemptions-cli.yaml (optional) exemptions added with `shield exempt`

Loading is atomic: either the whole tree is valid and a Config is returned, or every problem found is returned at once as Errors.

func LoadEnrolled 

func LoadEnrolled(local, received string) (*Config, error)

LoadEnrolled loads an enrolled agent's configuration (SPEC §5.9, §6): the rule language — parsers, rules, policies, report presets, trust — comes from received, the tree the controller assembled for this machine; agent.yaml, secrets and the machine's own exemptions come from local and add to what was received. An empty received directory (nothing fetched yet) leaves the agent on its local files, so a machine enrolled before its first sync still defends itself.

func (*Config) Orphans 

func (c *Config) Orphans() []string

Orphans reports categories that connect nothing to nothing: a log input whose lines no parser reads, and a parser offered no lines. It is not an error — a configuration is assembled in pieces, and a category can be half-built for a minute — but it is the shape of the two mistakes that produce no symptom at all. A machine can watch the wrong path for months and look exactly like a quiet one.

type Conversion 

type Conversion struct {
	Written []string
	Notes   []string
	// contains filtered or unexported fields
}

Conversion is what Convert did: the files written and the notes worth reading — what it dropped, what it guessed.

func Convert 

func Convert(src, dst string) (*Conversion, error)

Convert rewrites a configuration tree written in the format of the 0.0.x releases into the current one, file by file, comments kept — under dst, leaving src untouched. It converts what it recognises and says what it changed; the result still has to load (run -t on it).

What changes (2026-08-18, the format made simple): 

parsers   pattern → regex, category → log, ports dropped (they belong to the
          machine's log inputs now: agent.yaml gets them by log)
rules     match → parsers (pattern → regex), threshold+window → threshold: N/window,
          thresholds → threshold (a set per level keeps its standard set — the level
          scales it now), report_as → report, group_by → count, alert → notify,
          category (of inline patterns) → log, placement dropped, report: never/repeat-only dropped
policies  decay → memory, ports per rung → one scope (all | service)
agent.yaml  log_inputs → logs, category → log, ports added by log
packs/<name>/  directories become parsers/<name>/, rules/, policies/, reports/ and packs/<name>.yaml (a list)
agents/*.yaml  (controller) log_inputs → logs, rules.<r>.thresholds → threshold

type Duration 

type Duration time.Duration

Duration is a time span as written in config files. On top of Go's s/m/h units it accepts d (days) and w (weeks), because ban durations are written in days far more often than in hours.

func ParseDuration 

func ParseDuration(s string) (Duration, error)

ParseDuration parses "90s", "10m", "1h30m", "30d", "4w".

func (*Duration) UnmarshalYAML 

func (d *Duration) UnmarshalYAML(n *yaml.Node) error

type Durations 

type Durations []Duration

Durations is a policy's `durations:` — one ("24h") or the ladder ("[10m, 1h, 24h]": first offence, second, third — the last repeats).

func (*Durations) UnmarshalYAML 

func (d *Durations) UnmarshalYAML(n *yaml.Node) error

type Error 

type Error struct {
	File string
	Line int
	Msg  string
}

Error is one problem in one file. Msg is plain and self-contained.

func (Error) Error 

func (e Error) Error() string

type Errors 

type Errors []Error

Errors is every problem found in one load, sorted by file then line.

func (Errors) Error 

func (es Errors) Error() string

type Exemption 

type Exemption struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`

	IP    string `yaml:"ip"`
	Range string `yaml:"range"`
	// Until ends an exemption on its own, which is what the common case
	// wants: "let me back in while I fix this". Empty means permanent.
	Until     string   `yaml:"until"`
	ASN       string   `yaml:"asn"`
	Country   string   `yaml:"country"`
	OnlyPaths []string `yaml:"only_paths"` // conditional: exempt only on these paths
	Note      string   `yaml:"note"`
}

Exemption never sanctions its subject; the traffic is still logged. Exactly one of the scope keys is set.

type GeoIPFiles 

type GeoIPFiles struct {
	Country string `yaml:"country"`
	ASN     string `yaml:"asn"`
}

GeoIPFiles is `geoip:` in agent.yaml — files of the operator's own.

type Hook 

type Hook struct {
	// URL receives a POST per enforced ban, JSON body. Credentials go
	// in the URL itself (a token query parameter, basic auth in the
	// authority) — this file is local and the URL never leaves the
	// machine.
	URL string `yaml:"url"`
	// Timeout bounds one delivery. Default five seconds, like the
	// report provider: nothing waits on a hook.
	Timeout Duration `yaml:"timeout"`
}

Hook is one operator endpoint that receives the agent's decisions: a webhook, never a command — configuration is data and stays data.

type Limits 

type Limits struct {
	// Memory is the ceiling the agent holds itself to. Left unset — the
	// normal case — it is derived from the machine: an operator should
	// not have to guess a number for a program that can read how much
	// memory the box has. The shipped systemd unit sets MemoryMax to
	// twice whatever it works out to, so the kernel's limit is a backstop
	// for a runaway, not the working limit.
	Memory Size `yaml:"memory"`
}

Limits is the ceiling the agent imposes on itself. It runs on a machine it does not own, so saturation must be a shed with a defined order and a metric, never an OOM kill of the box it protects (SPEC §6).

type LogInput 

type LogInput struct {
	Path string `yaml:"path" json:"path,omitempty"`
	Unit string `yaml:"unit" json:"unit,omitempty"`
	// Category is the kind of line the input holds — http, ssh, mail… —
	// the word that joins a log file to the parsers that read it. In the
	// files it is `log:` (and on the wire, the API's JSON, too).
	Category string `yaml:"log" json:"log"`
	// Ports are the service's ports on THIS machine (an sshd on 2222, a
	// site on 8443): what a policy with `ports: service` scopes a ban to.
	// Declared here, where they are true, never in a parser.
	Ports []uint16 `yaml:"ports" json:"ports,omitempty"`
	// Target names what every line of this input is about — the vhost,
	// the site — for a log whose path does not carry `{target}` (one
	// vhost, one fixed file: a proxy's access log, a panel's). Rules
	// filter on it (`targets:`/`except_targets:`) and the console shows
	// it, exactly as if the path had named it.
	Target string `yaml:"target" json:"target,omitempty"`
}

LogInput is one log stream this machine watches: a file or a journald unit, exactly one of the two. Discovery will propose these; for now they are declared.

A path may be a glob, and on a machine hosting sites it usually is: a panel gives every site its own log directory, sites are added and removed by people who are not thinking about this agent, and a list of paths written by hand is a list that goes stale silently — the one failure this program must never have, because nothing about it looks like a failure. The pattern is re-read as the agent runs, so a site created this afternoon is watched this afternoon.

func (LogInput) Expand 

func (in LogInput) Expand() ([]string, error)

Expand returns the files this input names right now, in a stable order. A plain path is returned whether or not it exists — a log file that is not there yet is an ordinary state, and the tail waits for it. A pattern returns only what it matches at this moment, which is why the agent asks again as it runs.

func (LogInput) HasTarget 

func (in LogInput) HasTarget() bool

HasTarget reports whether the path names its target — the domain, the site, the mailbox a file belongs to — with a `{target}` segment: `/var/www/vhosts/system/{target}/logs/access_log`, `/var/log/nginx/ {target}.access.log`. It globs like `*` and, once a file is matched, the text it stood for is the target every line of that file is about.

func (LogInput) IsGlob 

func (in LogInput) IsGlob() bool

IsGlob reports whether the path is a pattern to expand rather than one file to open. A `{target}` segment is a pattern too.

func (LogInput) Name 

func (in LogInput) Name() string

Name identifies the input in positions, logs and metrics.

func (LogInput) TargetOf 

func (in LogInput) TargetOf(file string) string

TargetOf reads, off a file the pattern matched, what `{target}` stood for — the declared `target:` when the path names none, "" when neither says anything or the file does not match.

type MatchEntry 

type MatchEntry struct {
	Parser string `yaml:"parser"`
	Weight int    `yaml:"weight"` // default 1; negative argues against a ban

	Pattern      string   `yaml:"regex"`
	Prefilter    string   `yaml:"prefilter"`
	PrefilterAny []string `yaml:"prefilter_any"`
}

MatchEntry is one parser a rule scores, with the weight of each hit — a shared parser named, or a regex written right here (SPEC §5.2: the regex and the rule that scores it in one file). An inline entry is a parser in every respect but the file: the loader synthesizes it as <rule>#<n> reading the rule's log, and everything downstream — evidence, hits, `shield why` — names it that way.

func (MatchEntry) Inline 

func (m MatchEntry) Inline() bool

Inline reports whether the entry carries its own pattern.

func (*MatchEntry) UnmarshalYAML 

func (e *MatchEntry) UnmarshalYAML(n *yaml.Node) error

UnmarshalYAML accepts a bare parser name (one entry, weight 1) or a list of {parser, weight} entries. An omitted weight is 1. UnmarshalYAML lets a list entry be a bare parser name — `match: [a, b]` is how a jail with several patterns reads — or the {parser, weight} map.

type Matches 

type Matches []MatchEntry

Matches accepts a bare parser name or a list of weighted entries.

func (*Matches) UnmarshalYAML 

func (m *Matches) UnmarshalYAML(n *yaml.Node) error

type Multiplier 

type Multiplier float64

Multiplier is "3x" or "0.5x" — above 1 demands more evidence, below 1 demands less.

func (*Multiplier) UnmarshalYAML 

func (m *Multiplier) UnmarshalYAML(n *yaml.Node) error

type Note 

type Note struct {
	// Warning: something an operator asked for will not happen.
	// Notice: something is prepared but not in use, which is usually
	// deliberate and occasionally a forgotten step.
	Level string
	File  string
	Line  int
	Msg   string
}

Note is something worth saying that must not stop the agent.

The distinction is not cosmetic. Everything this package refuses is something that would make the agent behave wrongly; a note is something that makes it behave LESS, and refusing to start over it would be the wrong trade every time. A machine must not go undefended because its abuse reporting is misconfigured.

func (Note) String 

func (n Note) String() string

type Origin 

type Origin struct {
	Kind  OriginKind
	Value string // normalised: upper-case country, "AS<n>", canonical prefix
}

Origin is where traffic comes from — a country, an ASN or an address range — auto-detected from its written form: "FR", "AS3215", "198.51.100.0/24" or a bare address.

func ParseOrigin 

func ParseOrigin(s string) (Origin, error)

ParseOrigin classifies and normalises an origin.

func (*Origin) UnmarshalYAML 

func (o *Origin) UnmarshalYAML(n *yaml.Node) error

type OriginKind 

type OriginKind string
const (
	OriginCountry OriginKind = "country"
	OriginASN     OriginKind = "asn"
	OriginRange   OriginKind = "range"
)

type Pack 

type Pack struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`

	Name        string   `yaml:"pack"`
	Description string   `yaml:"description"`
	Rules       []string `yaml:"rules"`
}

Pack is packs/<name>.yaml: a set of rules named together — what a controller assigns to a machine as a whole. On a standalone machine it is documentation: every rule in the tree runs.

type Parser 

type Parser struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`
	// Dir is the sub-directory the file sits in (parsers/<dir>/…), the
	// source it belongs to by convention — nginx, sshd, dovecot — and
	// nothing more: an ordering for people, no meaning for the loader.
	Dir string `yaml:"-"`
	// Inline marks a parser synthesized from a regex written inside a
	// rule's parsers — named <rule>#<n>, owned by that rule, never a
	// file of its own.
	Inline bool `yaml:"-"`

	Name string `yaml:"parser"`
	// Category is the kind of line it reads (`log:` in the file): http,
	// ssh, mail — the same word a log input on the machine carries.
	Category  string `yaml:"log"`
	Prefilter string `yaml:"prefilter"`
	// PrefilterAny replaces Prefilter for list parsers whose entries
	// share no literal: the line must contain at least one of these
	// tokens (case-insensitive) before the regex runs. Exactly one of
	// the two forms per parser.
	PrefilterAny []string `yaml:"prefilter_any"`
	Pattern      string   `yaml:"regex"`

	// TimeField names the capture group holding the line's own timestamp,
	// and TimeFormat says how to read it: a named format ("nginx",
	// "syslog", "rfc3339", "unix") or a Go layout for anything else.
	//
	// A live tail does not use them — it stamps a line with the moment it
	// read it, which is the same thing and costs nothing. They are what
	// makes replaying a log's past possible at all: without them every
	// line of a week-old file claims to have happened now, and rules that
	// count events in a window would ban whoever appears in it.
	TimeField  string `yaml:"time_field"`
	TimeFormat string `yaml:"time_format"`
}

Parser recognises something in one category of logs: a mandatory literal prefilter gates an RE2 pattern whose named groups become fields.

type Policy 

type Policy struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`
	Dir  string `yaml:"-"`

	Name      string    `yaml:"policy"`
	Durations Durations `yaml:"durations"` // one, or the ladder
	// Ports: all (the default), service (the ports the machine declares
	// for the rule's log), or a list of ports.
	Ports PortScope `yaml:"ports"`
	// Decay is `memory:` in the file — how long after a ban ENDED the
	// machine still holds it against the address. Unset: for ever.
	Decay         Duration `yaml:"memory"`
	DurationStart string   `yaml:"duration_start"` // evidence | application
	// Reputation lets the controller lengthen a ban when the address's
	// AbuseIPDB confidence is at or above `above` — asynchronously, never
	// between a hit and a sanction (SPEC principle 5). The agent carries
	// it, the controller acts on it.
	Reputation *ReputationEscalation `yaml:"reputation"`
	// Tighten makes a returning offender easier to convict: each past
	// episode within decay divides every rule's threshold count by this
	// (ceiling; never below one). "2x" turns 10 hits into 5 the second
	// time, 3 the third. Unset: the count never moves.
	Tighten Multiplier `yaml:"tighten"`
}

Policy is what a ban IS, named once and used by any rule: the ladder of durations (first offence, second, third — the last repeats), the ports it covers, how long the machine remembers an address after its ban ended (a return within that time climbs the ladder), and the optional sharpening on repeat and on reputation.

type PortScope 

type PortScope struct {
	Mode string   // "" (all) | all | service | list
	List []uint16 // when Mode is list
}

PortScope is a policy's `ports:` — all (the default), service (the ports the machine declares for the rule's log), or a list of ports.

func (PortScope) IsAll 

func (p PortScope) IsAll() bool

IsAll reports a ban that covers every port.

func (PortScope) String 

func (p PortScope) String() string

func (*PortScope) UnmarshalYAML 

func (p *PortScope) UnmarshalYAML(n *yaml.Node) error

type ProviderSecret 

type ProviderSecret struct {
	Key string `yaml:"key"`
	// Enabled turns reporting off without losing the key: absent means
	// on (the key being set is the decision), false means keep the key
	// and send nothing. Read it through On.
	Enabled *bool `yaml:"enabled"`
	// DailyLimit is the operator's plan: a number, or "auto" to learn it
	// from the provider.
	//
	// A number is the agent's own guard and may deliberately sit BELOW
	// the plan — sharing one account's allowance between machines, for
	// instance. "auto" lets the first request of the day find out, and
	// every response keep it current: the same account may be reporting
	// from several places, so what this machine has sent is not what has
	// been sent, and only the provider knows.
	DailyLimit ReportLimit `yaml:"daily_limit"`
}

ProviderSecret is one provider's credential and what the operator's subscription allows.

func (ProviderSecret) On 

func (p ProviderSecret) On() bool

On reports whether reporting is switched on: absent means yes — the key being set is the decision — and only an explicit false says no.

type RenewSpan 

type RenewSpan struct {
	Never bool
	Every Duration
}

RenewSpan is Reporting.Renew: a duration, or "never".

func (RenewSpan) Span 

func (r RenewSpan) Span() time.Duration

Span answers the interval to use: never → 0, absent → a day.

func (*RenewSpan) UnmarshalYAML 

func (r *RenewSpan) UnmarshalYAML(n *yaml.Node) error

UnmarshalYAML accepts a duration or the word never.

type ReportLimit 

type ReportLimit struct {
	N    int
	Auto bool
}

ReportLimit is a number of reports a day, or "auto".

func (ReportLimit) Set 

func (l ReportLimit) Set() bool

Set reports whether reporting has an allowance to work with.

func (ReportLimit) String 

func (l ReportLimit) String() string

func (*ReportLimit) UnmarshalYAML 

func (l *ReportLimit) UnmarshalYAML(node *yaml.Node) error

type ReportPreset 

type ReportPreset struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`
	Dir  string `yaml:"-"`

	Name string `yaml:"report"`
	// To is the destination: abuseipdb (the default, and the only one
	// today). When says on which sanctions: always (default) or
	// repeat-only — a returning offender's.
	To   string `yaml:"to"`
	When string `yaml:"when"`
	// Categories are the provider's own vocabulary, by name
	// ("brute-force", "ssh") rather than by number: a configuration
	// full of bare integers cannot be reviewed, and reviewing what
	// leaves the machine is the point.
	Categories []string `yaml:"categories"`
	// Says is the sentence the report carries. It must describe what
	// the offender did and never how this machine is arranged or
	// defended — a report is published, and it is not the place to
	// tell an attacker what stopped them.
	Says string `yaml:"says"`
	// Include names the parser fields that may appear in the report
	// alongside that sentence. DEFAULT-DENY: a field not named here
	// cannot leave, whatever a log line happens to contain. This is the
	// whole redaction mechanism, and it is a list rather than a filter
	// because "emit only what is named" is a property and "strip what
	// looks sensitive" is a judgement made once and wrong later.
	//
	// The offender's address is always sent — it is what a report IS —
	// and needs no naming here.
	Include []string `yaml:"include"`
}

ReportPreset is what a report SAYS, named and referenced by rules the way a policy is. One per file, in reports/.

It exists as data because only the provider's API belongs in the program (SPEC §5.5). The alternative, seen in the system this replaces, is a two-hundred-line chain of conditions in a central template whose whole job is to map a rule's name to a sentence and a few category numbers — which every new rule must remember to go and edit, in a file that knows about every rule there has ever been. Declared on the rule, that chain does not exist.

type Reporting 

type Reporting struct {
	// Provider is the only thing that turns reporting on. Empty means
	// off, and off is the default.
	Provider string `yaml:"provider"` // abuseipdb

	// Endpoint overrides the provider's URL. Not for production: it is
	// what lets the whole path be exercised against a server of one's
	// own before a single report reaches a real provider, and what
	// lets egress go through a proxy an operator controls.
	Endpoint string `yaml:"endpoint"`

	// Interval is how often the sender looks at what the agent decided
	// since it last looked. Default thirty seconds: a report is a
	// courtesy and nothing waits on it. Within a pass the batch goes at
	// full speed — a burst of bans is a burst of requests, which is
	// what a provider expects from a machine under attack.
	//
	// Every attempt logs its outcome on its own line: sent, already
	// reported, or not sent and why. A report gets one chance, at the
	// moment of the detection; what could not leave stays on record,
	// owed, and `shield report` sends it later if the operator decides
	// it still matters.
	Interval Duration `yaml:"interval"`

	// Renew is how long a banned address may keep attacking before it is
	// reported again: a ban that is extended because the hits go on
	// (check 5) reopens its report once the last one is this old. The
	// provider hears that the abuse continues; the operator sees a report
	// as recent as the attack. Default one day; `never` turns it off.
	Renew RenewSpan `yaml:"renew"`
}

Reporting is this machine's abuse reporting: whether it does it at all, to whom, and how loudly.

type ReputationEscalation 

type ReputationEscalation struct {
	Above    int      `yaml:"above"`
	Duration Duration `yaml:"duration"`
}

ReputationEscalation is a policy's `reputation:` — a longer duration for an address whose provider confidence score reaches a bar.

type Rule 

type Rule struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`
	Dir  string `yaml:"-"` // rules/<dir>/…, an ordering for people

	Name        string `yaml:"rule"`
	Description string `yaml:"description"`
	// Category is where the rule is filed — web, mail, ssh, database,
	// panel… — for people and the console; the loader gives it no
	// meaning. Defaults to the sub-directory the file sits in.
	Category string `yaml:"category"`
	// Log, TimeField and TimeFormat belong to the regexes written inline
	// in parsers (a rule that only names parsers has no use for them):
	// the kind of line those regexes read, and where the line's own
	// timestamp is, exactly as a parser file says it.
	Log        string  `yaml:"log"`
	TimeField  string  `yaml:"time_field"`
	TimeFormat string  `yaml:"time_format"`
	Match      Matches `yaml:"parsers"`
	// Threshold: one crossing — `5/10m`, five points in ten minutes — or
	// several, `[5/10m, 20/1h]`, the first crossed fires. Effective is
	// what this machine evaluates: the same, scaled by its level.
	Thresholds Thresholds  `yaml:"threshold"`
	Effective  []Threshold `yaml:"-"`
	// Ban names the policy — how long, on which ports, how the machine
	// remembers. Notify raises a notification (info, warning, critical)
	// besides, or instead of, the ban. A rule with neither only records.
	Ban    string `yaml:"ban"`
	Alert  string `yaml:"notify"`
	Report string `yaml:"report"` // the report preset; "default" when unset
	// GroupBy is what hits are counted per (`count:` in the file):
	// address (the default), network — a /24 —, and later asn, country.
	// Held here as the evaluator's key: ip, range:/24, asn, country.
	GroupBy  string `yaml:"count"`
	Distinct string `yaml:"distinct"` // count distinct values of this field
	When     string `yaml:"when"`     // extra condition over parsed fields
	// Where the offender is from — the rule counts only hits from
	// addresses in these countries (ISO codes) or ASNs, or from all but
	// these. Needs the geo databases on the machine (pulled from the
	// controller, or files named in agent.yaml): without them a rule
	// with any of these is inactive, and says so.
	Countries       Words `yaml:"countries"`
	ExceptCountries Words `yaml:"except_countries"`
	ASNs            Words `yaml:"asns"`
	ExceptASNs      Words `yaml:"except_asns"`
	// What the hit was aimed at — the rule counts only hits whose target
	// (the site, the domain: a {target} segment in the log input's path,
	// or the domain of the user named) matches one of these, or all but
	// these. A name, or a pattern with `*` (`*.example.org`, `cloud.*`).
	// A hit with no target: counted by except_targets, not by targets.
	Targets       Words  `yaml:"targets"`
	ExceptTargets Words  `yaml:"except_targets"`
	State         string `yaml:"state"` // on | detect-only | off
	// Share says whether this rule's bans may be offered to the fleet;
	// yes unless "no".
	Share string `yaml:"share"`
}

Rule scores hits and applies a sanction — or notifies, or only records — when the weighted sum crosses its threshold within the window. Every hit contributes its parser's weight (1 unless said otherwise; negative for traffic that argues against a ban), so one rule can accumulate varied behaviour across several parsers.

The file reads in the order things happen: what it reads (log, only for a regex written here), what it recognises (parsers), from how much (threshold), what it does (ban: a policy; notify), what it says (report). Everything else is optional and defaulted.

func (*Rule) Crossings 

func (r *Rule) Crossings() []Threshold

Crossings is what the rule fires on: Effective when Load resolved it, else the threshold as written — so a rule built in code rather than loaded from files still evaluates. Empty means the rule cannot fire.

func (*Rule) FiltersOrigin 

func (r *Rule) FiltersOrigin() bool

FiltersOrigin reports whether the rule looks at where an offender is from — and so needs the geo databases to run.

func (*Rule) FiltersTarget 

func (r *Rule) FiltersTarget() bool

FiltersTarget reports whether the rule looks at what a hit was aimed at.

func (*Rule) ReportName 

func (r *Rule) ReportName() string

ReportName is the preset this rule reports as.

func (*Rule) Shared 

func (r *Rule) Shared() bool

Shared reports whether the rule's bans may leave for the fleet.

type Secrets 

type Secrets struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`

	AbuseIPDB ProviderSecret `yaml:"abuseipdb"`
}

Secrets is secrets.yaml: credentials and the allowances that go with them, in a file of their own.

Separate from everything else on purpose. A file that exists only to hold secrets is unambiguous to exclude from a backup, a copy or a repository; "the configuration, except the parts with keys in them" is a judgement someone eventually gets wrong, and a key in a repository is a key that has to be rotated.

type Size 

type Size struct {
	Bytes   int64   // absolute, when written that way
	Percent float64 // a share of the machine, when written that way
}

Size is an amount of memory as written in config files, either absolute — "512KB", "64MB", "2GB", "1TB", in powers of 1024, the units the kernel thinks in — or a share of the machine's memory, "50%".

The share form exists because the right ceiling for this program is not a constant: the same binary runs on a small virtual server and on a host with a terabyte, and an operator should not have to work out a number for a program that can read how much memory the box has.

func ParseSize 

func ParseSize(s string) (Size, error)

ParseSize parses "64MB", "2GB", "4096B" or "50%".

func (Size) IsZero 

func (z Size) IsZero() bool

IsZero reports whether nothing was written.

func (Size) Resolve 

func (z Size) Resolve(total int64) int64

Resolve turns a size into bytes against the machine it is running on. A share of a machine that will not say how much memory it has cannot be resolved, and the caller falls back to its own default.

func (Size) String 

func (z Size) String() string

func (*Size) UnmarshalYAML 

func (z *Size) UnmarshalYAML(n *yaml.Node) error

type Threshold 

type Threshold struct {
	Points int
	Window Duration
}

Threshold is one crossing condition: this many points within this window. Written "5/10m" — the count first because it is what one tunes.

func ParseThreshold 

func ParseThreshold(s string) (Threshold, error)

ParseThreshold parses "5/10m".

func Scale 

func Scale(level string, ths []Threshold) []Threshold

Scale is what a level does to a rule's thresholds: instant makes every crossing one point; strict halves the points (rounded up, never below one); lenient doubles them; standard leaves them. Windows never move.

func (Threshold) String 

func (t Threshold) String() string

func (*Threshold) UnmarshalYAML 

func (t *Threshold) UnmarshalYAML(n *yaml.Node) error

type Thresholds 

type Thresholds struct {
	Any []Threshold
}

Thresholds is a rule's `threshold:` — one crossing ("5/10m") or a list, the first crossed fires. Any holds them; the name survives from the days a rule wrote a set per level.

func Crossing 

func Crossing(points int, window Duration) Thresholds

Crossing is a Thresholds of one crossing — for rules built in code.

func (Thresholds) IsZero 

func (t Thresholds) IsZero() bool

IsZero reports an absent threshold key.

func (*Thresholds) UnmarshalYAML 

func (t *Thresholds) UnmarshalYAML(n *yaml.Node) error

type TrustEntry 

type TrustEntry struct {
	File string `yaml:"-"`
	Line int    `yaml:"-"`

	Origin   Origin     `yaml:"origin"`
	Require  Multiplier `yaml:"require"`   // evidence multiplier, e.g. "3x"
	RangeBan string     `yaml:"range_ban"` // "never" to forbid range bans inside
}

TrustEntry is one graduated weight on an origin — never on a target.

type Words 

type Words []string

Words is a list written either as a YAML list or as one string of comma- or space-separated words — `countries: [FR, MA]` and `countries: FR, MA` say the same thing. Kept as written otherwise.

func (*Words) UnmarshalYAML 

func (w *Words) UnmarshalYAML(n *yaml.Node) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.