traefik_botfilter

README

Traefik Bot Filter

traefik-botfilter is a dependency-free Traefik middleware plugin that rejects common scanner requests and temporarily bans suspicious clients with bounded, in-memory per-IP scoring. It is a lightweight first layer of protection, not a replacement for a WAF.

Features

• Blocks configured paths, extensions, and User-Agent substrings before the request reaches the backend.
• Optionally requires User-Agent, Accept, and Host headers.
• Detects internally inconsistent browser User-Agents.
• Scores suspicious requests and backend 404 responses, then applies a temporary ban.
• Supports CIDR allowlists and trusted proxy client-IP headers.
• Uses only the Go standard library and keeps its per-client cache bounded.

Install From The Plugin Catalog

After a release tag has been accepted by the catalog, declare the plugin in Traefik's static configuration:

experimental:
  plugins:
    botfilter:
      moduleName: github.com/hoelee/traefik-botfilter
      version: v0.1.1

Declare a middleware in dynamic configuration and attach it to a router:

http:
  middlewares:
    botfilter:
      plugin:
        botfilter:
          requireUserAgent: true
          requireAccept: true
          blockedPaths:
            - /.env
            - /.git
          blockedUserAgents:
            - sqlmap
            - nikto

  routers:
    app:
      rule: Host(`app.example.com`)
      middlewares:
        - botfilter
      service: app

Plugin names have two distinct roles above: botfilter is the local Traefik plugin identifier, while github.com/hoelee/traefik-botfilter is its module path. Keep the identifier the same under experimental.plugins and http.middlewares.<name>.plugin.

Local Development

Traefik local plugins must be placed under a directory matching the module path:

plugins-local/
  src/
    github.com/
      hoelee/
        traefik-botfilter/
          .traefik.yml
          go.mod
          botfilter.go
          ...

Use localPlugins instead of plugins in static configuration:

experimental:
  localPlugins:
    botfilter:
      moduleName: github.com/hoelee/traefik-botfilter

Example

example/traefik.yml enables the local plugin and reads example/dynamic.yml. The dynamic configuration protects example.localhost and forwards accepted requests to a backend listening on 127.0.0.1:8081.

Start a test backend, for example:

go run github.com/traefik/whoami@latest --port 8081

Start Traefik with the example configuration, making the repository available at /plugins-local/src/github.com/hoelee/traefik-botfilter in the Traefik process or container. Then test it:

# A browser-like request reaches the backend.
curl -i http://example.localhost/ \
  -H "User-Agent: Mozilla/5.0 Chrome/120.0 AppleWebKit/537.36 Safari/537.36" \
  -H "Accept: text/html"

# Scanner paths and blocked user agents receive 403.
curl -i http://example.localhost/.env
curl -i http://example.localhost/ -A sqlmap

Configuration

Option	Default	Description
statusCode	403	HTTP status returned for blocked requests.
requireUserAgent	false	Ban clients with no User-Agent.
requireAccept	false	Ban clients with no Accept header.
requireHost	false	Ban requests with no Host.
browserValidation	false	Score inconsistent browser User-Agents.
temporaryBanMinutes	15	Duration of a temporary ban.
scoreThreshold	100	Score at which a client is banned.
scoreWindowMinutes	15	Time window for score accumulation.
maxTrackedIPs	50000	Maximum cache entries.
maxScoreEventsPerIP	16	Maximum score events retained per client.
whitelistCIDRs	none	CIDRs that bypass all filtering.
blockedUserAgents	none	Case-insensitive User-Agent substrings to block.
blockedPaths	none	Paths and path prefixes to block.
blockedExtensions	none	File extensions to block.
randomArticlePatterns	/content/	Path prefixes that add a small first-request score.
clientIPHeader	empty	Client-IP header accepted from trusted proxies.
trustedProxyCIDRs	none	Proxy CIDRs allowed to supply clientIPHeader.
logBlockedRequests	false	Log blocked requests through Traefik's process logger.

Score options are emptyUserAgentScore (40), missingAcceptScore (20), blockedUserAgentScore (80), badPathScore (50), randomArticleScore (15), notFoundScore (40), and fakeBrowserScore (40). Set a score to 0 to disable that signal.

Proxy Configuration

Only configure clientIPHeader when Traefik receives traffic from a proxy you trust. Also set trustedProxyCIDRs; the plugin ignores the header for all other remote addresses to prevent clients from choosing their own cache identity.

clientIPHeader: X-Forwarded-For
trustedProxyCIDRs:
  - 10.0.0.0/8

Ensure the trusted proxy overwrites, rather than appends untrusted values to, that header.

Publishing To The Catalog

The repository must be public, have the traefik-plugin GitHub topic, contain a valid root .traefik.yml manifest, and have an annotated or lightweight Git tag for each release. Push a new semantic-version tag after merging the package-name fix, for example:

git tag v1.0.1
git push origin v1.0.1

The package is intentionally named traefik_botfilter: Traefik's Yaegi loader derives that Go identifier from the final module path segment, replacing - with _. This must remain aligned with github.com/hoelee/traefik-botfilter for CreateConfig and New to load from the catalog.

Development

go test ./...
go vet ./...
gofmt -w *.go

Limitations

This plugin reduces opportunistic scans and low-effort bot traffic. It does not reliably protect against distributed botnets, residential proxies, sophisticated browser automation, or application vulnerabilities. Pair it with rate limiting, a CDN or WAF, TLS, and application-level security controls for Internet-facing services.

Contributing

Issues and pull requests are welcome. Include the Traefik version, plugin version, relevant configuration, and logs when reporting a problem.

License

This project is licensed under the MIT License.

Documentation

Overview

Package traefik_botfilter is a Traefik middleware plugin that blocks known scans and applies a bounded, in-memory per-IP suspicion score.

Index

• func New(_ context.Context, next http.Handler, config *Config, name string) (http.Handler, error)
• type Config
  • func CreateConfig() *Config

Functions

func New

func New(_ context.Context, next http.Handler, config *Config, name string) (http.Handler, error)

New constructs a Traefik middleware. Its signature is the interface used by Traefik's Go plugin runtime.

Types

type Config

type Config struct {
	StatusCode            int      `json:"statusCode,omitempty" yaml:"statusCode,omitempty" toml:"statusCode,omitempty"`
	RequireUserAgent      bool     `json:"requireUserAgent,omitempty" yaml:"requireUserAgent,omitempty" toml:"requireUserAgent,omitempty"`
	RequireAccept         bool     `json:"requireAccept,omitempty" yaml:"requireAccept,omitempty" toml:"requireAccept,omitempty"`
	RequireHost           bool     `json:"requireHost,omitempty" yaml:"requireHost,omitempty" toml:"requireHost,omitempty"`
	BrowserValidation     bool     `json:"browserValidation,omitempty" yaml:"browserValidation,omitempty" toml:"browserValidation,omitempty"`
	WhitelistCIDRs        []string `json:"whitelistCIDRs,omitempty" yaml:"whitelistCIDRs,omitempty" toml:"whitelistCIDRs,omitempty"`
	TemporaryBanMinutes   int      `json:"temporaryBanMinutes,omitempty" yaml:"temporaryBanMinutes,omitempty" toml:"temporaryBanMinutes,omitempty"`
	BlockedUserAgents     []string `json:"blockedUserAgents,omitempty" yaml:"blockedUserAgents,omitempty" toml:"blockedUserAgents,omitempty"`
	BlockedPaths          []string `json:"blockedPaths,omitempty" yaml:"blockedPaths,omitempty" toml:"blockedPaths,omitempty"`
	BlockedExtensions     []string `json:"blockedExtensions,omitempty" yaml:"blockedExtensions,omitempty" toml:"blockedExtensions,omitempty"`
	ScoreThreshold        int      `json:"scoreThreshold,omitempty" yaml:"scoreThreshold,omitempty" toml:"scoreThreshold,omitempty"`
	ScoreWindowMinutes    int      `json:"scoreWindowMinutes,omitempty" yaml:"scoreWindowMinutes,omitempty" toml:"scoreWindowMinutes,omitempty"`
	MaxTrackedIPs         int      `json:"maxTrackedIPs,omitempty" yaml:"maxTrackedIPs,omitempty" toml:"maxTrackedIPs,omitempty"`
	MaxScoreEventsPerIP   int      `json:"maxScoreEventsPerIP,omitempty" yaml:"maxScoreEventsPerIP,omitempty" toml:"maxScoreEventsPerIP,omitempty"`
	EmptyUserAgentScore   int      `json:"emptyUserAgentScore,omitempty" yaml:"emptyUserAgentScore,omitempty" toml:"emptyUserAgentScore,omitempty"`
	MissingAcceptScore    int      `json:"missingAcceptScore,omitempty" yaml:"missingAcceptScore,omitempty" toml:"missingAcceptScore,omitempty"`
	BlockedUserAgentScore int      `json:"blockedUserAgentScore,omitempty" yaml:"blockedUserAgentScore,omitempty" toml:"blockedUserAgentScore,omitempty"`
	BadPathScore          int      `json:"badPathScore,omitempty" yaml:"badPathScore,omitempty" toml:"badPathScore,omitempty"`
	RandomArticleScore    int      `json:"randomArticleScore,omitempty" yaml:"randomArticleScore,omitempty" toml:"randomArticleScore,omitempty"`
	NotFoundScore         int      `json:"notFoundScore,omitempty" yaml:"notFoundScore,omitempty" toml:"notFoundScore,omitempty"`
	FakeBrowserScore      int      `json:"fakeBrowserScore,omitempty" yaml:"fakeBrowserScore,omitempty" toml:"fakeBrowserScore,omitempty"`
	RandomArticlePatterns []string `json:"randomArticlePatterns,omitempty" yaml:"randomArticlePatterns,omitempty" toml:"randomArticlePatterns,omitempty"`
	ClientIPHeader        string   `json:"clientIPHeader,omitempty" yaml:"clientIPHeader,omitempty" toml:"clientIPHeader,omitempty"`
	TrustedProxyCIDRs     []string `json:"trustedProxyCIDRs,omitempty" yaml:"trustedProxyCIDRs,omitempty" toml:"trustedProxyCIDRs,omitempty"`
	LogBlockedRequests    bool     `json:"logBlockedRequests,omitempty" yaml:"logBlockedRequests,omitempty" toml:"logBlockedRequests,omitempty"`
}

Config is the plugin configuration exposed by Traefik's dynamic file provider. All durations are expressed as integers because Traefik plugin configuration is deliberately kept YAML-friendly.

func CreateConfig

func CreateConfig() *Config

CreateConfig creates the default configuration. The defaults protect common public HTTP services without requiring a third-party dependency.