go-imap-assassin

command module
v0.0.0-...-48d6722 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 30, 2020 License: GPL-3.0 Imports: 12 Imported by: 0

README

go-imap-assassin

go-imap-assassin is a command-line application that leverages the popular spam classifiers SpamAssassin or Rspamd to remove spam in IMAP mailboxes.

Motivation

Most mailbox providers do some form of spam-checking for clients. The quality of the filter heavily depends on the specific provider, and the providers offer different levels of control over the filter, such as improving the personal classifier by "learning" known bad (spam) or good (ham) mails.

If the chosen mail providers' spam classification doesn't work well enough, short of switching mailboxes (which usually means also switching addresses), the client is usually bound by the level of control the provider offers.

This is where go-imap-assassin comes into play: it uses the popular Spamassassin or Rspamd anti-spam platform to classify mail that is already in the client's mailbox by retrieving it via IMAP. According to the classification done by SpamAssassin or Rspamd, the respective spam mails can be moved, deleted etc. by go-imap-assassin.

This implementation uses the same concept implemented in isbg. However, this is a full re-implementation that leverages Go's very rich standard library & concurrency features. See below for a comparison with isbg. I decided to create this re-write because my mailbox contained mails that Python3's mail implementation couldn't process and after a couple of hours of trying to solve it in Python3, I was unable to get it done. Go's standard library has no problems with the mails I tried.

Features

  • Broad IMAP compatibility, making use of IMAP extensions when available
  • Efficient handling of IMAP specifics, such as UIDVALIDITY changes
  • Robust mail parsing via Go's standard library
  • Concurrent access to SpamAssassin or Rspamd to improve classification throughput
  • Stores mail UIDs plus metadata in a standard sqlite database

Development progress

Although the core functionality is implemented and I'm slowly starting to use this on my personal mailbox, this is not a finished product.

I'm working on this in my spare time and will address the remaining to-dos as time permits. The following should give an approximate overview:

  • Implement IMAP & SpamAssassin access
  • Implement Rspamd as a classifier
  • Implement mail filter & delete/move
  • Implement filter training
  • Implement compatibility for IMAP servers without MOVE or UIDPLUS
  • Do some basic testing with multiple imap servers
  • Add unit tests
  • Add basic CI/CD chain
  • Add multiplatform release CI/CD using xgo
  • Document build steps
  • Document application code
  • Document setup steps (including SpamAssassin and Rspamd setup)
  • Document configuration
  • Add continuous running mode using IMAP-IDLE
  • Broader tests with more IMAP servers

Comparison with isbg

See below for a comparison with isbg. Most information is taken from the Gitlab page, some are my interpretations of the source code.

Feature go-imap-assassin isbg
Programming language Go Python3
Classifier support SpamAssassin, Rspamd SpamAssassin
Classifiers access concurrent single-threaded
Already-processed detection UID- and header-based, fast diff mechanism, sqlite storage UID-based, UIDVALIDITY change will trigger rescan, json file storage
Configuration file-based, toml configuration commandline-parameter based configuration
Maturity Work in progress Stable with considerable userbase
Tests Work in progress Unit tests
Maintainership & community Maintained and used by @CrawX Maintained by multiple people, mostly by @baldurmen, community which reports and fixes bugs

Build instructions

go-imap-assassin uses go-sqlite3 which is a cgo-enabled package. You'll need a working c compiler like gcc to build go-imap-assassin.

# install asset generation
GO111MODULE="off" go get -u github.com/mjibson/esc
GO111MODULE="off" go install github.com/mjibson/esc

# download sources
git clone 'https://github.com/CrawX/go-imap-assassin.git'
cd go-imap-assassin
# generate migration assets
cd persistence/migrations
go generate
cd ../..
# download dependecies & build go-imap-assassin
go mod download
go build

Rspamd setup

The following docker-compose.yml can be used as a starting point to deploy a docker-based installation of rspamd to use with go-imap-assassin.

Generate your rspamd admin password to store in worker-controller.inc as described in the rspamd image documentation.

version: "3.8"

services:
  unbound:
    image: klutchell/unbound:latest

  dccifd:
    image: a16bitsysop/dccifd:latest

  redis:
    image: redis:latest
    volumes:
      - MY-ABSOLUTE-REDIS-DATA-DIR:/data

  rspamd:
    image: crawxx/rspamd-go-imap-assassin
    volumes:
      - MY-ABSOLUTE-RSPAMD-DATA-DIR:/var/lib/rspamd
      - MY-GENERATED-worker-controller.inc:/etc/rspamd/override.d/worker-controller.inc
    ports:
      - "11332:11332"
      - "11334:11334"

Acknowledgements

  • isbg for an alternative and more mature implementation of the same concept
  • go-imap for a fully fledged IMAP library
  • sqlite for being a fantastic embeddable database and go-sqlite3 for allow easy integration with Go
  • SpamAssassin for offering a very good classifier thatn can easily outperform the classifier offered by my mailbox provider
  • Rspamd for offering a more modern and more efficient alternative to SpamAssassin
  • spamc for offering a Go-native way to access spamassassin
  • Various other libraries I used such as toml, sqlx or logrus

License

This application is licensed under GPLv3. There is no warranty, to the extent permitted by law.

Documentation

Overview

SPDX-License-Identifier: GPL-3.0-or-later

Source Files

View all Source files

Directories

Path Synopsis
SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later
rspamd
SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
spamassassin
SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later
SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later
SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later SPDX-License-Identifier: GPL-3.0-or-later
SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later
SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later
SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later
migrations
SPDX-License-Identifier: GPL-3.0-or-later
 SPDX-License-Identifier: GPL-3.0-or-later

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.