yarilo

module

v1.23.0 Latest Latest Go to latest Published: May 31, 2026 License: GPL-3.0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/0kaba0hub/yarilo

Links

Open Source Insights

README ¶

yarilo

Production-grade IMAP/Submission/LMTP/JMAP mail server written in Go. Three-tier cluster (proxy → director → backend), pluggable storage (Maildir / dbox / mdbox / S3), Sieve filtering, full Dovecot 2.3 protocol compatibility.

Architecture

Yarilo is a single binary that runs in one of three roles (mode: proxy | director | backend).

  Internet
     |
     | IMAP / IMAPs / SUBMISSION / POP3 / JMAP
     v
+------------------+     +------------------+
|     PROXY        |     |     PROXY        |  ...N
|                  |     |                  |
| TLS termination  |     | TLS termination  |
| Auth (passdb)    |     | Auth (passdb)    |
| Route lookup     |     | Route lookup     |
+--------+---------+     +--------+---------+
         |  TAB-delimited protocol           |
         v                                   v
+------------------------------------------------+
|                  DIRECTOR                      |
|                                                |
| Consistent hashing: user@domain -> backend     |
| Sticky sessions (one user = one backend)       |
| Backend registry + health checks              |
| Failover: reassign on node failure             |
+--------+----------+----------+----------------+
         |          |          |
         v          v          v
  +----------+ +----------+ +----------+
  | BACKEND  | | BACKEND  | | BACKEND  |  ...N
  |          | |          | |          |
  | IMAP     | | IMAP     | | IMAP     |
  | POP3     | | POP3     | | POP3     |
  | JMAP     | | JMAP     | | JMAP     |
  | Sub/LMTP | | Sub/LMTP | | Sub/LMTP |
  | Sieve    | | Sieve    | | Sieve    |
  | Quota    | | Quota    | | Quota    |
  | ACL      | | ACL      | | ACL      |
  +----+-----+ +----+-----+ +----+-----+
       |             |            |
       v             v            v
  +-----------+ +-----------+ +-----------+
  | Mailbox   | | Mailbox   | | Mailbox   |
  | + Index   | | + Index   | | + Index   |
  +-----------+ +-----------+ +-----------+
       |                            |
       v                            v
  [Maildir/dbox/mdbox]          [obox -> S3]
  [FileIndex / SQLite]          [Cassandra]

Proxy — accepts client connections, terminates TLS (SNI per-domain), authenticates via passdb, queries Director for routing, forwards the raw TCP stream to the target backend. Stateless; scale horizontally without limits.

Director — consistent-hashing ring (MD5, 100 vhosts/backend). One user always lands on the same backend. Detects failures and reassigns. All intra-cluster traffic uses the TAB-delimited yarilo-director protocol (see INTERNALS.md §2).

Backend — full mail server: IMAP, POP3, JMAP, Submission, LMTP, ManageSieve, Sieve execution engine, Quota, ACL, FTS. Speaks natively to the mailbox + index layer.

Protocol support

Protocol	Standard	Extensions
IMAP4rev2	RFC 9051	IDLE, MOVE, CONDSTORE, UNSELECT, NAMESPACE, QUOTA, ACL, BINARY, UIDPLUS, SORT, THREAD, ESEARCH, NOTIFY, QRESYNC, URLAUTH, SPECIAL-USE
Submission	RFC 6409	STARTTLS, AUTH PLAIN, SIZE, PIPELINING, relay to upstream MTA, ALPN matching, configurable `Received:` header
LMTP	RFC 2033	per-recipient status codes, proxy/director mode, HAProxy, XCLIENT (incl. `DESTADDR`/`DESTPORT`), STARTTLS, Delivered-To header, per-user concurrency limit, client workarounds
POP3	RFC 1939	STLS, UIDL, CAPA, XCLIENT
JMAP	RFC 8620, RFC 8621	HTTP dispatch, WebSocket push (RFC 8887)
ManageSieve	RFC 5804	full Sieve script management
Sieve	RFC 5228	fileinto, reject, vacation, notify, include, variables, date, relational, imap4flags, editheader, extlists, vnd.dovecot.*
SASL	—	PLAIN, LOGIN, GSSAPI, SCRAM-SHA-256, XOAUTH2, OAUTHBEARER
TLS	—	SNI, per-domain context reload

Storage

Layer	Backend	Notes
Mailbox	Maildir	one file per message, local FS
Mailbox	dbox (sdbox)	single-file dbox, local FS
Mailbox	mdbox	multi-message dbox, local FS
Mailbox	obox	S3-compatible object storage via minio-go
Index	FileIndex	custom binary format (`.index` / `.index.log` / `.index.cache`) alongside mailbox
Index	SQLite	dev / small deployments
Index	Cassandra	multi-node / large scale
FTS	built-in tokenizer	snowball stemmer + ICU normalizer
FTS	Solr	HTTP XML bridge

Deployment modes

Mode	Process layout	Use case
Single node	proxy + director + backend in one process	dev / small server
Multi-node	separate proxy / director / backend	production
Cloud	proxy + director + backend (obox + Cassandra)	large scale

Cluster components

Component	Role	Status
yarilo-auth	passdb chain, auth cache, SASL dispatch — TCP+mTLS	✅ v1.0.0
yarilo-director	consistent hashing, sticky sessions, failover	planned
yarilo-anvil	connection rate limiting + penalty algorithm	planned
yarilo-locks	cross-process write coordination — embedded (standalone) / remote Redis-backed (backend per tag)	planned
yarilo-dict	Redis / SQLite key-value abstraction	planned
yarilo-admin	control socket (kick, reload, stats)	planned
yarilo-stats	per-user / per-domain connection metrics	planned
imap-hibernate	idle IMAP connection parking via FD-passing	planned
yarilo-indexer	background FTS indexing service	planned

All intra-cluster protocols are TAB-delimited text with LF termination and a version handshake. Full wire-format specification: INTERNALS.md.

Quick start (single-node)

# yarilo.yaml
mode: single   # proxy | director | backend | single

general:
  ssl:
    tls_cert: /etc/ssl/yarilo/cert.pem
    tls_key:  /etc/ssl/yarilo/key.pem
  haproxy:
    trusted_nets: ["127.0.0.1/32", "10.0.0.0/8"]
    timeout: 3
  xclient:
    trusted_nets: ["127.0.0.1/32", "10.0.0.0/8"]
  limits:
    mail_max_userip_connections: 10

services:
  imaps:
    enabled: true
    port: 993
    ssl_mode: ssl
  imap:
    enabled: true
    port: 143
    ssl_mode: starttls
    disable_plaintext_auth: true
  submission:
    enabled: true
    port: 587
    ssl_mode: starttls
    disable_plaintext_auth: true
  lmtp:
    enabled: true
    port: 24
    ssl_mode: no
    xclient_protocol: true

protocol:
  imap:
    imap_idle_notify_interval: 120
    imap_max_line_length: 65536
    imap_id_send: "name *"
  submission:
    hostname: mail.example.com
    max_message_size: 41943040
    recipient_delimiter: "+"

auth:
  passdb:
    - driver: postgres
      dsn: "${DB_URL}"

# yarilo-auth standalone process (multi-process mode).
# Listen address and mTLS cert paths used by the yarilo-auth binary.
auth_service:
  listen: ":9100"
  mtls:
    cert: /etc/yarilo/tls/tls.crt
    key:  /etc/yarilo/tls/tls.key
    ca:   /etc/yarilo/tls/ca.crt
  shutdown:
    session_grace_period: 30  # seconds to drain sessions on SIGTERM
    kill_timeout: 5           # seconds before forced exit

storage:
  mailbox: maildir
  maildir_root: /var/mail/vhosts
  mail_home_template: "%d/%n"   # %d=domain %n=local %u=full email

log:
  level: info

yarilo -config yarilo.yaml

Set LOG_LEVEL=debug to enable verbose protocol tracing without restarting.

Mailbox backends

Value	Format	Description
`maildir`	Maildir	one file per message, `cur/` + `new/` + `tmp/`
`dbox`	sdbox	one file per message in dbox wire format (GUID + metadata embedded)
`mdbox`	mdbox	multiple messages per `m.<id>` file, higher density

yarilo-migrate \
  --from /var/mail/vhosts \
  --to   /var/mail/dbox \
  --format dbox   # or mdbox; --dry-run to preview

Documentation

Document	Contents
PLAN.md	Implementation plan, phases, library strategy, timelines
INTERNALS.md	Wire-format specs for all internal protocols (33 sections)
docs/GENERAL.md	`general`: shared SSL, HAProxy, XClient, connection limits
docs/SERVICES.md	`services`: per-listener config for all 7 listeners
docs/IMAP.md	`protocol.imap`: IDLE interval, line length, ID, logout format
docs/NAMESPACE.md	IMAP namespaces (RFC 2342 / 9051 §6.3.10): personal / shared / other_users config, separator semantics, phase roadmap (NS-1a wire → NS-1b storage routing → ACL → NS-3 cross-pod)
docs/SUBMISSION.md	`protocol.submission`: hostname, size limits, relay
docs/LMTP.md	`protocol.lmtp`: local delivery, proxy/director mode, HAProxy, XCLIENT, TLS, headers
docs/POP3.md	`protocol.pop3`: UIDL format, soft-delete, migration
docs/JMAP.md	JMAP (planned, Phase 5)
INSTALL.md	End-to-end Kubernetes deploy with cert-manager + Let's Encrypt
docs/AUTH.md	`auth.passdb`: SQL backends (SQLite/MySQL/Postgres), password schemes
docs/SMOKE.md	End-to-end smoke test: AUTH → LMTP → IMAPS/POP3S read
docs/DIRECTOR.md	`director_service`: ring config, peers, HAProxy, XCLIENT, mTLS, Helm values
docs/MONITOR.md	`yarilo-monitor` sidecar: backend health probes, tag credentials, Helm values, Prometheus metrics
docs/DIRECTOR-API.md	Director HTTP admin API (ring / backends / users / peers): auth, IP whitelist, endpoints
docs/BACKEND-API.md	Backend-plane HTTP admin API (`yarilo-backend-api` on `:9105`): `/api/backend/<service>/...` — dict surface today; ACL / quota / folder / user / mailbox in subsequent phases
docs/YARILO-ADMIN.md	`yarilo-admin` CLI: two top-level planes (`director`, `backend`), commands, flags, env vars, usage examples
docs/DICT.md	`pkg/dict` KV-store abstraction: drivers (file/redis/sql/memory/fail), YAML schema, `yarilo-admin dict` CLI

License

GNU General Public License v3.0 — see LICENSE.

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
app
smoketest command smoketest verifies a live yarilo deployment.	smoketest verifies a live yarilo deployment.
smoketest-e2e command smoketest-e2e drives a live yarilo instance end-to-end:	smoketest-e2e drives a live yarilo instance end-to-end:
smoketest-e2e/seed command seed creates the yarilo_users table (if missing) and inserts a single user with a bcrypt-hashed password.	seed creates the yarilo_users table (if missing) and inserts a single user with a bcrypt-hashed password.
yarilo command
yarilo-admin command yarilo-admin is the unified yarilo operator CLI.	yarilo-admin is the unified yarilo operator CLI.
yarilo-anvil command yarilo-anvil is the connection-accounting service for the yarilo mail server.	yarilo-anvil is the connection-accounting service for the yarilo mail server.
yarilo-auth command yarilo-auth is the standalone authentication service for the yarilo mail server.	yarilo-auth is the standalone authentication service for the yarilo mail server.
yarilo-backend-api command yarilo-backend-api is the backend-plane admin HTTP API.	yarilo-backend-api is the backend-plane admin HTTP API.
yarilo-director command yarilo-director is the consistent-hash routing service for the yarilo mail server.	yarilo-director is the consistent-hash routing service for the yarilo mail server.
yarilo-imap command yarilo-imap is the IMAP/IMAPS backend for the yarilo mail server.	yarilo-imap is the IMAP/IMAPS backend for the yarilo mail server.
yarilo-imap-login command yarilo-imap-login is the IMAP/IMAPS login proxy for the yarilo mail server.	yarilo-imap-login is the IMAP/IMAPS login proxy for the yarilo mail server.
yarilo-lmtp command yarilo-lmtp is the LMTP backend for the yarilo mail server.	yarilo-lmtp is the LMTP backend for the yarilo mail server.
yarilo-locks command yarilo-locks is the cross-process write-coordination service for yarilo.	yarilo-locks is the cross-process write-coordination service for yarilo.
yarilo-migrate command yarilo-migrate converts a Maildir tree to dbox or mdbox format in-place.	yarilo-migrate converts a Maildir tree to dbox or mdbox format in-place.
yarilo-monitor command yarilo-monitor is a sidecar health checker for yarilo-director pods.	yarilo-monitor is a sidecar health checker for yarilo-director pods.
yarilo-pop3 command yarilo-pop3 is the POP3/POP3S backend for the yarilo mail server.	yarilo-pop3 is the POP3/POP3S backend for the yarilo mail server.
yarilo-pop3-login command yarilo-pop3-login is the POP3/POP3S login proxy for the yarilo mail server.	yarilo-pop3-login is the POP3/POP3S login proxy for the yarilo mail server.
yarilo-submission command yarilo-submission is the SMTP submission proxy for the yarilo mail server.	yarilo-submission is the SMTP submission proxy for the yarilo mail server.
yarilo-submission-login command yarilo-submission-login is the SMTP submission login proxy for the yarilo mail server.	yarilo-submission-login is the SMTP submission login proxy for the yarilo mail server.
internal
anvil Package anvil implements the yarilo-anvil TCP+mTLS connection-accounting server.	Package anvil implements the yarilo-anvil TCP+mTLS connection-accounting server.
auth/protocol Package protocol implements the yarilo-auth TCP+mTLS protocol.	Package protocol implements the yarilo-auth TCP+mTLS protocol.
auth/sql Package sql implements SQL passdb/userdb for yarilo-auth.	Package sql implements SQL passdb/userdb for yarilo-auth.
backend Package backend wires all server components for the backend (or single) mode.	Package backend wires all server components for the backend (or single) mode.
backendapi Package backendapi is the backend-plane admin HTTP API.	Package backendapi is the backend-plane admin HTTP API.
cluster/proto Package proto implements the yarilo-director TAB-delimited wire protocol.	Package proto implements the yarilo-director TAB-delimited wire protocol.
cluster/ring Package ring implements consistent hashing for yarilo-director.	Package ring implements consistent hashing for yarilo-director.
connlimit Package connlimit provides per-user@IP connection counting for IMAP/POP3/SMTP.	Package connlimit provides per-user@IP connection counting for IMAP/POP3/SMTP.
director Package director — PeerDialer connects to sibling director replicas and propagates ring and user-directory changes so all replicas converge on the same state without a central coordinator.	Package director — PeerDialer connects to sibling director replicas and propagates ring and user-directory changes so all replicas converge on the same state without a central coordinator.
imap Package imap wires go-imap/v2 to yarilo's mailbox and index backends.	Package imap wires go-imap/v2 to yarilo's mailbox and index backends.
lmtp Package lmtp implements an LMTP server (RFC 2033) for local mail delivery.	Package lmtp implements an LMTP server (RFC 2033) for local mail delivery.
login Package login implements the yarilo mail-protocol login proxy.	Package login implements the yarilo mail-protocol login proxy.
monitor Package monitor implements yarilo-monitor: a sidecar health checker that probes backend pods via IMAP/POP3/LMTP login and reports state changes to the director (BACKEND-FLUSH on failure, BACKEND-UP on recovery).	Package monitor implements yarilo-monitor: a sidecar health checker that probes backend pods via IMAP/POP3/LMTP login and reports state changes to the director (BACKEND-FLUSH on failure, BACKEND-UP on recovery).
pop3 Package pop3 implements a POP3 server (RFC 1939).	Package pop3 implements a POP3 server (RFC 1939).
storage/index/file Package file implements the FileIndex IndexBackend.	Package file implements the FileIndex IndexBackend.
storage/mailbox/dbox Package dbox implements MailboxBackend for the sdbox (single-message dbox) format.	Package dbox implements MailboxBackend for the sdbox (single-message dbox) format.
storage/mailbox/maildir Package maildir implements MailboxBackend for Maildir format.	Package maildir implements MailboxBackend for Maildir format.
storage/mailbox/mdbox Package mdbox implements MailboxBackend for the mdbox (multi-message dbox) format.	Package mdbox implements MailboxBackend for the mdbox (multi-message dbox) format.
submission Package submission implements the submission servers (port 587 / 465).	Package submission implements the submission servers (port 587 / 465).
submission/proxy Package proxy implements the outbound submission proxy.	Package proxy implements the outbound submission proxy.
telemetry Package telemetry exposes /healthz, /readyz, and /metrics on a dedicated port.	Package telemetry exposes /healthz, /readyz, and /metrics on a dedicated port.
xclient Package xclient implements the XCLIENT SMTP extension (Postfix-compatible).	Package xclient implements the XCLIENT SMTP extension (Postfix-compatible).
pkg
config
dict Package dict is yarilo's pluggable key-value store abstraction.	Package dict is yarilo's pluggable key-value store abstraction.
dict/dicttest Package dicttest is a contract test suite that any pkg/dict driver can run against.	Package dicttest is a contract test suite that any pkg/dict driver can run against.
dict/drivers/all Package all blank-imports every in-tree dict driver so a single import enables all of them in the importing binary.	Package all blank-imports every in-tree dict driver so a single import enables all of them in the importing binary.
dict/fail Package fail is a dict driver that fails every operation.	Package fail is a dict driver that fails every operation.
dict/file Package file is a JSON-backed local-filesystem dict driver.	Package file is a JSON-backed local-filesystem dict driver.
dict/memory Package memory is an in-process dict driver backed by a sync.Map.	Package memory is an in-process dict driver backed by a sync.Map.
dict/redis Package redis is a Redis-backed dict driver.	Package redis is a Redis-backed dict driver.
dict/sql Package sql is a database/sql-backed dict driver.	Package sql is a database/sql-backed dict driver.
dict/varexpand Package varexpand performs %-variable substitution for templated path / prefix strings used by dict drivers.	Package varexpand performs %-variable substitution for templated path / prefix strings used by dict drivers.
locks Package locks is the single cross-process write-coordination abstraction for yarilo.	Package locks is the single cross-process write-coordination abstraction for yarilo.
mailbox
mtls Package mtls provides helpers for building mutual-TLS configs used between yarilo components (auth, anvil, director, imap-login → imap, etc.).	Package mtls provides helpers for building mutual-TLS configs used between yarilo components (auth, anvil, director, imap-login → imap, etc.).