store

package
v0.0.15 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 18, 2025 License: MIT Imports: 56 Imported by: 9

Documentation

Overview

Package store implements storage for accounts, their mailboxes, IMAP subscriptions and messages, and broadcasts updates (e.g. mail delivery) to interested sessions (e.g. IMAP connections).

Layout of storage for accounts:

<DataDir>/accounts/<name>/index.db
<DataDir>/accounts/<name>/msg/[a-zA-Z0-9_-]+/<id>

Index.db holds tables for user information, mailboxes, and messages. Message contents are stored in the msg/ subdirectory, each in their own file. The on-disk message does not contain headers generated during an incoming SMTP transaction, such as Received and Authentication-Results headers. Those are in the database to prevent having to rewrite incoming messages (e.g. Authentication-Result for DKIM signatures can only be determined after having read the message). Messages must be read through MsgReader, which transparently adds the prefix from the database.

Index

Constants

View Source
const MessageParseVersionLatest = 2

Variables

View Source
var (
	ErrUnknownMailbox     = errors.New("no such mailbox")
	ErrUnknownCredentials = errors.New("credentials not found")
	ErrAccountUnknown     = errors.New("no such account")
	ErrOverQuota          = errors.New("account over quota")
	ErrLoginDisabled      = errors.New("login disabled for account")
)
View Source
var AuthDB *bstore.DB

AuthDB and AuthDBTypes are exported for ../backup.go.

View Source
var CheckConsistencyOnClose = true

If true, each time an account is closed its database file is checked for consistency. If an inconsistency is found, panic is called. Set by default because of all the packages with tests, the mox main function sets it to false again.

View Source
var CommPendingChangesMax = 10000

CommPendingChangesMax is the maximum number of changes kept for a Comm before registering a notification overflow and flushing changes. Variable because set to low value during tests.

Types stored in DB.

View Source
var DefaultInitialMailboxes = config.InitialMailboxes{
	SpecialUse: config.SpecialUseMailboxes{
		Sent:    "Sent",
		Archive: "Archive",
		Trash:   "Trash",
		Draft:   "Drafts",
		Junk:    "Junk",
	},
}
View Source
var ErrMailboxExpunged = errors.New("mailbox was deleted")
View Source
var ErrNoJunkFilter = errors.New("junkfilter: not configured")

ErrNoJunkFilter indicates user did not configure/enable a junk filter.

View Source
var FlagsAll = Flags{true, true, true, true, true, true, true, true, true, true}

FlagsAll is all flags set, for use as mask.

View Source
var InitialUIDValidity = func() uint32 {
	return uint32(time.Now().Unix() >> 1)
}

InitialUIDValidity returns a UIDValidity used for initializing an account. It can be replaced during tests with a predictable value.

Functions

func BroadcastChanges added in v0.0.6

func BroadcastChanges(acc *Account, ch []Change)

BroadcastChanges ensures changes are sent to all listeners on the accoount.

func CheckKeyword added in v0.0.6

func CheckKeyword(kw string) error

CheckKeyword returns an error if kw is not a valid keyword. Kw should already be in lower-case.

func CheckMailboxName added in v0.0.6

func CheckMailboxName(name string, allowInbox bool) (normalizedName string, isInbox bool, rerr error)

CheckMailboxName checks if name is valid, returning an INBOX-normalized name. I.e. it changes various casings of INBOX and INBOX/* to Inbox and Inbox/*. Name is invalid if it contains leading/trailing/double slashes, or when it isn't unicode-normalized, or when empty or has special characters.

If name is the inbox, and allowInbox is false, this is indicated with the isInbox return parameter. For that case, and for other invalid names, an error is returned.

func Close added in v0.0.14

func Close() error

Close closes auth.db and stops the login writer.

func CloseRemoveTempFile added in v0.0.8

func CloseRemoveTempFile(log mlog.Log, f *os.File, descr string)

CloseRemoveTempFile closes and removes f, a file described by descr. Often used in a defer after creating a temporary file.

func CreateMessageTemp

func CreateMessageTemp(log mlog.Log, pattern string) (*os.File, error)

CreateMessageTemp creates a temporary file, e.g. for delivery. The is created in subdirectory tmp of the data directory, so the file is on the same file system as the accounts directory, so renaming files can succeed. The caller is responsible for closing and possibly removing the file. The caller should ensure the contents of the file are synced to disk before attempting to deliver the message.

func ExportMessages

func ExportMessages(ctx context.Context, log mlog.Log, db *bstore.DB, accountDir string, archiver Archiver, maildir bool, mailboxOpt string, messageIDsOpt []int64, recursive bool) error

ExportMessages writes messages to archiver. Either in maildir format, or otherwise in mbox. If mailboxOpt is non-empty, all messages from that mailbox are exported. If messageIDsOpt is non-empty, only those message IDs are exported. If both are empty, all mailboxes and all messages are exported. mailboxOpt and messageIDsOpt cannot both be non-empty.

Some errors are not fatal and result in skipped messages. In that happens, a file "errors.txt" is added to the archive describing the errors. The goal is to let users export (hopefully) most messages even in the face of errors.

func Init added in v0.0.14

func Init(ctx context.Context) error

Init opens auth.db and starts the login writer.

func LastKnown added in v0.0.14

func LastKnown() (current, lastknown updates.Version, mtime time.Time, rerr error)

LastKnown returns the last known version that has been mentioned in an update email, or the current application.

func LoginAttemptAdd added in v0.0.15

func LoginAttemptAdd(ctx context.Context, log mlog.Log, a LoginAttempt)

LoginAttemptAdd logs a login attempt (with result), and upserts it in the database and possibly cleans up old entries in the database.

Use account name "(admin)" for admin logins.

Writes are done in a background routine, unless we are shutting down or when there are many pending writes.

func LoginAttemptCleanup added in v0.0.15

func LoginAttemptCleanup(ctx context.Context) error

LoginAttemptCleanup removes any LoginAttempt entries older than 30 days, for all accounts.

func LoginAttemptTLS added in v0.0.15

func LoginAttemptTLS(state *tls.ConnectionState) string

LoginAttemptTLS returns a string for use as LoginAttempt.TLS. Returns an empty string if "c" is not a TLS connection.

func MergeKeywords added in v0.0.5

func MergeKeywords(l, add []string) ([]string, bool)

MergeKeywords adds keywords from add into l, returning whether it added any keyword, and the slice with keywords, a new slice if modifications were made. Keywords are only added if they aren't already present. Should only be used with keywords, not with system flags like \Seen.

func MessagePath

func MessagePath(messageID int64) string

MessagePath returns the filename of the on-disk filename, relative to the containing directory such as <account>/msg or queue. Returns names like "AB/1".

func MessageRuleset

func MessageRuleset(log mlog.Log, dest config.Destination, m *Message, msgPrefix []byte, msgFile *os.File) *config.Ruleset

MessageRuleset returns the first ruleset (if any) that matches the message represented by msgPrefix and msgFile, with smtp and validation fields from m.

func MsgFilesPerDirShiftSet added in v0.0.15

func MsgFilesPerDirShiftSet(shift int)

func ParseDovecotKeywordsFlags added in v0.0.6

func ParseDovecotKeywordsFlags(r io.Reader, log mlog.Log) ([]string, error)

ParseDovecotKeywordsFlags attempts to parse a dovecot-keywords file. It only returns valid flags/keywords, as lower-case. If an error is encountered and returned, any keywords that were found are still returned. The returned list has both system/well-known flags and custom keywords.

func RemoveKeywords added in v0.0.5

func RemoveKeywords(l, remove []string) ([]string, bool)

RemoveKeywords removes keywords from l, returning whether any modifications were made, and a slice, a new slice in case of modifications. Keywords must have been validated earlier, e.g. through ParseFlagKeywords or CheckKeyword. Should only be used with valid keywords, not with system flags like \Seen.

func SessionAdd added in v0.0.9

func SessionAdd(ctx context.Context, log mlog.Log, accountName, loginAddress string) (session SessionToken, csrf CSRFToken, rerr error)

SessionAdd creates a new session token, with csrf token, and adds it to the database and in-memory session cache. If there are too many sessions, the oldest is removed.

func SessionAddToken added in v0.0.9

func SessionAddToken(ctx context.Context, log mlog.Log, ls *LoginSession) error

SessionAddTokens adds a prepared or pre-existing LoginSession to the database and cache. Can be used to restore a session token that was used to reset a password.

func SessionRemove added in v0.0.9

func SessionRemove(ctx context.Context, log mlog.Log, accountName string, sessionToken SessionToken) error

SessionRemove removes a session from the database and in-memory cache. Future operations using the session token will fail.

func StartAuthCache added in v0.0.2

func StartAuthCache()

StartAuthCache starts a goroutine that regularly clears the auth cache.

func StoreLastKnown added in v0.0.14

func StoreLastKnown(v updates.Version) error

StoreLastKnown stores the the last known version. Future update checks compare against it, or the currently running version, whichever is newer.

func Switchboard

func Switchboard() (stop func())

Switchboard distributes changes to accounts to interested listeners. See Comm and Change.

func TLSPublicKeyAdd added in v0.0.14

func TLSPublicKeyAdd(ctx context.Context, pubKey *TLSPublicKey) error

TLSPublicKeyAdd adds a new tls public key.

Caller is responsible for checking the account and email address are valid.

func TLSPublicKeyRemove added in v0.0.14

func TLSPublicKeyRemove(ctx context.Context, fingerprint string) error

TLSPublicKeyRemove removes a tls public key.

func TLSPublicKeyUpdate added in v0.0.14

func TLSPublicKeyUpdate(ctx context.Context, pubKey *TLSPublicKey) error

TLSPublicKeyUpdate updates an existing tls public key.

Caller is responsible for checking the account and email address are valid.

Types

type Account

type Account struct {
	Name   string     // Name, according to configuration.
	Dir    string     // Directory where account files, including the database, bloom filter, and mail messages, are stored for this account.
	DBPath string     // Path to database with mailboxes, messages, etc.
	DB     *bstore.DB // Open database connection.

	// Write lock must be held when modifying account/mailbox/message/flags/annotations
	// if the change needs to be synchronized with client connections by broadcasting
	// the changes. Changes that are not protocol-visible do not require a lock, the
	// database transactions isolate activity, though locking may be necessary to
	// protect in-memory-only access.
	//
	// Read lock for reading mailboxes/messages as a consistent snapsnot (i.e. not
	// concurrent changes). For longer transactions, e.g. when reading many messages,
	// the lock can be released while continuing to read from the transaction.
	//
	// When making changes to mailboxes/messages, changes must be broadcasted before
	// releasing the lock to ensure proper UID ordering.
	sync.RWMutex
	// contains filtered or unexported fields
}

Account holds the information about a user, includings mailboxes, messages, imap subscriptions.

func OpenAccount

func OpenAccount(log mlog.Log, name string, checkLoginDisabled bool) (*Account, error)

OpenAccount opens an account by name.

No additional data path prefix or ".db" suffix should be added to the name. A single shared account exists per name.

func OpenAccountDB added in v0.0.6

func OpenAccountDB(log mlog.Log, accountDir, accountName string) (a *Account, rerr error)

OpenAccountDB opens an account database file and returns an initialized account or error. Only exported for use by subcommands that verify the database file. Almost all account opens must go through OpenAccount/OpenEmail/OpenEmailAuth.

func OpenEmail

func OpenEmail(log mlog.Log, email string, checkLoginDisabled bool) (*Account, string, config.Destination, error)

OpenEmail opens an account given an email address.

The email address may contain a catchall separator.

Returns account on success, may return non-empty account name even on error.

func OpenEmailAuth

func OpenEmailAuth(log mlog.Log, email string, password string, checkLoginDisabled bool) (racc *Account, raccName string, rerr error)

OpenEmailAuth opens an account given an email address and password.

The email address may contain a catchall separator. For invalid credentials, a nil account is returned, but accName may be non-empty.

func (*Account) AddMessageSize added in v0.0.9

func (a *Account) AddMessageSize(log mlog.Log, tx *bstore.Tx, size int64) error

AddMessageSize adjusts the DiskUsage.MessageSize by size.

func (*Account) AssignThreads added in v0.0.7

func (a *Account) AssignThreads(ctx context.Context, log mlog.Log, txOpt *bstore.Tx, startMessageID int64, batchSize int, xprogressWriter io.Writer) error

AssignThreads assigns thread-related fields to messages with ID >= startMessageID. Changes are committed each batchSize changes if txOpt is nil (i.e. during automatic account upgrade, we don't want to block database access for a long time). If txOpt is not nil, all changes are made in that transaction.

When resetting thread assignments, the caller must first clear the existing thread fields.

Messages are processed in order of ID, so when added to the account, not necessarily by received/date. Most threaded messages can immediately be matched to their parent message. If not, we keep track of the missing message-id and resolve as soon as we encounter it. At the end, we resolve all remaining messages, they start with a cycle.

Does not set Seen flag for muted threads.

Progress is written to progressWriter, every 100k messages.

func (*Account) CanAddMessageSize added in v0.0.9

func (a *Account) CanAddMessageSize(tx *bstore.Tx, size int64) (ok bool, maxSize int64, err error)

CanAddMessageSize checks if a message of size bytes can be added, depending on total message size and configured quota for account.

func (*Account) CheckConsistency added in v0.0.6

func (a *Account) CheckConsistency() error

CheckConsistency checks the consistency of the database and returns a non-nil error for these cases:

- Missing or unexpected on-disk message files. - Mismatch between message size and length of MsgPrefix and on-disk file. - Incorrect mailbox counts. - Incorrect total message size. - Message with UID >= mailbox uid next. - Mailbox uidvalidity >= account uid validity. - Mailbox ModSeq > 0, CreateSeq > 0, CreateSeq <= ModSeq, and Modseq >= highest message ModSeq. - Mailbox must have a live parent ID if they are live themselves, live names must be unique. - Message ModSeq > 0, CreateSeq > 0, CreateSeq <= ModSeq. - All messages have a nonzero ThreadID, and no cycles in ThreadParentID, and parent messages the same ThreadParentIDs tail. - Annotations must have ModSeq > 0, CreateSeq > 0, ModSeq >= CreateSeq and live keys must be unique per mailbox. - Recalculate junk filter (words and counts) and check they are the same.

func (*Account) Close

func (a *Account) Close() error

Close reduces the reference count, and closes the database connection when it was the last user.

func (*Account) Conf

func (a *Account) Conf() (config.Account, bool)

Conf returns the configuration for this account if it still exists. During an SMTP session, a configuration update may drop an account.

func (*Account) DeliverDestination added in v0.0.7

func (a *Account) DeliverDestination(log mlog.Log, dest config.Destination, m *Message, msgFile *os.File) error

DeliverDestination delivers an email to dest, based on the configured rulesets.

Returns ErrOverQuota when account would be over quota after adding message.

Caller must hold account wlock (mailbox may be created). Message delivery, possible mailbox creation, and updated mailbox counts are broadcasted.

func (*Account) DeliverMailbox

func (a *Account) DeliverMailbox(log mlog.Log, mailbox string, m *Message, msgFile *os.File) (rerr error)

DeliverMailbox delivers an email to the specified mailbox.

Returns ErrOverQuota when account would be over quota after adding message.

Caller must hold account wlock (mailbox may be created). Message delivery, possible mailbox creation, and updated mailbox counts are broadcasted.

func (*Account) HasJunkFilter added in v0.0.15

func (a *Account) HasJunkFilter() bool

func (*Account) HighestDeletedModSeq added in v0.0.6

func (a *Account) HighestDeletedModSeq(tx *bstore.Tx) (ModSeq, error)

func (*Account) MailboxCreate added in v0.0.6

func (a *Account) MailboxCreate(tx *bstore.Tx, name string, specialUse SpecialUse) (nmb Mailbox, changes []Change, created []string, exists bool, rerr error)

MailboxCreate creates a new mailbox, including any missing parent mailboxes, the total list of created mailboxes is returned in created. On success, if exists is false and rerr nil, the changes must be broadcasted by the caller.

The mailbox is created with special-use flags, with those flags taken away from other mailboxes if they have them, reflected in the returned changes.

Name must be in normalized form, see CheckMailboxName.

func (*Account) MailboxDelete added in v0.0.6

func (a *Account) MailboxDelete(ctx context.Context, log mlog.Log, tx *bstore.Tx, mb *Mailbox) (changes []Change, hasChildren bool, rerr error)

MailboxDelete marks a mailbox as deleted, including its annotations. If it has children, the return value indicates that and an error is returned.

Caller should broadcast the changes (deleting all messages in the mailbox and deleting the mailbox itself).

func (*Account) MailboxEnsure

func (a *Account) MailboxEnsure(tx *bstore.Tx, name string, subscribe bool, specialUse SpecialUse, modseq *ModSeq) (mb Mailbox, changes []Change, rerr error)

Ensure mailbox is present in database, adding records for the mailbox and its parents if they aren't present.

If subscribe is true, any mailboxes that were created will also be subscribed to.

The leaf mailbox is created with special-use flags, taking the flags away from other mailboxes, and reflecting that in the returned changes.

Modseq is used, and initialized if 0, for created mailboxes.

Name must be in normalized form, see CheckMailboxName.

Caller must hold account wlock. Caller must propagate changes if any.

func (*Account) MailboxExists added in v0.0.3

func (a *Account) MailboxExists(tx *bstore.Tx, name string) (bool, error)

MailboxExists checks if mailbox exists. Caller must hold account rlock.

func (*Account) MailboxFind added in v0.0.3

func (a *Account) MailboxFind(tx *bstore.Tx, name string) (*Mailbox, error)

MailboxFind finds a mailbox by name, returning a nil mailbox and nil error if mailbox does not exist.

func (*Account) MailboxRename added in v0.0.6

func (a *Account) MailboxRename(tx *bstore.Tx, mbsrc *Mailbox, dst string, modseq *ModSeq) (changes []Change, isInbox, alreadyExists bool, rerr error)

MailboxRename renames mailbox mbsrc to dst, including children of mbsrc, and adds missing parents for dst.

Name must be in normalized form, see CheckMailboxName, and cannot be Inbox.

func (*Account) MessageAdd added in v0.0.15

func (a *Account) MessageAdd(log mlog.Log, tx *bstore.Tx, mb *Mailbox, m *Message, msgFile *os.File, opts AddOpts) (rerr error)

MessageAdd delivers a mail message to the account.

The file is hardlinked or copied, the caller must clean up the original file. If this call succeeds, but the database transaction with the change can't be committed, the caller must clean up the delivered message file identified by m.ID.

If the message does not fit in the quota, an error with ErrOverQuota is returned and the mailbox and message are unchanged and the transaction can continue. For other errors, the caller must abort the transaction.

The message, with msg.MsgPrefix and msgFile combined, must have a header section. The caller is responsible for adding a header separator to msg.MsgPrefix if missing from an incoming message.

If UID is not set, it is assigned automatically.

If the message ModSeq is zero, it is assigned automatically. If the message CreateSeq is zero, it is set to ModSeq. The mailbox ModSeq is set to the message ModSeq.

If the message does not fit in the quota, an error with ErrOverQuota is returned and the mailbox and message are unchanged and the transaction can continue. For other errors, the caller must abort the transaction.

If the destination mailbox has the Sent special-use flag, the message is parsed for its recipients (to/cc/bcc). Their domains are added to Recipients for use in reputation classification.

Must be called with account write lock held.

Caller must save the mailbox after MessageAdd returns, and broadcast changes for new the message, updated mailbox counts and possibly new mailbox keywords.

func (*Account) MessagePath

func (a *Account) MessagePath(messageID int64) string

MessagePath returns the file system path of a message.

func (*Account) MessageReader

func (a *Account) MessageReader(m Message) *MsgReader

MessageReader opens a message for reading, transparently combining the message prefix with the original incoming message.

func (*Account) MessageRemove added in v0.0.15

func (a *Account) MessageRemove(log mlog.Log, tx *bstore.Tx, modseq ModSeq, mb *Mailbox, opts RemoveOpts, l ...Message) (chremuids ChangeRemoveUIDs, chmbc ChangeMailboxCounts, rerr error)

MessageRemove markes messages as expunged, updates mailbox counts for the messages, sets a new modseq on the messages and mailbox, untrains the junk filter and queues the messages for erasing when the last reference has gone.

Caller must save the modified mailbox to the database.

The disk usage is not immediately updated. That will happen when the message is actually removed from disk.

The junk filter is untrained for the messages if it was trained. Useful as optimization when messages are moved and the junk/nonjunk flags do not change (which can happen due to automatic junk/nonjunk flags for mailboxes).

An empty list of messages results in an error.

Caller must broadcast changes.

Must be called with wlock held.

func (*Account) NextModSeq added in v0.0.6

func (a *Account) NextModSeq(tx *bstore.Tx) (ModSeq, error)

NextModSeq returns the next modification sequence, which is global per account, over all types.

func (*Account) NextUIDValidity

func (a *Account) NextUIDValidity(tx *bstore.Tx) (uint32, error)

NextUIDValidity returns the next new/unique uidvalidity to use for this account.

func (*Account) OpenJunkFilter

func (a *Account) OpenJunkFilter(ctx context.Context, log mlog.Log) (*junk.Filter, *config.JunkFilter, error)

OpenJunkFilter returns an opened junk filter for the account. If the account does not have a junk filter enabled, ErrNotConfigured is returned. Do not forget to save the filter after modifying, and to always close the filter when done. An empty filter is initialized on first access of the filter.

func (*Account) QuotaMessageSize added in v0.0.9

func (a *Account) QuotaMessageSize() int64

QuotaMessageSize returns the effective maximum total message size for an account. Returns 0 if there is no maximum.

func (*Account) RejectsRemove

func (a *Account) RejectsRemove(log mlog.Log, rejectsMailbox, messageID string) error

RejectsRemove removes a message from the rejects mailbox if present.

Caller most hold account wlock. Changes are broadcasted.

func (*Account) Remove added in v0.0.15

func (a *Account) Remove(ctx context.Context) error

Remove schedules an account for removal. New opens will fail. When the last reference is closed, the account files are removed.

func (*Account) ReparseMessages added in v0.0.15

func (a *Account) ReparseMessages(ctx context.Context, log mlog.Log) (int, error)

ReparseMessages reparses all messages, updating the MIME structure in Message.ParsedBuf.

Typically called during automatic account upgrade, or manually.

Returns total number of messages, all of which were reparsed.

func (*Account) ResetThreading added in v0.0.7

func (a *Account) ResetThreading(ctx context.Context, log mlog.Log, batchSize int, clearIDs bool) (int, error)

ResetThreading resets the MessageID and SubjectBase fields for all messages in the account. If clearIDs is true, all Thread* fields are also cleared. Changes are made in transactions of batchSize changes. The total number of updated messages is returned.

ModSeq is not changed. Callers should bump the uid validity of the mailboxes to propagate the changes to IMAP clients.

func (*Account) RetrainMessage

func (a *Account) RetrainMessage(ctx context.Context, log mlog.Log, tx *bstore.Tx, jf *junk.Filter, m *Message) error

RetrainMessage untrains and/or trains a message, if relevant given m.TrainedJunk and m.Junk/m.Notjunk. Updates m.TrainedJunk after retraining.

func (*Account) RetrainMessages

func (a *Account) RetrainMessages(ctx context.Context, log mlog.Log, tx *bstore.Tx, msgs []Message) (rerr error)

RetrainMessages (un)trains messages, if relevant given their flags. Updates m.TrainedJunk after retraining.

func (*Account) SendLimitReached added in v0.0.6

func (a *Account) SendLimitReached(tx *bstore.Tx, recipients []smtp.Path) (msglimit, rcptlimit int, rerr error)

SendLimitReached checks whether sending a message to recipients would reach the limit of outgoing messages for the account. If so, the message should not be sent. If the returned numbers are >= 0, the limit was reached and the values are the configured limits.

To limit damage to the internet and our reputation in case of account compromise, we limit the max number of messages sent in a 24 hour window, both total number of messages and number of first-time recipients.

func (*Account) SessionsClear added in v0.0.15

func (a *Account) SessionsClear(ctx context.Context, log mlog.Log) error

SessionsClear invalidates all (web) login sessions for the account.

func (*Account) SetPassword

func (a *Account) SetPassword(log mlog.Log, password string) error

SetPassword saves a new password for this account. This password is used for IMAP, SMTP (submission) sessions and the HTTP account web page.

Callers are responsible for checking if the account has NoCustomPassword set.

func (*Account) SetSkipMessageModSeqZeroCheck added in v0.0.15

func (a *Account) SetSkipMessageModSeqZeroCheck(skip bool)

SetSkipMessageModSeqZeroCheck skips consistency checks for Message.ModSeq and Message.CreateSeq being zero.

func (*Account) Subjectpass

func (a *Account) Subjectpass(email string) (key string, err error)

Subjectpass returns the signing key for use with subjectpass for the given email address with canonical localpart.

func (*Account) SubscriptionEnsure added in v0.0.3

func (a *Account) SubscriptionEnsure(tx *bstore.Tx, name string) ([]Change, error)

SubscriptionEnsure ensures a subscription for name exists. The mailbox does not have to exist. Any parents are not automatically subscribed. Changes are returned and must be broadcasted by the caller.

func (*Account) ThreadingWait added in v0.0.7

func (a *Account) ThreadingWait(log mlog.Log) error

ThreadingWait blocks until the one-time account threading upgrade for the account has completed, and returns an error if not successful.

To be used before starting an import of messages.

func (*Account) TidyRejectsMailbox

func (a *Account) TidyRejectsMailbox(log mlog.Log, tx *bstore.Tx, mbRej *Mailbox) (changes []Change, hasSpace bool, rerr error)

TidyRejectsMailbox removes old reject emails, and returns whether there is space for a new delivery.

The changed mailbox is saved to the database.

Caller most hold account wlock. Caller must broadcast changes.

func (*Account) TrainMessage

func (a *Account) TrainMessage(ctx context.Context, log mlog.Log, jf *junk.Filter, ham bool, m Message) (bool, error)

TrainMessage trains the junk filter based on the current m.Junk/m.Notjunk flags, disregarding m.TrainedJunk and not updating that field.

func (*Account) WaitClosed added in v0.0.15

func (a *Account) WaitClosed()

WaitClosed waits until the last reference to this account is gone and the account is closed. Used during tests, to ensure the consistency checks run after expunged messages have been erased.

func (*Account) WithRLock

func (a *Account) WithRLock(fn func())

WithRLock runs fn with account read lock held. Needed for message delivery.

func (*Account) WithWLock

func (a *Account) WithWLock(fn func())

WithWLock runs fn with account writelock held. Necessary for account/mailbox modification. For message delivery, a read lock is required.

type AccountRemove added in v0.0.15

type AccountRemove struct {
	AccountName string
}

AccountRemove represents the scheduled removal of an account, when its last reference goes away.

type AddOpts added in v0.0.15

type AddOpts struct {
	SkipCheckQuota bool

	// If set, the message size is not added to the disk usage. Caller must do that,
	// e.g. for many messages at once. If used together with SkipCheckQuota, the
	// DiskUsage is not read for database when adding a message.
	SkipUpdateDiskUsage bool

	// Do not fsync the delivered message file. Useful when copying message files from
	// another mailbox. The hardlink created during delivery only needs a directory
	// fsync.
	SkipSourceFileSync bool

	// The directory in which the message file is delivered, typically with a hard
	// link, is not fsynced. Useful when delivering many files. A single or few
	// directory fsyncs are more efficient.
	SkipDirSync bool

	// Do not assign thread information to a message. Useful when importing many
	// messages and assigning threads efficiently after importing messages.
	SkipThreads bool

	// If JunkFilter is set, it is used for training. If not set, and the filter must
	// be trained for a message, the junk filter is opened, modified and saved to disk.
	JunkFilter *junk.Filter

	SkipTraining bool

	// If true, a preview will be generated if the Message doesn't already have one.
	SkipPreview bool
}

AddOpts influence which work MessageAdd does. Some callers can batch checks/operations efficiently. For convenience and safety, a zero AddOpts does all the checks and work.

type Annotation added in v0.0.15

type Annotation struct {
	ID int64

	CreateSeq ModSeq
	ModSeq    ModSeq `bstore:"index"`
	Expunged  bool

	// Can be zero, indicates global (per-account) annotation.
	MailboxID int64 `bstore:"ref Mailbox,index MailboxID+Key"`

	// "Entry name", always starts with "/private/" or "/shared/". Stored lower-case,
	// comparisons must be done case-insensitively.
	Key string `bstore:"nonzero"`

	IsString bool // If true, the value is a string instead of bytes.
	Value    []byte
}

Annotation is a per-mailbox or global (per-account) annotation for the IMAP metadata extension, currently always a private annotation.

func (Annotation) Change added in v0.0.15

func (a Annotation) Change(mailboxName string) ChangeAnnotation

Change returns a broadcastable change for the annotation.

type Archiver

type Archiver interface {
	// Add file to archive. If name ends with a slash, it is created as a directory and
	// the returned io.WriteCloser can be ignored.
	Create(name string, size int64, mtime time.Time) (io.WriteCloser, error)
	Close() error
}

Archiver can archive multiple mailboxes and their messages.

type AuthResult added in v0.0.15

type AuthResult string

AuthResult is the result of a login attempt.

const (
	AuthSuccess           AuthResult = "ok"
	AuthBadUser           AuthResult = "baduser"
	AuthBadPassword       AuthResult = "badpassword"
	AuthBadCredentials    AuthResult = "badcreds"
	AuthBadChannelBinding AuthResult = "badchanbind"
	AuthBadProtocol       AuthResult = "badprotocol"
	AuthLoginDisabled     AuthResult = "logindisabled"
	AuthError             AuthResult = "error"
	AuthAborted           AuthResult = "aborted"
)

type CRAMMD5

type CRAMMD5 struct {
	Ipad hash.Hash
	Opad hash.Hash
}

CRAMMD5 holds HMAC ipad and opad hashes that are initialized with the first block with (a derivation of) the key/password, so we don't store the password in plain text.

func (CRAMMD5) MarshalBinary

func (c CRAMMD5) MarshalBinary() ([]byte, error)

BinaryMarshal is used by bstore to store the ipad/opad hash states.

func (*CRAMMD5) UnmarshalBinary

func (c *CRAMMD5) UnmarshalBinary(buf []byte) error

BinaryUnmarshal is used by bstore to restore the ipad/opad hash states.

type CSRFToken added in v0.0.9

type CSRFToken string

type Change

type Change interface {
	ChangeModSeq() ModSeq // returns -1 for "modseq not applicable"
}

Change to mailboxes/subscriptions/messages in an account. One of the Change* types in this package.

type ChangeAddMailbox

type ChangeAddMailbox struct {
	Mailbox
	Flags []string // For flags like \Subscribed.
}

ChangeAddMailbox is sent for a newly created mailbox.

func (ChangeAddMailbox) ChangeModSeq added in v0.0.15

func (c ChangeAddMailbox) ChangeModSeq() ModSeq

type ChangeAddSubscription

type ChangeAddSubscription struct {
	MailboxName string
	ListFlags   []string // For additional IMAP flags like \NonExistent.
}

ChangeAddSubscription is sent for an added subscription to a mailbox.

func (ChangeAddSubscription) ChangeModSeq added in v0.0.15

func (c ChangeAddSubscription) ChangeModSeq() ModSeq

type ChangeAddUID

type ChangeAddUID struct {
	MailboxID int64
	UID       UID
	ModSeq    ModSeq
	Flags     Flags    // System flags.
	Keywords  []string // Other flags.

	// For IMAP NOTIFY.
	MessageCountIMAP uint32
	Unseen           uint32
}

ChangeAddUID is sent for a new message in a mailbox.

func (ChangeAddUID) ChangeModSeq added in v0.0.15

func (c ChangeAddUID) ChangeModSeq() ModSeq

type ChangeAnnotation added in v0.0.15

type ChangeAnnotation struct {
	MailboxID   int64  // Can be zero, meaning global (per-account) annotation.
	MailboxName string // Empty for global (per-account) annotation.
	Key         string // Also called "entry name", e.g. "/private/comment".
	ModSeq      ModSeq
}

ChangeAnnotation is sent when an annotation is added/updated/removed, either for a mailbox or a global per-account annotation. The value is not included.

func (ChangeAnnotation) ChangeModSeq added in v0.0.15

func (c ChangeAnnotation) ChangeModSeq() ModSeq

type ChangeFlags

type ChangeFlags struct {
	MailboxID int64
	UID       UID
	ModSeq    ModSeq
	Mask      Flags    // Which flags are actually modified.
	Flags     Flags    // New flag values. All are set, not just mask.
	Keywords  []string // Non-system/well-known flags/keywords/labels.

	// For IMAP NOTIFY.
	UIDValidity uint32
	Unseen      uint32
}

ChangeFlags is sent for an update to flags for a message, e.g. "Seen".

func (ChangeFlags) ChangeModSeq added in v0.0.15

func (c ChangeFlags) ChangeModSeq() ModSeq

type ChangeMailboxCounts added in v0.0.6

type ChangeMailboxCounts struct {
	MailboxID   int64
	MailboxName string
	MailboxCounts
}

ChangeMailboxCounts is sent when the number of total/deleted/unseen/unread messages changes.

func (ChangeMailboxCounts) ChangeModSeq added in v0.0.15

func (c ChangeMailboxCounts) ChangeModSeq() ModSeq

type ChangeMailboxKeywords added in v0.0.6

type ChangeMailboxKeywords struct {
	MailboxID   int64
	MailboxName string
	Keywords    []string
}

ChangeMailboxKeywords is sent when keywords are changed for a mailbox. For example, when a message is added with a previously unseen keyword.

func (ChangeMailboxKeywords) ChangeModSeq added in v0.0.15

func (c ChangeMailboxKeywords) ChangeModSeq() ModSeq

type ChangeMailboxSpecialUse added in v0.0.6

type ChangeMailboxSpecialUse struct {
	MailboxID   int64
	MailboxName string
	SpecialUse  SpecialUse
	ModSeq      ModSeq
}

ChangeMailboxSpecialUse is sent when a special-use flag changes.

func (ChangeMailboxSpecialUse) ChangeModSeq added in v0.0.15

func (c ChangeMailboxSpecialUse) ChangeModSeq() ModSeq

type ChangeRemoveMailbox

type ChangeRemoveMailbox struct {
	MailboxID int64
	Name      string
	ModSeq    ModSeq
}

ChangeRemoveMailbox is sent for a removed mailbox.

func (ChangeRemoveMailbox) ChangeModSeq added in v0.0.15

func (c ChangeRemoveMailbox) ChangeModSeq() ModSeq

type ChangeRemoveSubscription added in v0.0.15

type ChangeRemoveSubscription struct {
	MailboxName string
	ListFlags   []string // For additional IMAP flags like \NonExistent.
}

ChangeRemoveSubscription is sent for a removed subscription of a mailbox.

func (ChangeRemoveSubscription) ChangeModSeq added in v0.0.15

func (c ChangeRemoveSubscription) ChangeModSeq() ModSeq

type ChangeRemoveUIDs

type ChangeRemoveUIDs struct {
	MailboxID int64
	UIDs      []UID // Must be in increasing UID order, for IMAP.
	ModSeq    ModSeq
	MsgIDs    []int64 // Message.ID, for erasing, order does not necessarily correspond with UIDs!

	// For IMAP NOTIFY.
	UIDNext          UID
	MessageCountIMAP uint32
	Unseen           uint32
}

ChangeRemoveUIDs is sent for removal of one or more messages from a mailbox.

func (ChangeRemoveUIDs) ChangeModSeq added in v0.0.15

func (c ChangeRemoveUIDs) ChangeModSeq() ModSeq

type ChangeRenameMailbox

type ChangeRenameMailbox struct {
	MailboxID int64
	OldName   string
	NewName   string
	Flags     []string
	ModSeq    ModSeq
}

ChangeRenameMailbox is sent for a rename mailbox.

func (ChangeRenameMailbox) ChangeModSeq added in v0.0.15

func (c ChangeRenameMailbox) ChangeModSeq() ModSeq

type ChangeThread added in v0.0.7

type ChangeThread struct {
	MessageIDs []int64
	Muted      bool
	Collapsed  bool
}

ChangeThread is sent when muted/collapsed changes.

func (ChangeThread) ChangeModSeq added in v0.0.15

func (c ChangeThread) ChangeModSeq() ModSeq

type Comm

type Comm struct {
	Pending chan struct{} // Receives block until changes come in, e.g. for IMAP IDLE.

	sync.Mutex
	// contains filtered or unexported fields
}

Comm handles communication with the goroutine that maintains the account/mailbox/message state.

func RegisterComm

func RegisterComm(acc *Account) *Comm

Register starts a Comm for the account. Unregister must be called.

func (*Comm) Broadcast

func (c *Comm) Broadcast(ch []Change)

Broadcast ensures changes are sent to other Comms.

func (*Comm) Get

func (c *Comm) Get() (overflow bool, changes []Change)

Get retrieves all pending changes. If no changes are pending a nil or empty list is returned. If too many changes were pending, overflow is true, and this Comm stopped getting new changes. The caller should usually return an error to its connection. Even with overflow, changes may still be non-empty. On ChangeRemoveUIDs, the RemovalSeen must still be called by the caller.

func (*Comm) RemovalSeen added in v0.0.15

func (c *Comm) RemovalSeen(ch ChangeRemoveUIDs)

RemovalSeen must be called by consumers when they have applied the removal to their session. The switchboard tracks references of expunged messages, and removes/cleans the message up when the last reference is gone.

func (*Comm) Unregister

func (c *Comm) Unregister()

Unregister stops this Comm.

type DirArchiver

type DirArchiver struct {
	Dir string
}

DirArchiver is an Archiver that writes to a directory.

func (DirArchiver) Close

func (a DirArchiver) Close() error

Close on a dir does nothing.

func (DirArchiver) Create

func (a DirArchiver) Create(name string, size int64, mtime time.Time) (io.WriteCloser, error)

Create creates name in the file system, in dir. name must always use forwarded slashes.

type DiskUsage added in v0.0.9

type DiskUsage struct {
	ID          int64 // Always one record with ID 1.
	MessageSize int64 // Sum of all messages, for quota accounting.
}

DiskUsage tracks quota use.

type Flags

type Flags struct {
	Seen      bool
	Answered  bool
	Flagged   bool
	Forwarded bool
	Junk      bool
	Notjunk   bool
	Deleted   bool
	Draft     bool
	Phishing  bool
	MDNSent   bool
}

Flags for a mail message.

func ParseFlagsKeywords added in v0.0.6

func ParseFlagsKeywords(l []string) (flags Flags, keywords []string, rerr error)

ParseFlagsKeywords parses a list of textual flags into system/known flags, and other keywords. Keywords are lower-cased and sorted and check for valid syntax.

func (Flags) Changed added in v0.0.6

func (f Flags) Changed(other Flags) (mask Flags)

Changed returns a mask of flags that have been between f and other.

func (Flags) Set

func (f Flags) Set(mask, flags Flags) Flags

Set returns a copy of f, with each flag that is true in mask set to the value from flags.

func (Flags) Strings added in v0.0.11

func (f Flags) Strings() []string

Strings returns the flags that are set in their string form.

type FromAddressSettings added in v0.0.11

type FromAddressSettings struct {
	FromAddress string // Unicode.
	ViewMode    ViewMode
}

FromAddressSettings are webmail client settings per "From" address.

type LoginAttempt added in v0.0.15

type LoginAttempt struct {
	// Hash of all fields after "Count" below. We store a single entry per key,
	// updating its Last and Count fields.
	Key []byte

	// Last has an index for efficient removal of entries after 30 days.
	Last  time.Time `bstore:"nonzero,default now,index"`
	First time.Time `bstore:"nonzero,default now"`
	Count int64     // Number of login attempts for the combination of fields below.

	// Admin logins use "(admin)". If no account is known, "-" is used.
	// AccountName has an index for efficiently removing failed login attempts at the
	// end of the list when there are too many, and for efficiently removing all records
	// for an account.
	AccountName string `bstore:"index AccountName+Last"`

	LoginAddress         string // Empty for attempts to login in as admin.
	RemoteIP             string
	LocalIP              string
	TLS                  string // Empty if no TLS, otherwise contains version, algorithm, properties, etc.
	TLSPubKeyFingerprint string
	Protocol             string // "submission", "imap", "webmail", "webaccount", "webadmin"
	UserAgent            string // From HTTP header, or IMAP ID command.
	AuthMech             string // "plain", "login", "cram-md5", "scram-sha-256-plus", "(unrecognized)", etc
	Result               AuthResult
	// contains filtered or unexported fields
}

LoginAttempt is a successful or failed login attempt, stored for auditing purposes.

At most 10000 failed attempts are stored per account, to prevent unbounded growth of the database by third parties.

func LoginAttemptList added in v0.0.15

func LoginAttemptList(ctx context.Context, accountName string, limit int) ([]LoginAttempt, error)

LoginAttemptList returns LoginAttempt records for the accountName. If accountName is empty, all records are returned. Use "(admin)" for admin logins. Use "-" for login attempts for which no account was found. If limit is greater than 0, at most limit records, most recent first, are returned.

type LoginAttemptState added in v0.0.15

type LoginAttemptState struct {
	AccountName string // "-" is used when no account is present, for unknown addresses.

	// Number of LoginAttempt records for login failures. For preventing unbounded
	// growth of logs.
	RecordsFailed int
}

LoginAttemptState keeps track of the number of failed LoginAttempt records per account. For efficiently removing records beyond 10000.

type LoginSession added in v0.0.9

type LoginSession struct {
	ID                 int64
	Created            time.Time `bstore:"nonzero,default now"` // Of original login.
	Expires            time.Time `bstore:"nonzero"`             // Extended each time it is used.
	SessionTokenBinary [16]byte  `bstore:"nonzero"`             // Stored in cookie, like "webmailsession" or "webaccountsession".
	CSRFTokenBinary    [16]byte  // For API requests, in "x-mox-csrf" header.
	AccountName        string    `bstore:"nonzero"`
	LoginAddress       string    `bstore:"nonzero"`
	// contains filtered or unexported fields
}

LoginSession represents a login session. We keep a limited number of sessions for a user, removing the oldest session when a new one is created.

func SessionUse added in v0.0.9

func SessionUse(ctx context.Context, log mlog.Log, accountName string, sessionToken SessionToken, csrfToken CSRFToken) (LoginSession, error)

SessionUse checks if a session is valid. If csrfToken is the empty string, no CSRF check is done. Otherwise it must be the csrf token associated with the session token.

type Mailbox

type Mailbox struct {
	ID int64

	CreateSeq ModSeq
	ModSeq    ModSeq `bstore:"index"` // Of last change, or when deleted.
	Expunged  bool

	ParentID int64 `bstore:"ref Mailbox"` // Zero for top-level mailbox.

	// "Inbox" is the name for the special IMAP "INBOX". Slash separated for hierarchy.
	// Names must be unique for mailboxes that are not expunged.
	Name string `bstore:"nonzero"`

	// If UIDs are invalidated, e.g. when renaming a mailbox to a previously existing
	// name, UIDValidity must be changed. Used by IMAP for synchronization.
	UIDValidity uint32

	// UID likely to be assigned to next message. Used by IMAP to detect messages
	// delivered to a mailbox.
	UIDNext UID

	SpecialUse

	// Keywords as used in messages. Storing a non-system keyword for a message
	// automatically adds it to this list. Used in the IMAP FLAGS response. Only
	// "atoms" are allowed (IMAP syntax), keywords are case-insensitive, only stored in
	// lower case (for JMAP), sorted.
	Keywords []string

	HaveCounts    bool // Deprecated. Covered by Upgrade.MailboxCounts. No longer read.
	MailboxCounts      // Statistics about messages, kept up to date whenever a change happens.
}

Mailbox is collection of messages, e.g. Inbox or Sent.

func MailboxID added in v0.0.15

func MailboxID(tx *bstore.Tx, id int64) (Mailbox, error)

MailboxID gets a mailbox by ID.

Returns bstore.ErrAbsent if the mailbox does not exist. Returns ErrMailboxExpunged if the mailbox is expunged.

func (*Mailbox) CalculateCounts added in v0.0.6

func (mb *Mailbox) CalculateCounts(tx *bstore.Tx) (mc MailboxCounts, err error)

CalculateCounts calculates the full current counts for messages in the mailbox.

func (Mailbox) ChangeAddMailbox added in v0.0.15

func (mb Mailbox) ChangeAddMailbox(flags []string) ChangeAddMailbox

func (Mailbox) ChangeCounts added in v0.0.6

func (mb Mailbox) ChangeCounts() ChangeMailboxCounts

CountsChange returns a change with mailbox counts.

func (Mailbox) ChangeKeywords added in v0.0.6

func (mb Mailbox) ChangeKeywords() ChangeMailboxKeywords

ChangeKeywords returns a change with new keywords for a mailbox (e.g. after setting a new keyword on a message in the mailbox), for broadcasting to other connections.

func (Mailbox) ChangeRemoveMailbox added in v0.0.15

func (mb Mailbox) ChangeRemoveMailbox() ChangeRemoveMailbox

func (Mailbox) ChangeSpecialUse added in v0.0.6

func (mb Mailbox) ChangeSpecialUse() ChangeMailboxSpecialUse

ChangeSpecialUse returns a change for special-use flags, for broadcasting to other connections.

func (Mailbox) KeywordsChanged added in v0.0.6

func (mb Mailbox) KeywordsChanged(origmb Mailbox) bool

KeywordsChanged returns whether the keywords in a mailbox have changed.

func (*Mailbox) UIDNextAdd added in v0.0.15

func (mb *Mailbox) UIDNextAdd(n int) error

UIDNextAdd increases the UIDNext value by n, returning an error on overflow.

type MailboxCounts added in v0.0.6

type MailboxCounts struct {
	Total   int64 // Total number of messages, excluding \Deleted. For JMAP.
	Deleted int64 // Number of messages with \Deleted flag. Used for IMAP message count that includes messages with \Deleted.
	Unread  int64 // Messages without \Seen, excluding those with \Deleted, for JMAP.
	Unseen  int64 // Messages without \Seen, including those with \Deleted, for IMAP.
	Size    int64 // Number of bytes for all messages.
}

MailboxCounts tracks statistics about messages for a mailbox.

func (*MailboxCounts) Add added in v0.0.6

func (mc *MailboxCounts) Add(delta MailboxCounts)

Add increases mailbox counts mc with those of delta.

func (MailboxCounts) MessageCountIMAP added in v0.0.15

func (mc MailboxCounts) MessageCountIMAP() uint32

MessageCountIMAP returns the total message count for use in IMAP. In IMAP, message marked \Deleted are included, in JMAP they those messages are not visible at all.

func (MailboxCounts) String added in v0.0.6

func (mc MailboxCounts) String() string

func (*MailboxCounts) Sub added in v0.0.6

func (mc *MailboxCounts) Sub(delta MailboxCounts)

Add decreases mailbox counts mc with those of delta.

type MaildirReader

type MaildirReader struct {
	// contains filtered or unexported fields
}

func NewMaildirReader

func NewMaildirReader(log mlog.Log, createTemp func(log mlog.Log, pattern string) (*os.File, error), newf, curf *os.File) *MaildirReader

func (*MaildirReader) Next

func (mr *MaildirReader) Next() (*Message, *os.File, string, error)

type MboxArchiver added in v0.0.11

type MboxArchiver struct {
	Writer io.Writer
	// contains filtered or unexported fields
}

MboxArchive fakes being an archiver to which a single mbox file can be written. It returns an error when a second file is added. It returns its writer for the first file to be written, leaving parameters unused.

func (*MboxArchiver) Close added in v0.0.11

func (a *MboxArchiver) Close() error

Close on an mbox archiver does nothing.

func (*MboxArchiver) Create added in v0.0.11

func (a *MboxArchiver) Create(name string, size int64, mtime time.Time) (io.WriteCloser, error)

Create returns the underlying writer for the first call, and an error on later calls.

type MboxReader

type MboxReader struct {
	// contains filtered or unexported fields
}

MboxReader reads messages from an mbox file, implementing MsgSource.

func NewMboxReader

func NewMboxReader(log mlog.Log, createTemp func(log mlog.Log, pattern string) (*os.File, error), filename string, r io.Reader) *MboxReader

func (*MboxReader) Next

func (mr *MboxReader) Next() (*Message, *os.File, string, error)

Next returns the next message read from the mbox file. The file is a temporary file and must be removed/consumed. The third return value is the position in the file.

func (*MboxReader) Position

func (mr *MboxReader) Position() string

Position returns "<filename>:<lineno>" for the current position.

type Message

type Message struct {
	// ID of the message, determines path to on-disk message file. Set when adding to a
	// mailbox. When a message is moved to another mailbox, the mailbox ID is changed,
	// but for synchronization purposes, a new Message record is inserted (which gets a
	// new ID) with the Expunged field set and the MailboxID and UID copied.
	ID int64

	// UID, for IMAP. Set when adding to mailbox. Strictly increasing values, per
	// mailbox. The UID of a message can never change (though messages can be copied),
	// and the contents of a message/UID also never changes.
	UID UID `bstore:"nonzero"`

	MailboxID int64 `bstore:"nonzero,unique MailboxID+UID,index MailboxID+Received,index MailboxID+ModSeq,ref Mailbox"`

	// Modification sequence, for faster syncing with IMAP QRESYNC and JMAP.
	// ModSeq is the last modification. CreateSeq is the Seq the message was inserted,
	// always <= ModSeq. If Expunged is set, the message has been removed and should not
	// be returned to the user. In this case, ModSeq is the Seq where the message is
	// removed, and will never be changed again.
	// We have an index on both ModSeq (for JMAP that synchronizes per account) and
	// MailboxID+ModSeq (for IMAP that synchronizes per mailbox).
	// The index on CreateSeq helps efficiently finding created messages for JMAP.
	// The value of ModSeq is special for IMAP. Messages that existed before ModSeq was
	// added have 0 as value. But modseq 0 in IMAP is special, so we return it as 1. If
	// we get modseq 1 from a client, the IMAP server will translate it to 0. When we
	// return modseq to clients, we turn 0 into 1.
	ModSeq    ModSeq `bstore:"index"`
	CreateSeq ModSeq `bstore:"index"`
	Expunged  bool

	// If set, this message was delivered to a Rejects mailbox. When it is moved to a
	// different mailbox, its MailboxOrigID is set to the destination mailbox and this
	// flag cleared.
	IsReject bool

	// If set, this is a forwarded message (through a ruleset with IsForward). This
	// causes fields used during junk analysis to be moved to their Orig variants, and
	// masked IP fields cleared, so they aren't used in junk classifications for
	// incoming messages. This ensures the forwarded messages don't cause negative
	// reputation for the forwarding mail server, which may also be sending regular
	// messages.
	IsForward bool

	// MailboxOrigID is the mailbox the message was originally delivered to. Typically
	// Inbox or Rejects, but can also be a mailbox configured in a Ruleset, or
	// Postmaster, TLS/DMARC reporting addresses. MailboxOrigID is not changed when the
	// message is moved to another mailbox, e.g. Archive/Trash/Junk. Used for
	// per-mailbox reputation.
	//
	// MailboxDestinedID is normally 0, but when a message is delivered to the Rejects
	// mailbox, it is set to the intended mailbox according to delivery rules,
	// typically that of Inbox. When such a message is moved out of Rejects, the
	// MailboxOrigID is corrected by setting it to MailboxDestinedID. This ensures the
	// message is used for reputation calculation for future deliveries to that
	// mailbox.
	//
	// These are not bstore references to prevent having to update all messages in a
	// mailbox when the original mailbox is removed. Use of these fields requires
	// checking if the mailbox still exists.
	MailboxOrigID     int64
	MailboxDestinedID int64

	// Received indicates time of receival over SMTP, or of IMAP APPEND.
	Received time.Time `bstore:"default now,index"`

	// SaveDate is the time of copy/move/save to a mailbox, used with IMAP SAVEDATE
	// extension. Must be updated each time a message is copied/moved to another
	// mailbox. Can be nil for messages from before this functionality was introduced.
	SaveDate *time.Time `bstore:"default now"`

	// Full IP address of remote SMTP server. Empty if not delivered over SMTP. The
	// masked IPs are used to classify incoming messages. They are left empty for
	// messages matching a ruleset for forwarded messages.
	RemoteIP        string
	RemoteIPMasked1 string `bstore:"index RemoteIPMasked1+Received"` // For IPv4 /32, for IPv6 /64, for reputation.
	RemoteIPMasked2 string `bstore:"index RemoteIPMasked2+Received"` // For IPv4 /26, for IPv6 /48.
	RemoteIPMasked3 string `bstore:"index RemoteIPMasked3+Received"` // For IPv4 /21, for IPv6 /32.

	// Only set if present and not an IP address. Unicode string. Empty for forwarded
	// messages.
	EHLODomain        string         `bstore:"index EHLODomain+Received"`
	MailFrom          string         // With localpart and domain. Can be empty.
	MailFromLocalpart smtp.Localpart // SMTP "MAIL FROM", can be empty.
	// Only set if it is a domain, not an IP. Unicode string. Empty for forwarded
	// messages, but see OrigMailFromDomain.
	MailFromDomain  string         `bstore:"index MailFromDomain+Received"`
	RcptToLocalpart smtp.Localpart // SMTP "RCPT TO", can be empty.
	RcptToDomain    string         // Unicode string.

	// Parsed "From" message header, used for reputation along with domain validation.
	MsgFromLocalpart smtp.Localpart
	MsgFromDomain    string `bstore:"index MsgFromDomain+Received"`    // Unicode string.
	MsgFromOrgDomain string `bstore:"index MsgFromOrgDomain+Received"` // Unicode string.

	// Simplified statements of the Validation fields below, used for incoming messages
	// to check reputation.
	EHLOValidated     bool
	MailFromValidated bool
	MsgFromValidated  bool

	EHLOValidation     Validation // Validation can also take reverse IP lookup into account, not only SPF.
	MailFromValidation Validation // Can have SPF-specific validations like ValidationSoftfail.
	MsgFromValidation  Validation // Desirable validations: Strict, DMARC, Relaxed. Will not be just Pass.

	// Domains with verified DKIM signatures. Unicode string. For forwarded messages, a
	// DKIM domain that matched a ruleset's verified domain is left out, but included
	// in OrigDKIMDomains.
	DKIMDomains []string `bstore:"index DKIMDomains+Received"`

	// For forwarded messages,
	OrigEHLODomain  string
	OrigDKIMDomains []string

	// Canonicalized Message-Id, always lower-case and normalized quoting, without
	// <>'s. Empty if missing. Used for matching message threads, and to prevent
	// duplicate reject delivery.
	MessageID string `bstore:"index"`

	// For matching threads in case there is no References/In-Reply-To header. It is
	// lower-cased, white-space collapsed, mailing list tags and re/fwd tags removed.
	SubjectBase string `bstore:"index"`

	// Hash of message. For rejects delivery in case there is no Message-ID, only set
	// when delivered as reject.
	MessageHash []byte

	// ID of message starting this thread.
	ThreadID int64 `bstore:"index"`
	// IDs of parent messages, from closest parent to the root message. Parent messages
	// may be in a different mailbox, or may no longer exist. ThreadParentIDs must
	// never contain the message id itself (a cycle), and parent messages must
	// reference the same ancestors. Moving a message to another mailbox keeps the
	// message ID and changes the MailboxID (and UID) of the message, leaving threading
	// parent ids intact.
	ThreadParentIDs []int64
	// ThreadMissingLink is true if there is no match with a direct parent. E.g. first
	// ID in ThreadParentIDs is not the direct ancestor (an intermediate message may
	// have been deleted), or subject-based matching was done.
	ThreadMissingLink bool
	// If set, newly delivered child messages are automatically marked as read. This
	// field is copied to new child messages. Changes are propagated to the webmail
	// client.
	ThreadMuted bool
	// If set, this (sub)thread is collapsed in the webmail client, for threading mode
	// "on" (mode "unread" ignores it). This field is copied to new child message.
	// Changes are propagated to the webmail client.
	ThreadCollapsed bool

	// If received message was known to match a mailing list rule (with modified junk
	// filtering).
	IsMailingList bool

	// If this message is a DSN, generated by us or received. For DSNs, we don't look
	// at the subject when matching threads.
	DSN bool

	ReceivedTLSVersion     uint16 // 0 if unknown, 1 if plaintext/no TLS, otherwise TLS cipher suite.
	ReceivedTLSCipherSuite uint16
	ReceivedRequireTLS     bool // Whether RequireTLS was known to be used for incoming delivery.

	Flags
	// For keywords other than system flags or the basic well-known $-flags. Only in
	// "atom" syntax (IMAP), they are case-insensitive, always stored in lower-case
	// (for JMAP), sorted.
	Keywords    []string `bstore:"index"`
	Size        int64
	TrainedJunk *bool  // If nil, no training done yet. Otherwise, true is trained as junk, false trained as nonjunk.
	MsgPrefix   []byte // Typically holds received headers and/or header separator.

	// If non-nil, a preview of the message based on text and/or html parts of the
	// message. Used in the webmail and IMAP PREVIEW extension. If non-nil, it is empty
	// if no preview could be created, or the message has not textual content or
	// couldn't be parsed.
	// Previews are typically created when delivering a message, but not when importing
	// messages, for speed. Previews are generated on first request (in the webmail, or
	// through the IMAP fetch attribute "PREVIEW" (without "LAZY")), and stored with
	// the message at that time.
	// The preview is at most 256 characters (can be more bytes), with detected quoted
	// text replaced with "[...]". Previews typically end with a newline, callers may
	// want to strip whitespace.
	Preview *string

	// ParsedBuf message structure. Currently saved as JSON of message.Part because
	// bstore wasn't able to store recursive types when this was implemented. Created
	// when first needed, and saved in the database.
	// todo: once replaced with non-json storage, remove date fixup in ../message/part.go.
	ParsedBuf []byte
}

Message stored in database and per-message file on disk.

Contents are always the combined data from MsgPrefix and the on-disk file named based on ID.

Messages always have a header section, even if empty. Incoming messages without header section must get an empty header section added before inserting.

func (Message) ChangeAddUID added in v0.0.6

func (m Message) ChangeAddUID(mb Mailbox) ChangeAddUID

func (Message) ChangeFlags added in v0.0.6

func (m Message) ChangeFlags(orig Flags, mb Mailbox) ChangeFlags

func (Message) ChangeThread added in v0.0.7

func (m Message) ChangeThread() ChangeThread

func (*Message) JunkFlagsForMailbox

func (m *Message) JunkFlagsForMailbox(mb Mailbox, conf config.Account)

JunkFlagsForMailbox sets Junk and Notjunk flags based on mailbox name if configured. Often used when delivering/moving/copying messages to a mailbox. Mail clients are not very helpful with setting junk/notjunk flags. But clients can move/copy messages to other mailboxes. So we set flags when clients move a message.

func (Message) LoadPart

func (m Message) LoadPart(r io.ReaderAt) (message.Part, error)

LoadPart returns a message.Part by reading from m.ParsedBuf.

func (Message) MailboxCounts added in v0.0.6

func (m Message) MailboxCounts() (mc MailboxCounts)

MailboxCounts returns the delta to counts this message means for its mailbox.

func (Message) NeedsTraining

func (m Message) NeedsTraining() bool

NeedsTraining returns whether message needs a training update, based on TrainedJunk (current training status) and new Junk/Notjunk flags.

func (*Message) PrepareThreading added in v0.0.7

func (m *Message) PrepareThreading(log mlog.Log, part *message.Part)

PrepareThreading sets MessageID, SubjectBase and DSN (used in threading) based on the part.

type MessageErase added in v0.0.15

type MessageErase struct {
	ID int64 // Same ID as Message.ID.

	// Whether to subtract the size from the total disk usage. Useful for moving
	// messages, which involves duplicating the message temporarily, while there are
	// still references in the old mailbox, but which isn't counted as using twice the
	// disk space..
	SkipUpdateDiskUsage bool
}

MessageErase represents the need to remove a message file from disk, and clear message fields from the database, but only when the last reference to the message is gone (all IMAP sessions need to have applied the changes indicating message removal).

type ModSeq added in v0.0.6

type ModSeq int64

ModSeq represents a modseq as stored in the database. ModSeq 0 in the database is sent to the client as 1, because modseq 0 is special in IMAP. ModSeq coming from the client are of type int64.

func ModSeqFromClient added in v0.0.6

func ModSeqFromClient(modseq int64) ModSeq

ModSeqFromClient converts a modseq from a client to a modseq for internal use, e.g. in a database query. ModSeq 1 is turned into 0 (the Go zero value for ModSeq).

func (ModSeq) Client added in v0.0.6

func (ms ModSeq) Client() int64

type MsgReader

type MsgReader struct {
	// contains filtered or unexported fields
}

MsgReader provides access to a message. Reads return the "MsgPrefix" in the database (typically received headers), followed by the on-disk msg file contents. MsgReader is an io.Reader, io.ReaderAt and io.Closer.

func FileMsgReader

func FileMsgReader(prefix []byte, msgFile *os.File) *MsgReader

FileMsgReader makes a MsgReader for an open file. If initialization fails, reads will return the error. Only call close on the returned MsgReader if you want to close msgFile.

func (*MsgReader) Close

func (m *MsgReader) Close() error

Close ensures the msg file is closed. Further reads will fail.

func (*MsgReader) Read

func (m *MsgReader) Read(buf []byte) (int, error)

Read reads data from the msg, taking prefix and on-disk msg file into account. The read offset is adjusted after the read.

func (*MsgReader) ReadAt

func (m *MsgReader) ReadAt(buf []byte, off int64) (n int, err error)

ReadAt reads data from the msg, taking prefix and on-disk msg file into account. The read offset is not affected by ReadAt.

func (*MsgReader) Reset

func (m *MsgReader) Reset()

Reset rewinds the offset and clears error conditions, making it usable as a fresh reader.

func (*MsgReader) Size

func (m *MsgReader) Size() int64

Size returns the total size of the contents of the message.

type MsgSource

type MsgSource interface {
	// Return next message, or io.EOF when there are no more.
	Next() (*Message, *os.File, string, error)
}

MsgSource is implemented by readers for mailbox file formats.

type NextUIDValidity

type NextUIDValidity struct {
	ID   int // Just a single record with ID 1.
	Next uint32
}

NextUIDValidity is a singleton record in the database with the next UIDValidity to use for the next mailbox.

type Outgoing added in v0.0.3

type Outgoing struct {
	ID        int64
	Recipient string    `bstore:"nonzero,index"` // Canonical international address with utf8 domain.
	Submitted time.Time `bstore:"nonzero,default now"`
}

Outgoing is a message submitted for delivery from the queue. Used to enforce maximum outgoing messages.

type Password

type Password struct {
	Hash        string  // bcrypt hash for IMAP LOGIN, SASL PLAIN and HTTP basic authentication.
	CRAMMD5     CRAMMD5 // For SASL CRAM-MD5.
	SCRAMSHA1   SCRAM   // For SASL SCRAM-SHA-1.
	SCRAMSHA256 SCRAM   // For SASL SCRAM-SHA-256.
}

Password holds credentials in various forms, for logging in with SMTP/IMAP.

type Quoting added in v0.0.11

type Quoting string

Quoting is a setting for how to quote in replies/forwards.

const (
	Default Quoting = "" // Bottom-quote if text is selected, top-quote otherwise.
	Bottom  Quoting = "bottom"
	Top     Quoting = "top"
)

type Recipient

type Recipient struct {
	ID        int64
	MessageID int64     `bstore:"nonzero,ref Message"`            // Ref gives it its own index, useful for fast removal as well.
	Localpart string    `bstore:"nonzero"`                        // Encoded localpart.
	Domain    string    `bstore:"nonzero,index Domain+Localpart"` // Unicode string.
	OrgDomain string    `bstore:"nonzero,index"`                  // Unicode string.
	Sent      time.Time `bstore:"nonzero"`
}

Recipient represents the recipient of a message. It is tracked to allow first-time incoming replies from users this account has sent messages to. When a mailbox is added to the Sent mailbox the message is parsed and recipients are inserted as recipient. Recipients are never removed other than for removing the message. On move/copy of a message, recipients aren't modified either. For IMAP, this assumes a client simply appends messages to the Sent mailbox (as opposed to copying messages from some place).

type RecipientDomainTLS added in v0.0.8

type RecipientDomainTLS struct {
	Domain     string    // Unicode.
	Updated    time.Time `bstore:"default now"`
	STARTTLS   bool      // Supports STARTTLS.
	RequireTLS bool      // Supports RequireTLS SMTP extension.
}

RecipientDomainTLS stores TLS capabilities of a recipient domain as encountered during most recent connection (delivery attempt).

type RemoveOpts added in v0.0.15

type RemoveOpts struct {
	JunkFilter *junk.Filter // If set, this filter is used for training, instead of opening and saving the junk filter.
}

type RulesetNoListID added in v0.0.11

type RulesetNoListID struct {
	ID            int64
	RcptToAddress string `bstore:"nonzero"`
	ListID        string `bstore:"nonzero"`
	ToInbox       bool   // Otherwise from Inbox to other mailbox.
}

RulesetNoListID records a user "no" response to the question of creating/removing a ruleset after moving a message with list-id header from/to the inbox.

type RulesetNoMailbox added in v0.0.11

type RulesetNoMailbox struct {
	ID int64

	// The mailbox from/to which the move has happened.
	// Not a references, if mailbox is deleted, an entry becomes ineffective.
	MailboxID int64 `bstore:"nonzero"`
	ToMailbox bool  // Whether MailboxID is the destination of the move (instead of source).
}

RulesetNoMailbox represents a "never from/to this mailbox" response to the question of adding/removing a ruleset after moving a message.

type RulesetNoMsgFrom added in v0.0.11

type RulesetNoMsgFrom struct {
	ID             int64
	RcptToAddress  string `bstore:"nonzero"`
	MsgFromAddress string `bstore:"nonzero"` // Unicode.
	ToInbox        bool   // Otherwise from Inbox to other mailbox.
}

RulesetNoMsgFrom records a user "no" response to the question of creating/moveing a ruleset after moving a mesage with message "from" address from/to the inbox.

type SCRAM

type SCRAM struct {
	Salt           []byte
	Iterations     int
	SaltedPassword []byte
}

type SessionToken added in v0.0.9

type SessionToken string

SessionToken and CSRFToken are types to prevent mixing them up. Base64 raw url encoded.

type Settings added in v0.0.11

type Settings struct {
	ID uint8 // Singleton ID 1.

	Signature string
	Quoting   Quoting

	// Whether to show the bars underneath the address input fields indicating
	// starttls/dnssec/dane/mtasts/requiretls support by address.
	ShowAddressSecurity bool

	// Show HTML version of message by default, instead of plain text.
	ShowHTML bool

	// If true, don't show shortcuts in webmail after mouse interaction.
	NoShowShortcuts bool

	// Additional headers to display in message view. E.g. Delivered-To, User-Agent, X-Mox-Reason.
	ShowHeaders []string
}

Settings are webmail client settings.

type SpecialUse added in v0.0.6

type SpecialUse struct {
	Archive bool
	Draft   bool // "Drafts"
	Junk    bool
	Sent    bool
	Trash   bool
}

SpecialUse identifies a specific role for a mailbox, used by clients to understand where messages should go.

type Subjectpass

type Subjectpass struct {
	Email string // Our destination address (canonical, with catchall localpart stripped).
	Key   string
}

Subjectpass holds the secret key used to sign subjectpass tokens.

type Subscription

type Subscription struct {
	Name string
}

Subscriptions are separate from existence of mailboxes.

type SyncState added in v0.0.6

type SyncState struct {
	ID int // Just a single record with ID 1.

	// Last used, next assigned will be one higher. The first value we hand out is 2.
	// That's because 0 (the default value for old existing messages, from before the
	// Message.ModSeq field) is special in IMAP, so we return it as 1.
	LastModSeq ModSeq `bstore:"nonzero"`

	// Highest ModSeq of expunged record that we deleted. When a clients synchronizes
	// and requests changes based on a modseq before this one, we don't have the
	// history to provide information about deletions. We normally keep these expunged
	// records around, but we may periodically truly delete them to reclaim storage
	// space. Initially set to -1 because we don't want to match with any ModSeq in the
	// database, which can be zero values.
	HighestDeletedModSeq ModSeq
}

SyncState track ModSeqs.

type TLSPublicKey added in v0.0.14

type TLSPublicKey struct {
	// Raw-url-base64-encoded Subject Public Key Info of certificate.
	Fingerprint string
	Created     time.Time `bstore:"nonzero,default now"`
	Type        string    // E.g. "rsa-2048", "ecdsa-p256", "ed25519"

	// Descriptive name to identify the key, e.g. the device where key is used.
	Name string `bstore:"nonzero"`

	// If set, new immediate authenticated TLS connections are not moved to
	// "authenticated" state. For clients that don't understand it, and will try an
	// authenticate command anyway.
	NoIMAPPreauth bool

	CertDER      []byte `bstore:"nonzero"`
	Account      string `bstore:"nonzero"` // Key authenticates this account.
	LoginAddress string `bstore:"nonzero"` // Must belong to account.
}

TLSPublicKey is a public key for use with TLS client authentication based on the public key of the certificate.

func ParseTLSPublicKeyCert added in v0.0.14

func ParseTLSPublicKeyCert(certDER []byte) (TLSPublicKey, error)

ParseTLSPublicKeyCert parses a certificate, preparing a TLSPublicKey for insertion into the database. Caller must set fields that are not in the certificat, such as Account and LoginAddress.

func TLSPublicKeyGet added in v0.0.14

func TLSPublicKeyGet(ctx context.Context, fingerprint string) (TLSPublicKey, error)

TLSPublicKeyGet retrieves a single tls public key by fingerprint. If absent, bstore.ErrAbsent is returned.

func TLSPublicKeyList added in v0.0.14

func TLSPublicKeyList(ctx context.Context, accountOpt string) ([]TLSPublicKey, error)

TLSPublicKeyList returns tls public keys. If accountOpt is empty, keys for all accounts are returned.

type TarArchiver

type TarArchiver struct {
	*tar.Writer
}

TarArchiver is an Archiver that writes to a tar file.

func (TarArchiver) Create

func (a TarArchiver) Create(name string, size int64, mtime time.Time) (io.WriteCloser, error)

Create adds a file header to the tar file.

type UID

type UID uint32 // IMAP UID.

type Upgrade added in v0.0.7

type Upgrade struct {
	ID                  byte
	Threads             byte // 0: None, 1: Adding MessageID's completed, 2: Adding ThreadID's completed.
	MailboxModSeq       bool // Whether mailboxes have been assigned modseqs.
	MailboxParentID     bool // Setting ParentID on mailboxes.
	MailboxCounts       bool // Global flag about whether we have mailbox flags. Instead of previous per-mailbox boolean.
	MessageParseVersion int  // If different than latest, all messages will be reparsed.
}

type Validation

type Validation uint8

Validation of "message From" domain.

const (
	ValidationUnknown   Validation = 0
	ValidationStrict    Validation = 1 // Like DMARC, with strict policies.
	ValidationDMARC     Validation = 2 // Actual DMARC policy.
	ValidationRelaxed   Validation = 3 // Like DMARC, with relaxed policies.
	ValidationPass      Validation = 4 // For SPF.
	ValidationNeutral   Validation = 5 // For SPF.
	ValidationTemperror Validation = 6
	ValidationPermerror Validation = 7
	ValidationFail      Validation = 8
	ValidationSoftfail  Validation = 9  // For SPF.
	ValidationNone      Validation = 10 // E.g. No records.
)

func SPFValidation

func SPFValidation(status spf.Status) Validation

SPFValidation returns a Validation for an spf.Status.

type ViewMode added in v0.0.11

type ViewMode string

ViewMode how a message should be viewed: its text parts, html parts, or html with loading external resources.

const (
	ModeText    ViewMode = "text"
	ModeHTML    ViewMode = "html"
	ModeHTMLExt ViewMode = "htmlext" // HTML with external resources.
)

type WordSearch added in v0.0.6

type WordSearch struct {
	// contains filtered or unexported fields
}

WordSearch holds context for a search, with scratch buffers to prevent allocations for each message.

func PrepareWordSearch added in v0.0.6

func PrepareWordSearch(words, notWords []string) WordSearch

PrepareWordSearch returns a search context that can be used to match multiple messages (after each other, not concurrently).

func (WordSearch) MatchPart added in v0.0.6

func (ws WordSearch) MatchPart(log mlog.Log, p *message.Part, headerToo bool) (bool, error)

MatchPart returns whether the part/mail message p matches the search. The search terms are matched against content-transfer-decoded and charset-decoded bodies and optionally headers. HTML parts are currently treated as regular text, without parsing HTML.

type ZipArchiver

type ZipArchiver struct {
	*zip.Writer
}

ZipArchiver is an Archiver that writes to a zip file.

func (ZipArchiver) Create

func (a ZipArchiver) Create(name string, size int64, mtime time.Time) (io.WriteCloser, error)

Create adds a file header to the zip file.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL