Documentation
¶
Overview ¶
Package provider defines the mail provider interface (authenticate, list, fetch) along with its concrete implementations, e.g. Gmail and IMAP.
Index ¶
- Constants
- func Retry(ctx context.Context, cfg RetryConfig, retryable func(error) bool, ...) error
- func RetryValue[T any](ctx context.Context, cfg RetryConfig, retryable func(error) bool, ...) (T, error)
- func RetryableStatus(code int) bool
- func TrimSeenIDs(ids []string) []string
- type Fake
- type Provider
- type RawMessage
- type RetryConfig
- type SyncState
Constants ¶
const ( DefaultMaxAttempts = 5 DefaultBaseDelay = 500 * time.Millisecond DefaultMaxDelay = 30 * time.Second DefaultJitter = 0.5 )
Retry defaults. Providers that need different numbers build their own RetryConfig; zero fields fall back to these.
const MaxSeenIDs = 2000
MaxSeenIDs bounds the recently-seen message-id set carried in SyncState. The set is FIFO: once it is full, marking a new id evicts the oldest one.
Variables ¶
This section is empty.
Functions ¶
func Retry ¶
func Retry(ctx context.Context, cfg RetryConfig, retryable func(error) bool, op func(context.Context) error) error
Retry runs op until it succeeds, until an error is judged non-retryable, or until cfg.MaxAttempts is exhausted.
retryable decides whether an error is worth another attempt; a nil retryable means every error is retried. Errors that are (or wrap) ctx's error are never retried. On give-up the last error is returned wrapped, so errors.Is/As still see through to the provider's error.
func RetryValue ¶
func RetryValue[T any](ctx context.Context, cfg RetryConfig, retryable func(error) bool, op func(context.Context) (T, error)) (T, error)
RetryValue is Retry for an operation that produces a value.
func RetryableStatus ¶
RetryableStatus reports whether an HTTP status code is worth retrying: 408 (timeout), 429 (rate limited) and any 5xx.
Note for Google APIs: a rate-limited call can also come back as 403 with reason "rateLimitExceeded"/"userRateLimitExceeded". That needs the error body, which this helper does not see, so callers must add that case themselves.
func TrimSeenIDs ¶
TrimSeenIDs drops the oldest entries until at most MaxSeenIDs remain.
Types ¶
type Fake ¶
type Fake struct {
// ProviderName is returned by Name(); defaults to "fake".
ProviderName string
// Messages are replayed, in order, on every Fetch.
Messages []RawMessage
// FetchErr, when non-nil, is returned after the messages have been
// replayed — the "fetch died partway through" case, with the state
// accumulated so far still returned.
FetchErr error
// FailEarly makes Fetch stop after delivering FailAfter messages and
// return FetchErr (or a generic error), with the partial state.
FailEarly bool
// FailAfter is the number of messages delivered before FailEarly bites.
FailAfter int
// NextHistoryID, when non-zero, is written to the returned state.
NextHistoryID uint64
// NextExtra entries are merged into the returned state's Extra.
NextExtra map[string]string
// Now, when non-zero, is used as the returned LastSyncTime instead of
// time.Now(), so tests can assert on the persisted state exactly.
Now time.Time
// SkipSeen makes Fetch drop messages whose ID is already in
// state.SeenIDs, mimicking a real provider's dedup.
SkipSeen bool
// Observed by tests after the run.
Calls int
LastState SyncState
Delivered []string
}
Fake is a Provider that replays a canned slice of RawMessages. It exists so other packages (the pipeline, the run command) can be tested end to end without a network or a provider SDK.
A zero Fake is usable: it is a provider named "fake" with no messages.
func NewFake ¶
func NewFake(name string, msgs ...RawMessage) *Fake
NewFake returns a Fake named name that replays msgs.
func (*Fake) Fetch ¶
func (f *Fake) Fetch(ctx context.Context, state SyncState, fn func(RawMessage) error) (SyncState, error)
Fetch implements Provider: it replays Messages through fn, marking each delivered id seen and advancing the state. If fn returns an error the replay stops and the state accumulated so far is returned with that error.
type Provider ¶
type Provider interface {
// Name is the provider's short identifier, e.g. "gmail" or "imap".
Name() string
// Fetch streams messages newer than state via fn and returns the updated
// state. fn returning an error aborts the fetch; the state accumulated up
// to that point is returned along with the error.
//
// Implementations must respect ctx cancellation.
Fetch(ctx context.Context, state SyncState, fn func(RawMessage) error) (SyncState, error)
}
Provider is the seam that keeps the pipeline provider-agnostic: everything downstream sees RawMessage and SyncState, never a provider SDK type.
type RawMessage ¶
type RawMessage struct {
// ID is a provider-scoped stable identifier. It is used for dedup and, by
// the sinks, to derive idempotent filenames, so it must be stable across
// runs (Gmail: the message id; IMAP: account:mailbox:uidvalidity:uid).
ID string
// ThreadID is the provider's own conversation id, when it has one (Gmail:
// `threadId`, returned on the same users.messages.get response as `raw`, so
// it costs no extra call). Providers with no native threading (IMAP) leave
// it empty and model.Parse synthesizes one from the message's threading
// headers — see model.Message.ThreadID.
ThreadID string
// Raw is the complete RFC822 message, byte-faithful.
Raw []byte
// InternalDate is the time the provider received the message (not the
// Date: header, which the sender controls).
InternalDate time.Time
// Labels are provider-side labels/folders resolved to human names
// (Gmail label names; IMAP mailbox names).
Labels []string
}
RawMessage is one message as the provider delivered it: the untouched RFC822 bytes plus the handful of provider metadata fields the pipeline needs before it has parsed anything.
type RetryConfig ¶
type RetryConfig struct {
// MaxAttempts is the total number of calls, not the number of retries.
MaxAttempts int
// BaseDelay is the delay before the second attempt; it doubles thereafter.
BaseDelay time.Duration
// MaxDelay caps the pre-jitter delay.
MaxDelay time.Duration
// Jitter is the fraction of each delay that is randomised, in [0,1]. With
// the default 0.5 the actual delay is uniform in [0.5d, d]. Zero means the
// default; use a negative value to disable jitter entirely.
Jitter float64
// contains filtered or unexported fields
}
RetryConfig tunes exponential backoff with jitter. The zero value is usable and behaves like DefaultRetryConfig.
func DefaultRetryConfig ¶
func DefaultRetryConfig() RetryConfig
DefaultRetryConfig is the backoff every provider should use unless it has a reason not to: 5 attempts, 500ms base, doubling, capped at 30s, 50% jitter.
type SyncState ¶
type SyncState struct {
// HistoryID is the Gmail users.history cursor. Zero means "no incremental
// cursor", which forces a full scan.
HistoryID uint64 `json:"history_id,omitempty"`
// LastSyncTime is when the last successful fetch completed. Providers that
// have no cursor of their own use it to bound their query.
LastSyncTime time.Time `json:"last_sync_time"`
// Extra is free-form per-provider state. Keys are namespaced by the
// provider, e.g. the IMAP provider stores one pair per mailbox:
//
// imap.INBOX.uidvalidity = "1650000000"
// imap.INBOX.last_uid = "48213"
Extra map[string]string `json:"extra,omitempty"`
// SeenIDs is a bounded FIFO set of recently delivered RawMessage.IDs,
// oldest first. It is belt-and-braces dedup alongside the sinks' idempotent
// filenames: providers may skip ids already in it, and the pipeline marks
// each stored message with MarkSeen.
SeenIDs []string `json:"seen_ids,omitempty"`
}
SyncState is the per-account cursor a provider hands back after a fetch, so the next run can pick up where this one stopped. It is JSON-serializable and persisted verbatim by internal/state.
Providers should treat fields they do not own as opaque and carry them through unchanged.
func (SyncState) Clone ¶
Clone returns a deep copy, so a caller can hand a provider a state it may mutate freely without disturbing the copy on disk.
func (*SyncState) MarkSeen ¶
MarkSeen adds id to the recently-seen set, evicting the oldest entries once the set exceeds MaxSeenIDs. Marking an id that is already present is a no-op, so an id keeps its original position in the FIFO.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package gmail implements the Gmail provider: OAuth against a user-supplied desktop OAuth client, cached token storage, and (in later files) message fetching via the Gmail API.
|
Package gmail implements the Gmail provider: OAuth against a user-supplied desktop OAuth client, cached token storage, and (in later files) message fetching via the Gmail API. |
|
Package imap fetches mail from any IMAP4rev1/rev2 server behind internal/provider.Provider.
|
Package imap fetches mail from any IMAP4rev1/rev2 server behind internal/provider.Provider. |