engine

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 20, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package engine drives the archive: which families to read, in what order, how far back, and what to do with what comes off the wire.

Index

Constants

View Source 
const (
	// ProbeHonoured means the filter was applied.
	ProbeHonoured = "honoured"
	// ProbeIgnored means records came back that the filter excluded, so an
	// incremental read of this family would silently be a full one.
	ProbeIgnored = "ignored"
	// ProbeNoData means the family is empty, so nothing can be concluded.
	ProbeNoData = "no-data"
	// ProbeFailed means the probe itself errored.
	ProbeFailed = "failed"
	// ProbeNotApplicable means the family is not a paged collection, so
	// updated_since has nothing to filter.
	ProbeNotApplicable = "not applicable"
	// ProbeUnavailable means the company does not have this family.
	ProbeUnavailable = "unavailable"
)

Probe results.

View Source 
const DefaultBlobConcurrency = 8

DefaultBlobConcurrency is how many attachments download at once. Higher than the API concurrency because these go to a third-party host and spend no FreeAgent rate budget.

View Source 
const DefaultConcurrency = 4

DefaultConcurrency is how many jobs are read at once. The rate limiter is the binding constraint, so this only hides latency; more workers would just queue against the same budget.

View Source 
const DefaultOverlap = time.Hour

DefaultOverlap is how far back an incremental run reaches beyond its stored cursor. It absorbs clock skew and any lag between a record's updated_at and its visibility. Upserts are idempotent, so overlap costs requests, nothing else.

View Source 
const DefaultPayrollYears = 7

DefaultPayrollYears is how far back to look when the company record does not say when the books start. Seven years is the UK record-retention window, so it covers what anyone is obliged to keep.

View Source 
const DefaultReconcileInterval = 7 * 24 * time.Hour

DefaultReconcileInterval is how stale a family's last full sweep may get before --reconcile-if-due picks it up. FreeAgent has no deletions feed, so this cadence is the only thing that ever notices a removal.

View Source 
const DefaultReportWindow = 365 * 24 * time.Hour

DefaultReportWindow is how far back a report is taken when the run gives no window. A rolling year covers the current accounting period for most companies without asking the caller to know their own year end.

View Source 
const ProbeUpdatedSince = "updated_since"

ProbeUpdatedSince is the capability key recorded in the archive.

Variables

View Source 
var BlobUserAgent = "fasync-attachments"

BlobUserAgent identifies the downloader to the content host. Deliberately separate from the API user agent, because this client is not talking to FreeAgent and carries none of its credentials.

View Source 
var ErrBudgetExhausted = errors.New("engine: run budget exhausted")

ErrBudgetExhausted ends a run that hit its request or time limit. It is not a failure: the archive is consistent, there is simply more to do.

View Source 
var PDFFamilies = []string{"invoices", "estimates", "credit_notes"}

PDFFamilies are the records FreeAgent renders a document for.

These are the sales-side documents. Their PDF is the thing that was actually sent to a customer, so for an archive it is the counterpart of the scanned receipt on a bill: the evidence, not just the figures.

Functions

func Archivable 

func Archivable(meta freeagent.ResourceMeta) bool

Archivable reports whether this build can archive a family. The remaining classes need their own strategies and are not skipped silently: the engine reports them as unsupported so the gap is visible in the run output.

func Deferred 

func Deferred() map[string]Class

Deferred lists the families this build cannot archive yet, with the reason, so a run can report the gap rather than leaving the user to notice it.

func Probeable 

func Probeable(meta freeagent.ResourceMeta) bool

Probeable reports whether asking about updated_since means anything for a family. A singleton has nothing to filter, a report is recomputed on every request, and a year-addressed endpoint is not a collection at all. Probing them produced failures that said nothing about the API.

func SelectFamilies 

func SelectFamilies(names []string) ([]freeagent.ResourceMeta, error)

SelectFamilies resolves the caller's --family list against what this build can archive. An unknown or unsupported name is an error naming the reason, never a silent omission from the run.

Types

type BlobOptions 

type BlobOptions struct {
	// Concurrency is how many downloads run at once.
	Concurrency int
	// Limit caps how many attachments this pass takes on. Zero means all
	// outstanding.
	Limit int
	// Deadline stops the pass at a wall-clock time.
	Deadline time.Time
}

BlobOptions configures a download pass.

type BlobResult 

type BlobResult struct {
	Attempted int
	Stored    int
	Failed    int
	Skipped   int
	Bytes     int64
	Errs      []error
}

BlobResult is what a download pass achieved.

type Check 

type Check struct {
	Name    string
	Status  CheckStatus
	Summary string
	Detail  []string
}

Check is one verification result.

type CheckStatus 

type CheckStatus string

CheckStatus is how one check came out. 

const (
	// CheckPass means the check ran and found nothing wrong.
	CheckPass CheckStatus = "pass"
	// CheckFail means the check found something that should not be true.
	CheckFail CheckStatus = "fail"
	// CheckAdvisory means the check found a difference that may be legitimate.
	// Reported so a human can judge, not treated as a failure.
	CheckAdvisory CheckStatus = "advisory"
	// CheckSkipped means the check could not run: usually missing data.
	CheckSkipped CheckStatus = "skipped"
)

type Class 

type Class int

Class is how a family has to be read. It is derived from the SDK's own registry rather than from a list maintained here, so a family added upstream is classified automatically instead of being silently dropped. 

const (
	// ClassCollection is a plain paged collection: the common case.
	ClassCollection Class = iota
	// ClassGrouped splits its records across several envelope keys.
	ClassGrouped
	// ClassBankScoped rejects a request without a bank_account filter.
	ClassBankScoped
	// ClassSingleton has no id segment and returns one document.
	ClassSingleton
	// ClassReport is a derived, point-in-time answer rather than a record.
	ClassReport
	// ClassYearScoped is addressed by tax year, with no endpoint listing which
	// years exist.
	ClassYearScoped
	// ClassParentScoped rejects a request without a contact or project.
	ClassParentScoped
	// ClassUserScoped is nested under a user, so its own path does not exist.
	ClassUserScoped
	// ClassCustomEnvelope answers with a shape of its own.
	ClassCustomEnvelope
	// ClassChildOnly is reached through a parent record, never enumerated.
	ClassChildOnly
)

func Classify 

func Classify(meta freeagent.ResourceMeta) Class

Classify decides how a family is read. Order matters: the flags are not mutually exclusive, and the most restrictive one wins.

func (Class) String 

func (c Class) String() string

String names a class for logs and status output.

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine archives one account.

func New 

func New(db *store.DB, client *api.Client, report ui.Reporter, account store.Account) *Engine

New builds an engine. The client must be read-only, which is the only kind the api package can produce.

func NewOffline 

func NewOffline(db *store.DB, report ui.Reporter, account store.Account) *Engine

NewOffline builds an engine with no API client, for the work that reads only the archive. Only Verify is safe to call on one: everything else needs a client, and the absence is deliberate so a local command cannot quietly start making requests.

func (*Engine) FetchBlobs 

func (e *Engine) FetchBlobs(
	ctx context.Context, blobs *blob.Store, opts BlobOptions,
) (BlobResult, error)

FetchBlobs downloads outstanding attachments into the blob store.

Downloads go through a plain HTTP client that carries no OAuth token: the content URLs point at a third-party host, and sending FreeAgent credentials somewhere that is not FreeAgent would be a real leak. It also means these fetches cost none of the API rate budget.

func (*Engine) Probe 

func (e *Engine) Probe(ctx context.Context, names []string) ([]ProbeResult, error)

Probe establishes which families actually honour updated_since.

This exists because a family that ignores the filter returns everything, which looks exactly like success. Two cheap requests settle it: one to see whether the family has any records at all, and one asking for records changed after a date in the far future. Anything that comes back proves the filter was ignored.

func (*Engine) Pull 

func (e *Engine) Pull(ctx context.Context, opts Options) (Result, error)

Pull archives the selected families. A job that fails does not stop the others; the run reports partial and names what broke, because one flaky endpoint should not cost a night's sync of everything else.

func (*Engine) RenderDocuments 

func (e *Engine) RenderDocuments(
	ctx context.Context, blobs *blob.Store, opts RenderOptions,
) (RenderResult, error)

RenderDocuments fetches the PDF for every record whose document is missing or was rendered for an older version of it.

Unlike attachments, these come from the API and so cost rate budget: one request per document. That is why the pass is incremental rather than wholesale, keyed on the parent's modification time, and why it is not part of a routine pull by default.

func (*Engine) Verify 

func (e *Engine) Verify(ctx context.Context, opts VerifyOptions) (VerifyResult, error)

Verify checks the archive against itself and against FreeAgent's own arithmetic.

The checks are ordered by how much they prove. Internal consistency and dangling references are local and unambiguous. The reconciliation against the trial balance is advisory, because a trial balance total is cumulative from the start of an accounting period and a difference on a balance-sheet code can be an opening balance rather than a missing record.

type FamilyResult 

type FamilyResult struct {
	Family        string
	Scope         string
	Label         string
	Pages         int
	Stats         store.UpsertStats
	Deleted       int64
	Cursor        time.Time
	CursorAdvance bool
	Swept         bool
	FullScan      bool
	// Unavailable marks a family this company does not have: the API answered
	// 403 or 404. A fact about the company, not a failure.
	Unavailable bool
	Err         error
	// contains filtered or unexported fields
}

FamilyResult is what happened to one job: a family, or one scope of one.

func (FamilyResult) Name 

func (f FamilyResult) Name() string

Name is how this result is displayed.

type Options 

type Options struct {
	// Mode is one of the store.Mode constants. It decides whether the stored
	// cursor is read, advanced, or left entirely alone.
	Mode string
	// Families limits the run. Empty means every archivable family.
	Families []string
	// Window is the caller's explicit time bounds, recorded on the run.
	Window store.RunWindow
	// Overlap is how far back beyond the cursor an incremental run reaches.
	Overlap time.Duration
	// Concurrency is how many jobs are read at once.
	Concurrency int
	// MaxRequests stops the run once this many API calls have been made.
	// Zero means no limit.
	MaxRequests int64
	// Deadline stops the run at a wall-clock time. Zero means no limit.
	Deadline time.Time
	// Reconcile sweeps each family for records the far end no longer has.
	Reconcile bool
	// ReconcileIfDue sweeps only families whose last sweep is older than
	// ReconcileInterval, so one scheduled command covers both cadences.
	ReconcileIfDue bool
	// ReconcileInterval overrides DefaultReconcileInterval.
	ReconcileInterval time.Duration
}

Options configures one run.

type ProbeResult 

type ProbeResult struct {
	Family string
	Result string
	Detail string
	Err    error
}

ProbeResult is what one family answered.

type RenderOptions 

type RenderOptions struct {
	// Families limits which record types are rendered. Empty means all of
	// PDFFamilies.
	Families []string
	// Limit caps how many documents this pass renders. Zero means all
	// outstanding.
	Limit int
	// Deadline stops the pass at a wall-clock time.
	Deadline time.Time
	// MaxRequests stops the pass after this many API calls.
	MaxRequests int64
}

RenderOptions configures a rendering pass.

type RenderResult 

type RenderResult struct {
	Rendered  int
	Failed    int
	Remaining int
	Bytes     int64
	Errs      []error
}

RenderResult is what a rendering pass achieved.

type Result 

type Result struct {
	RunID    int64
	Mode     string
	Families []FamilyResult
	Stats    store.UpsertStats
	Deleted  int64
	Requests int64
	Outcome  string
	Deferred map[string]Class
}

Result is what happened to the run.

func (Result) Failed 

func (r Result) Failed() []FamilyResult

Failed lists the jobs that errored.

func (Result) Unavailable 

func (r Result) Unavailable() []FamilyResult

Unavailable lists the families this company does not have.

type UnknownFamilyError 

type UnknownFamilyError struct{ Name string }

UnknownFamilyError names a family the SDK has no entry for.

func (*UnknownFamilyError) Error 

func (e *UnknownFamilyError) Error() string

type UnsupportedFamilyError 

type UnsupportedFamilyError struct {
	Name  string
	Class Class
}

UnsupportedFamilyError names a family that exists but needs a strategy this build does not have yet.

func (*UnsupportedFamilyError) Error 

func (e *UnsupportedFamilyError) Error() string

type VerifyOptions 

type VerifyOptions struct {
	// Blobs enables re-hashing every stored attachment.
	Blobs *blob.Store
	// FromDate and ToDate bound the reconciliation against the trial balance.
	// Empty means use the newest snapshot's own window.
	FromDate string
	ToDate   string
	// MaxDetail caps how many lines a check reports, so one broken family
	// cannot bury the others.
	MaxDetail int
}

VerifyOptions configures a verification.

type VerifyResult 

type VerifyResult struct {
	Checks []Check
}

VerifyResult is the whole verification.

func (VerifyResult) Failed 

func (r VerifyResult) Failed() bool

Failed reports whether any check found something that should not be true. Advisory findings do not fail: they are differences a human has to judge.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.