imports

package module
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: MIT Imports: 47 Imported by: 0

README

Arandu

Arandu Import

An Arandu package that imports and exports records from files.

The person attaches a CSV or an XLSX, sees what the application understood of it, confirms, and watches the import run in the background. A report says how many rows went in, how many were updated and which were refused, with the row number and the reason, and the refused rows download as a CSV to correct and send again. The same screens export what is stored, as CSV or XLSX.

It is dynamic. The application registers a Definition for each kind of import — its columns, how a row is checked and where it is written — and the package draws the screens, reads the file, runs the job and keeps the history. Nothing in it knows what a plate, a customer or a product is.

  • CSV with ;, , or a tab, in UTF-8, UTF-16 or Latin-1/Windows-1252, with or without a heading row. XLSX, first sheet.
  • No cgo, no spreadsheet dependency. Reading and writing XLSX is the standard library's archive/zip and encoding/xml, so the binary of whoever installs it stays static.
  • Preview before anything is written, and a job on the queue with progress the screen follows.
  • Files on the application's Disk (in production a private bucket), under keys the server generates, read back only through an authorized route, and deleted by a daily clean after RetentionDays.
  • Tenant from the Grant, activity log entries at the start and the end of every run with counts and never the content of a row.

Install

go get github.com/hyz-is/arandu-import

Wire it

An Arandu application registers a module explicitly. There is no service provider, no container and no discovery, so these are the lines to paste into bootstrap/app.go, and the views add one import below.

The import, with the other module imports:

import (
	imports "github.com/hyz-is/arandu-import"
)

The construction, in App, once NewAppServiceProvider has returned and the queue and the disk exist. New refuses a wiring that cannot work and Configure has no way to return that error, so the module is built where the error can leave; the database and the session store are the ones the framework built from the configuration:

	importsModule, err := imports.New(imports.Config{
		Definitions: []imports.Definition{platesDefinition},
		Queue:       queue,
		Files:       disk,
		Activity:    activity.Logger(),
	}, drivers.DB, drivers.Sessions)
	if err != nil {
		_ = drivers.Close()
		return nil, err
	}

And the registration, in the WithProviders(...) chain the builder is made of:

	return Configure(cfg, drivers, app).
		WithProviders(importsModule).
		WithVersion(version, commit).
		Create()

The module hands the worker its two jobs (RegisterJobHandlers) and the scheduler its daily clean (Schedule); the framework reads both from a registered module. Then, once, before the application serves:

aru migrate

This package owns a table, which is why the migration step is not optional and why arandu.mod.toml says migrations = true.

A Definition

imports.Definition{
	Key: "plates", Label: "Allowed plates",
	Description: "The plates you allow.",
	Columns: []imports.Column{
		{Key: "plate", Label: "Plate", Required: true, Headers: []string{"Plates"}, Example: "ABC1D23"},
		{Key: "note", Label: "Note"},
	},
	MaxRows:      5000,
	ImportAction: "whitelist.manage",
	ExportAction: "whitelist.list",
	Validate: func(ctx context.Context, g security.Grant, row imports.Row) (imports.Row, error) {
		// Normalize and check. imports.Refuse("reason") turns the row away.
		return row, nil
	},
	Write: func(ctx context.Context, g security.Grant, row imports.Row) (imports.Outcome, error) {
		// Store it, with the Grant of whoever confirmed. An upsert: the queue
		// delivers a job at least once.
		return imports.Created, nil
	},
	Finish: func(ctx context.Context, g security.Grant, s imports.Summary) error { return nil },
	Export: &imports.Export{
		Each: func(ctx context.Context, g security.Grant, emit func(imports.Row) error) error {
			return nil // call emit once per record, read with g
		},
	},
}

A file is read by its headings, ignoring case, accents, spaces and marks, so Placa, PLACAS and placa are one column; a column named in Headers answers to those too. A file with no heading is read in the order of Columns. A file that lacks a required column, holds more than MaxRows rows or is not a table is refused whole, with the reason.

Validate writes nothing: it is called on every row to draw the preview and again by the job. Write is the application's own storage. An error made with imports.Refuse refuses one row and the import goes on; any other error stops the import and the run says it failed. Finish runs once at the end, for what depends on the whole file.

The policy asks about ImportAction and ExportAction, which are the application's own actions, given to subjects by its permission module. A subject that holds neither sees nothing.

Publish the views

This package carries the markup of its own pages and hands it over instead of rendering it from the inside, because a page you cannot edit is a page that says the wrong thing in your product.

aru vendor:publish --tag=view
aru vendor:publish --tag=view --apply

Nothing is written without --apply. The files land under resources/views/modules/imports/, and from that point they are yours: change the layout they extend, put the application's page head on them, translate them. Two steps are left to you. Compile what was written:

aru view:build

and import the directory it wrote into, with the other imports:

	_ "your/module/path/storage/framework/views/modules/imports"

Without that import the module refuses to boot, naming the view, the command and the import. No script and no style is inline: the drop area is Kyse's file-upload behaviour and the progress follows with YPY, both served from 'self'. The file is sent by the button under the area, after its name is shown.

The name of the chosen file

The drop area is Kyse's file-upload: it takes a dropped or a picked file into the real file input and says nothing more. The upload form names a behaviour of its own, import-file, that writes the name of the chosen file in the area. The behaviour is the application's, served from 'self' with the rest of its scripts, and a page without it is the page that always was (the area keeps its sentence and the button submits):

arandu.ui.define('import-file', {
	mounted: function (ctx) {
		var input = ctx.element.querySelector('input[type="file"]');
		var prompt = ctx.element.querySelector('[data-part="prompt"]');
		if (!input || !prompt) return;
		var said = prompt.textContent;
		ctx.show = function () {
			var file = input.files && input.files[0];
			prompt.textContent = file ? file.name : said;
		};
		input.addEventListener('change', ctx.show);
	},
	destroyed: function (ctx) {
		var input = ctx.element.querySelector('input[type="file"]');
		if (input && ctx.show) input.removeEventListener('change', ctx.show);
	}
});

Configuration

field required meaning
Definitions yes the kinds of import and export. At least one
Queue yes where the jobs are pushed
Files yes the Disk for the uploads, the reports and the exports
Prefix no where the routes are mounted. Defaults to /imports
Paths no the segments under the prefix. PortuguesePaths is shipped
Scope no turns the session's subject into the subject acting in the space the person selected, for an application where one person belongs to several. Nil acts as the session's subject
Middleware no wraps every route, after RequireAuth
Activity no the activity log: the start and the end of every run, with counts
OnChange no told after every progress write, to tell a live table
MaxFileBytes no the largest upload. Defaults to 10 MB, refused above 50 MB
RetentionDays no how long the files are kept. Defaults to 7, refused above 90
CleanSpec no the cron of the clean. Defaults to 20 3 * * *, - schedules none
Chrome, Locale, Translator, TitleSeparator, Location no the application's frame, language, words and zone

New returns an error rather than starting half-wired, so a setting that cannot work fails where it is written instead of on the first request that needed it.

Routes

method path name
GET /imports imports.index
GET /imports/{definition} imports.new
POST /imports/{definition} imports.upload
GET /imports/{definition}/template/{format} imports.template
POST /imports/{definition}/export imports.export
GET /imports/{definition}/runs/{id} imports.show
GET /imports/{definition}/runs/{id}/progress imports.progress
POST /imports/{definition}/runs/{id}/confirm imports.confirm
POST /imports/{definition}/runs/{id}/discard imports.discard
GET /imports/{definition}/runs/{id}/refused imports.refused
GET /imports/{definition}/runs/{id}/file imports.file

Every one of them sits behind RequireAuth: a visitor with no session is sent to /auth/login, and the handler of a request that has one reads who is asking with ctx.User(). A handler returns the error the service returned, and the router answers it: a policy refusal is 403, a run that is not there is 404, and a rejected file goes back where it came from with the message in the flash. Module.URLs() builds every address.

The data path

Run embeds model.Model, and runTable, beside it in model.go, declares its table once. aru model:build generates RunQuery.go from it: Runs(db), which starts a *RunQuery, and RunCollection. The file is committed and never edited by hand; aru model:build --check fails when it is out of date.

Service owns *data.DB and follows validate -> security.Authorize -> Grant -> Model terminal; handlers never hold the database or start a query. The job runs under the Grant that confirmed the run: the queue carries its tenant and its action, so Write and Export.Each receive a Grant of the same tenant and nothing they read comes from anywhere else.

A run keeps counts and never the content of a row. The first refused rows are on the run for the screen, and all of them are in a CSV on the Disk until the clean.

Layout

module.go         registration and routes
config.go         what the application passes in, and the route table
definition.go     what the application states for each kind of import
model.go          the run, its table, its own rules, and what it may answer with
RunQuery.go       the query aru model:build generates from model.go
policy.go         who may do what
service.go        the use cases, and Model access after authorization
job.go            the two jobs the queue runs
clean.go          the daily clean of the files
grid.go, xlsxread.go, writer.go, mapping.go   reading and writing the files
handlers.go, pages.go, screens.go             the routes and what the views draw
views.go          the files the application takes ownership of

What is already correct, and has to stay that way

The policy denies everything it was not told about. An action no Definition names is refused, and so is a subject that does not hold the one it names. The Service calls security.Authorize before its first Runs(db) reach, and every Model terminal requires the Grant that call produced. Looking at a run is authorized under a read-only form of the action, so a bug on that path cannot start an import.

The tenant comes from data.Tenant(g). Never from the path, the body, the query string or a header.

Who is asking comes from the guard. RequireAuth loads the session once and puts the subject on the request; nothing in the module loads it again.

arandu.mod.toml declares what the package does — network, filesystem, exec, migrations — and the suite compares the declaration against what the code calls. The package writes no file of its own: everything goes through the Disk the application passes in.

Tests

go test -race ./...

The denial suite constructs the Service with a nil database, so even building Runs(nil) would panic. The feature suite drives the whole flow over a real SQLite database: every delimiter and encoding, a real XLSX, the header in Portuguese, a refused row, the job and its progress, the isolation between tenants, the clean and the export.

Licence

MIT. See LICENSE.md. Copyright HYZIS.

Documentation

Overview

Package imports is an Arandu module for importing and exporting records from files: the person attaches a spreadsheet or a CSV, looks at what the application understood, confirms, watches the import run in the background and reads a report of what went in and what was refused.

It is dynamic. The application registers a Definition for each kind of import -- its columns, how a row is checked and where it is written -- and the package draws the screens, reads the file, runs the job and keeps the history. Nothing in it knows what a plate, a customer or a product is.

The files are laid out by role rather than by layer, so the whole package reads top to bottom:

module.go      -> registration and routes
config.go      -> what the application passes in
definition.go  -> what the application states for each kind of import
model.go       -> the run, and what it may answer with
policy.go      -> who may do what
service.go     -> the use cases, and Model access after authorization
job.go         -> the work the queue runs
handlers.go    -> the routes' handlers
views.go       -> the files the application takes ownership of

An application registers it explicitly. There is no service provider, no container and no discovery: the wiring is a few lines somebody wrote.

Index

Constants

View Source
const (
	ActivityLogName = "imports"
	ActivityRefType = "import_run"
)

ActivityLogName is the log every entry of the package is written to, and ActivityRefType the type of the thing the entries are about.

View Source
const (
	ActivityImportStarted  = "imports.import_started"
	ActivityImportFinished = "imports.import_finished"
	ActivityImportFailed   = "imports.import_failed"
	ActivityExportStarted  = "imports.export_started"
	ActivityExportFinished = "imports.export_finished"
	ActivityExportFailed   = "imports.export_failed"
	ActivityFileRead       = "imports.file_read"
	ActivityFilesCleaned   = "imports.files_cleaned"
)

The events the package writes. Each carries counts and identifiers and never the content of a row: the rows are the application's records, and the log that says an import happened is not a second place to keep them.

View Source
const (
	// DefaultPrefix is where the routes are mounted when Config leaves Prefix
	// empty.
	DefaultPrefix = "/imports"
	// DefaultQueue is the queue the jobs travel on when Config leaves QueueName
	// empty.
	DefaultQueue = "default"
	// DefaultPageSize is how many runs one page of the history answers with.
	DefaultPageSize = 25
	// MaxPageSize is the ceiling PageSize is refused above.
	MaxPageSize = 100
	// DefaultMaxFileBytes is the largest file an import takes when Config
	// leaves MaxFileBytes at zero.
	DefaultMaxFileBytes = 10 << 20
	// MaxFileBytesLimit is the ceiling MaxFileBytes is refused above: the whole
	// file is read into memory to be checked, so it has to stay small.
	MaxFileBytesLimit = 50 << 20
	// DefaultMaxRows is how many rows one import takes when its Definition
	// leaves MaxRows at zero.
	DefaultMaxRows = 5000
	// MaxRowsLimit is the ceiling a Definition's MaxRows is refused above.
	MaxRowsLimit = 100000
	// DefaultExportRows is how many rows one export holds when its Export
	// leaves MaxRows at zero.
	DefaultExportRows = 50000
	// DefaultRetentionDays is how long the files of a run are kept when Config
	// leaves RetentionDays at zero.
	DefaultRetentionDays = 7
	// MaxRetentionDays is the ceiling RetentionDays is refused above.
	MaxRetentionDays = 90
	// DefaultCleanSpec is when the scheduled clean runs: every day at 03:20.
	DefaultCleanSpec = "20 3 * * *"
	// DefaultProgressEvery is how many rows a job reads between two writes of
	// its progress.
	DefaultProgressEvery = 50
	// DefaultLocale is the language of the screens when Config names none, and
	// FallbackLocale the one every line exists in.
	DefaultLocale  = "pt-BR"
	FallbackLocale = "en"
	// DefaultLocation is the time zone dates are drawn in when Config names
	// none.
	DefaultLocation = "America/Sao_Paulo"
)

The defaults and the ceilings of the optional settings. They are constants rather than literals inside Config.withDefaults, so the value a reader finds here is the value the package uses.

View Source
const (
	// ImportJob reads the file of a confirmed upload and writes its rows.
	ImportJob = "imports.import"
	// ExportJob writes the file of an export.
	ExportJob = "imports.export"
)

The jobs the worker runs. Their names are what the queue routes by, so they are fixed once the package is published.

View Source
const (
	// KindImport reads a file and writes its rows.
	KindImport = "import"
	// KindExport reads the application's records and writes a file.
	KindExport = "export"
)

The two kinds of run.

View Source
const (
	FormatCSV  = "csv"
	FormatXLSX = "xlsx"
)

The formats of a file.

View Source
const (
	ViewIndex = "modules.imports.index"
	ViewShow  = "modules.imports.show"
	ViewState = "modules.imports.state"
)

The names the package renders its views by. They come from the paths of the files in resources/views, by the rule views.go states.

View Source
const (
	// CleanAction is the action the clean runs under, in the tenant of the runs
	// it reaches. No subject holds it: it is issued to the scheduler and to
	// nobody else.
	CleanAction security.Action = "imports.clean"
)
View Source
const PublishCommand = "aru vendor:publish --apply"

PublishCommand is what an application runs to take ownership of the views this package offers. It reads the modules the application registered and writes what each one declares.

View Source
const TranslationGroup = "imports"

TranslationGroup is the file every key of this package lives in, and the prefix of every key: "imports.screen.index_title".

Variables

View Source
var DefaultPaths = Paths{
	Runs: "/runs", Template: "/template", Export: "/export", Progress: "/progress", Confirm: "/confirm",
	Discard: "/discard", Refused: "/refused", File: "/file",
}

DefaultPaths are the segments in English.

View Source
var ErrNotFound = fmt.Errorf("imports: run not found: %w", model.ErrModelNotFound)

ErrNotFound is returned when no row matches, including when the row exists in another tenant. The two cases are deliberately indistinguishable.

It wraps model.ErrModelNotFound, so the router answers it with 404.

View Source
var PortuguesePaths = Paths{
	Runs: "/execucoes", Template: "/modelo", Export: "/exportar", Progress: "/progresso", Confirm: "/confirmar",
	Discard: "/descartar", Refused: "/recusadas", File: "/arquivo",
}

PortuguesePaths are the segments in Brazilian Portuguese.

Statuses lists them in the order a run goes through them.

Functions

func AcceptedExtensions

func AcceptedExtensions() []string

AcceptedExtensions are the extensions the screens offer to the file picker.

func Itoa

func Itoa(n int) string

Itoa writes a number for a view, which cannot import strconv under a name of its own.

func Lines

func Lines(locale string) translation.Lines

Lines are the sentences this package ships for one locale, keyed the way a translator asks for them ("imports.validation.required"), and nil for a locale it does not ship. An application reads them to see what there is to override, or to check a catalogue of its own against them.

func Locales

func Locales() []string

Locales are the languages the package ships.

func PublishedPaths

func PublishedPaths() []string

PublishedPaths are the files in the archive, each relative to the root of the application, sorted.

func Refuse

func Refuse(reason string) error

Refuse refuses the row it is returned for.

func StatusVariant

func StatusVariant(s Status) string

StatusVariant is the badge variant of a status.

func ViewNames

func ViewNames() []string

ViewNames are the names the published views are rendered by, sorted.

The name is the path under the view root with its separators turned into dots, which is what the view compiler writes into the registration call. It is derived from the same archive the publication carries, so a view that was renamed cannot keep an old name here.

func ViewOf

func ViewOf(a security.Action) security.Action

ViewOf is the read form of a Definition's action: the action the package authorizes to look at a run or a report. It is exported so an application can ask its own policy tests what the package asks.

func ViewPackages

func ViewPackages() []string

ViewPackages are the directories the compiled views land in, each relative to the root of the application, sorted and without repeats.

Importing one is what puts its views in the binary: a compiled view calls the registry from init(), and a package nothing imports is not linked at all. The import is named rather than written into bootstrap/app.go, because one line somebody reads beats a file that changed while they were not looking.

Types

type Change

type Change struct {
	// Tenant is the space the run belongs to.
	Tenant string
	// Run is the run's identifier and Definition the key of its definition.
	Run, Definition string
	// Kind is KindImport or KindExport, Status the status it is in now.
	Kind, Status string
	// Processed and Total say how far along it is.
	Processed, Total int64
}

Change is what a run just did, as OnChange is told.

type ChromeRequest

type ChromeRequest struct {
	// Title is the screen's title.
	Title string
	// Subject is who is looking at it.
	Subject security.Subject
}

ChromeRequest is what the application is asked to draw a frame for.

type Column

type Column struct {
	// Key is the name the application reads the value by: lowercase letters,
	// digits and underscores.
	Key string
	// Label is the heading the template, the report and the export write.
	Label string
	// Headers are the other headings a file may use for the column. The Key and
	// the Label are always accepted, whatever their accents, case and spaces.
	Headers []string
	// Required says a file without the column is refused as a whole. An empty
	// cell of a required column is for Validate to refuse, since only the
	// application knows what an empty value means.
	Required bool
	// Hint is a line of help shown beside the column on the import screen.
	Hint string
	// Example is the value the template shows when the definition has no
	// Examples of its own.
	Example string
}

Column is one field of a definition: what the file heads it with, what the server calls it, and whether a row without it is refused.

type ColumnView

type ColumnView struct {
	Label, Hint string
	Required    bool
	// Mark says "required" or "optional", in the screen's language.
	Mark string
}

ColumnView is a column as the help of the import screen lists it.

type Config

type Config struct {
	// Definitions are the kinds of import and export the application offers,
	// each with its columns, how a row is checked and where it is written.
	// At least one is required.
	Definitions []Definition

	// Queue is where the jobs go. Required: an import runs in the background.
	Queue queue.Queue

	// QueueName is the queue the jobs are pushed on. Empty means DefaultQueue.
	QueueName string

	// Files is the Disk the uploaded files, the reports and the exports are
	// kept on, in production a private bucket. Required. The keys are generated
	// by the server and a file is read back only through an authorized route.
	Files *filesystem.Disk

	// Prefix is the path the routes are mounted under. Empty means
	// DefaultPrefix.
	Prefix string

	// Paths are the segments of the routes under the prefix. A zero field keeps
	// its default.
	Paths Paths

	// Middleware wraps every route, after RequireAuth.
	Middleware []func(http.Handler) http.Handler

	// Activity is where the start and the end of every import and export are
	// written, with the counts and never the content of a row. Nil writes
	// nothing; an application with several tenants passes it.
	Activity *activitylog.Logger

	// OnChange is told after every write of a run's progress and every change
	// of its status, so the application can tell the sockets that watch a table.
	// It is called from the web process and from the worker, and what it does
	// fails silently: a run does not depend on it.
	OnChange func(ctx context.Context, change Change)

	// MaxFileBytes is the largest upload, in bytes. Zero means
	// DefaultMaxFileBytes; anything above MaxFileBytesLimit is refused.
	MaxFileBytes int64

	// RetentionDays is how long the files of a run are kept before the clean
	// deletes them. Zero means DefaultRetentionDays.
	RetentionDays int

	// CleanSpec is the five-field cron expression the clean runs on. Empty
	// means DefaultCleanSpec; "-" schedules none.
	CleanSpec string

	// PageSize is how many runs one page of the history holds. Zero means
	// DefaultPageSize.
	PageSize int

	// ProgressEvery is how many rows a job reads between two writes of its
	// progress. Zero means DefaultProgressEvery.
	ProgressEvery int

	// Translator holds the application's own lines, which win over the shipped
	// ones; nil uses the shipped catalogue alone.
	Translator *translation.Translator

	// Locale is the language of the screens. Empty means DefaultLocale.
	Locale string

	// TitleSeparator ends a page title with the application's name, as in
	// "Importar placas — Alpr.sh". Empty writes the title alone.
	TitleSeparator string

	// Scope turns the subject of the session into the subject acting in the
	// space the person works in. An application where one person belongs to
	// several spaces (an organization they selected) answers the subject scoped
	// to the selected one, with that space's actions: the tenant of every
	// statement is the one the Grant of that subject carries, and never one a
	// request named. Nil acts as the session's subject, which is right for an
	// application whose session already is the space.
	Scope func(ctx *fhttp.Context, who security.Subject) (security.Subject, error)

	// Chrome draws the application's frame around a screen: it returns what the
	// application's layout reads from the screen. Nil leaves the layout to the
	// page alone.
	Chrome func(ctx *fhttp.Context, req ChromeRequest) (any, error)

	// Location is the time zone dates are drawn in. Nil means DefaultLocation.
	Location *time.Location

	// Now is the clock; nil means time.Now.
	Now func() time.Time

	// Logger receives what the package reports. Nil means slog.Default.
	Logger *slog.Logger
}

Config is what the application passes when it wires this package.

A typed struct rather than a map: a misspelled key in a map is a setting that silently keeps its default. Here a field that does not exist does not compile.

It names no tenant. Every route sits behind RequireAuth, so the tenant of every statement is the one the Grant carries, and the job that runs later takes it from the row the Grant wrote.

func (Config) Validate

func (c Config) Validate() error

Validate reports what the configuration cannot be used with. New calls it, so an application with a setting that cannot work fails where it is wired.

type Definition

type Definition struct {
	// Key names the definition in the address: lowercase letters, digits and
	// hyphens. It is stored on every run, so changing it orphans the history.
	Key string
	// Label is its name on the screens: "Lista de permitidos".
	Label string
	// Description is the lead of the import screen.
	Description string
	// ListURL is where the application lists the records themselves. The page of
	// the definition links back to it; empty links back to the overview.
	ListURL string
	// Columns are the fields of a row, in the order the template writes them.
	Columns []Column
	// MaxRows is how many rows one file may hold. Zero means DefaultMaxRows,
	// and a file with more is refused whole rather than cut short.
	MaxRows int

	// ImportAction is the action a subject needs to import, and ExportAction
	// the one it needs to export and to see an export. Both are the
	// application's own actions, given to subjects by its permission module.
	ImportAction security.Action
	ExportAction security.Action

	// Validate checks one row and returns it normalized. It is called on the
	// whole file to draw the preview, and again by the job: it writes nothing.
	// An error made with Refuse refuses the row; the Grant is the one of whoever
	// is importing.
	Validate func(ctx context.Context, g security.Grant, row Row) (Row, error)

	// Write stores one validated row with the Grant of whoever confirmed the
	// import, and says what it did. The queue delivers a job at least once, so
	// Write has to be safe to run twice on the same row: an upsert.
	Write func(ctx context.Context, g security.Grant, row Row) (Outcome, error)

	// Finish runs once when an import ends, with the counts. The application
	// uses it for what depends on the whole file, such as telling a live table
	// that it changed. An error is logged and does not undo the import.
	Finish func(ctx context.Context, g security.Grant, s Summary) error

	// Examples are the rows the template shows. Empty uses each column's
	// Example.
	Examples []Row

	// Export, when set, offers the definition's records as a file.
	Export *Export
}

Definition is one kind of import, and optionally of export: the application states it once and the package draws the screens, reads the file, checks and writes the rows in the background and reports what happened.

func (Definition) CanExport

func (d Definition) CanExport() bool

CanExport says the definition writes files.

func (Definition) CanImport

func (d Definition) CanImport() bool

CanImport says the definition accepts files.

type DefinitionView

type DefinitionView struct {
	Key, Label, Description string
	// URL is the page of the definition, and ListURL where the application
	// lists its records.
	URL, ListURL string
	// CanImport and CanExport say what the subject may do with it.
	CanImport, CanExport bool
}

DefinitionView is a definition as a screen shows it.

type Download

type Download struct {
	Name, ContentType string
	Size              int64
	Body              io.ReadCloser
}

Download is a file of a run, ready to be served: the name it downloads as, the content type the package wrote for it and its bytes. The caller closes Body.

type Export

type Export struct {
	// Columns are the columns of the file. Empty means the definition's own.
	Columns []Column
	// Each calls emit once for every record, in the order the file lists them.
	// The Grant carries the tenant: the query reads nothing from anywhere else.
	// Each stops at the first error emit returns, and returns it.
	Each func(ctx context.Context, g security.Grant, emit func(Row) error) error
	// MaxRows is how many rows one export holds. Zero means DefaultExportRows,
	// and a file that would hold more is refused rather than cut short.
	MaxRows int
}

Export is how a definition writes its records out: the columns of the file and the application's own query, called with the Grant of whoever asked.

type FormState

type FormState struct{ hview.Page }

FormState is the part of a page a form component asks about.

func (FormState) ErrorSummary

func (f FormState) ErrorSummary() []string

ErrorSummary is every message of the form.

func (FormState) FieldError

func (f FormState) FieldError(name string) string

FieldError is the first message of a field.

func (FormState) FieldErrors

func (f FormState) FieldErrors(name string) []string

FieldErrors are the messages of one field.

func (FormState) HasErrors

func (f FormState) HasErrors() bool

HasErrors says the form was rejected.

func (FormState) LogValue

func (f FormState) LogValue() slog.Value

func (FormState) MarshalJSON

func (f FormState) MarshalJSON() ([]byte, error)

MarshalJSON and LogValue keep the form as quiet as the page.

type IndexPageData

type IndexPageData struct {
	Screen
	// Definition is the definition the page is about; nil on the overview.
	Definition *DefinitionView
	// Definitions are the ones the subject may use, on the overview.
	Definitions []DefinitionView

	// The upload: where it posts, what the picker offers, and the limits.
	UploadURL    string
	Accept       string
	MaxMB        string
	MaxRows      int
	Columns      []ColumnView
	TemplateCSV  string
	TemplateXLSX string
	ExportURL    string
	CanImport    bool
	CanExport    bool
	Retention    int

	// The history.
	Rows          []RunRow
	Query         string
	Statuses      []string
	SortKey       string
	SortDir       string
	URL           string
	Pager         components.Paginator
	CurrentPage   int
	Pages, Total  int
	PerPage       int
	StatusOptions []components.SelectOption
}

IndexPageData is what the index page of the package, and of one definition, is drawn from.

func (IndexPageData) BackLabel

func (d IndexPageData) BackLabel() string

BackLabel names where BackURL goes: the application's list when it said where that is, the overview otherwise.

func (IndexPageData) BackURL

func (d IndexPageData) BackURL() string

BackURL is where the page of a definition goes back to: the application's own list of the records when it said where that is, the overview otherwise.

func (IndexPageData) ExportLead

func (d IndexPageData) ExportLead() string

ExportLead says what an export does and how long its file is kept.

func (IndexPageData) Overview

func (d IndexPageData) Overview() bool

Overview says the page is the overview and not the page of one definition.

func (IndexPageData) TableState

func (d IndexPageData) TableState() components.DataTableState

TableState is where the history's table is, for its controls.

func (IndexPageData) UploadHint

func (d IndexPageData) UploadHint() string

UploadHint is the line under the file box: what it takes and how much.

type Labels

type Labels struct {
	Locale string
	// contains filtered or unexported fields
}

Labels are the sentences of this package in one locale.

func (Labels) F

func (l Labels) F(key string, pairs ...any) string

F is a line of the catalogue with its values given as pairs, for a view that has the numbers and the names at hand: F("screen.upload_hint", "mb", "10").

func (Labels) FailureText

func (l Labels) FailureText(code string) string

FailureText is the sentence a run's failure code is told in. A code with no line, such as one a later version wrote, reads as the generic failure.

func (Labels) Field

func (l Labels) Field(name string) string

Field is what a person reads in place of an input's name, in a summary of the rejected fields: "subject" reads "Assunto". A field with no line reads the way the input was named.

func (Labels) Kind

func (l Labels) Kind(kind string) string

Kind is the label of what a run does: importing or exporting.

func (Labels) Status

func (l Labels) Status(s Status) string

Status is the label of a run's status.

func (Labels) Summary

func (l Labels) Summary(errs validation.Errors) []string

Summary is one line per rejected message, the field named before it, for the banner above a form. Nil when nothing was rejected, so a layout that ranges over it draws no banner.

func (Labels) T

func (l Labels) T(key string) string

T reads a key of this package: "screen.index_title".

type Line

type Line struct {
	// Number is the row's place in the file, as the person's spreadsheet
	// numbers it.
	Number int `json:"number"`
	// Values are the cells by column key; Reason is why the row was refused, and
	// empty for a row that was fine.
	Values Row    `json:"values,omitempty"`
	Reason string `json:"reason,omitempty"`
}

Line is one row of a file as the preview and the report tell it.

type LineRow

type LineRow struct {
	Number int
	Cells  []string
	Reason string
}

LineRow is one row of the preview or of the refused ones.

type ListQuery

type ListQuery struct {
	// Definition limits the history to one definition; empty lists them all.
	Definition string
	// Search is part of a file's name.
	Search   string
	Statuses []Status
	// Sort is a key of sortable, Dir "asc" or "desc".
	Sort, Dir string
	Page      int
	PerPage   int
}

ListQuery is what a page of the history is asked for.

type MappedColumn

type MappedColumn struct {
	Key      string `json:"key"`
	Label    string `json:"label"`
	Heading  string `json:"heading,omitempty"`
	Required bool   `json:"required,omitempty"`
}

MappedColumn says which heading of the file became which column.

type MappedRow

type MappedRow struct {
	Label, Heading string
	Required       bool
	Found          bool
}

MappedRow says which heading of the file became which column.

type Module

type Module struct {
	// contains filtered or unexported fields
}

Module is what the application registers.

It implements foundation.Module, which is Name and Routes and nothing else. It also implements foundation.Migratable, because it owns a table; foundation.Publishable, because it hands view sources to the project; foundation.Bootable, because it refuses to start while those views are not linked; foundation.Schedulable, because it declares the clean of the files; and the job registration the framework reads from a module that has one.

func New

func New(cfg Config, db *data.DB, sessions *security.SessionStore) (*Module, error)

New returns the module, or the reason it cannot be built.

The collaborators are parameters and not fields somebody fills in afterwards: a module that could be registered half-wired is a module whose first request is the thing that reports the missing half. It returns an error rather than panicking, because everything it refuses is a wiring mistake, and a wiring mistake found at boot costs one restart.

func (*Module) Boot

func (m *Module) Boot(context.Context) error

Boot refuses to serve when a view this package renders is not in the binary.

A compiled view registers itself from init(), so by the time anything boots the question has one answer already: either the application published the files, compiled them and imported the package they became, or it did not. Asking here turns "did anybody run the install command" into one refusal at start-up that names the views and the command, instead of a 500 on the first request that reached one of them.

It also holds the destination. Every file the archive offers has to land under the module directory named after this module.

func (*Module) Clean

func (m *Module) Clean(ctx context.Context) error

Clean deletes, in every tenant, the files of the runs that expired, and forgets where they were. An upload that nobody confirmed ends discarded. The run itself stays: the history keeps the counts. Schedule runs it daily; an application with a scheduler of its own calls it from there.

func (*Module) Labels

func (m *Module) Labels() Labels

Labels answers the sentences of the module's locale.

func (*Module) Migrations

func (m *Module) Migrations() []foundation.Migration

Migrations declares the schema this module owns: the runs. The queue's own table is the application's, migrated with the queue.

They are returned in the order their names sort in, which is the order they apply in.

func (*Module) Name

func (m *Module) Name() string

Name is the module identifier: a lowercase slug, stable, no spaces. It is what `aru route:list` groups by and what the route names are prefixed with.

func (*Module) Publishes

func (m *Module) Publishes() []foundation.Publication

Publishes declares the files this package offers, each at the path it takes relative to the root of the project.

One tree, under one tag. A publication may carry a page, a component, a configuration file, a migration, a catalogue of sentences or an asset, and this package offers the first of those and nothing else. Each absence is a decision rather than an omission:

  • configuration is the Config struct New is handed, checked by the compiler and validated before the module exists. A file copied into the project beside it would be a second place to say the same thing, and only one of the two could be the one the code reads.
  • a stylesheet, a script or any other asset is registered with the view layer and served from an address derived from its own bytes. Copying one into the project would put a second copy of those bytes under a second address, and a page can only reference one of them.
  • translations are overridden by writing the lines the application wants into its own override tree, which the catalogue loader already reads. A copy of every line this package ships is a copy that goes stale, and it goes stale without saying so.
  • migrations are declared and collected, never copied. A copy in the project's own migration directory is found by the runner as well, so one schema change applies twice under two names.

What is left is the markup, and it is here for the reason the others are not: it is the one thing the project is expected to edit. A package cannot know what a screen should say in a product it has never seen.

func (*Module) RegisterJobHandlers

func (m *Module) RegisterJobHandlers(worker *queue.Worker) error

RegisterJobHandlers gives the worker the two jobs of the package. The worker is the application's, started by its worker command.

func (*Module) Routes

func (m *Module) Routes(r *fhttp.Router)

Routes registers the module's routes under the configured prefix. They are named, so a URL is built from a name rather than written out a second time.

Every route needs somebody signed in, so the guard is mounted once, on the router they are registered on: r is replaced by the guarded group before the first registration, and no unguarded router is left in scope for the next route to be added to.

func (*Module) Schedule

func (m *Module) Schedule() []foundation.Task

Schedule declares the daily clean of the files. The files of an upload, a report and an export are deleted when the run expires: they hold the rows of the person's records, and a copy that nobody remembers is a copy that leaks. CleanSpec "-" declares none.

func (*Module) Service

func (m *Module) Service() *Service

Service returns the use cases, for an application that starts an import or an export from somewhere other than these routes.

func (*Module) URLs

func (m *Module) URLs() URLs

URLs returns the addresses of the module, built from the configuration. An application links to the import of a definition with it, and never writes the path a second time.

type Outcome

type Outcome int

Outcome is what writing a row did.

const (
	// Created says the row made a new record.
	Created Outcome = iota + 1
	// Updated says the row changed a record that existed.
	Updated
	// Unchanged says the record already held what the row says.
	Unchanged
)

type Page

type Page struct {
	Rows []*Run
	// Total is how many runs match, Page the page this is and Pages how many
	// there are.
	Total, Page, Pages int
}

Page is a page of the history.

type Paths

type Paths struct {
	Runs, Template, Export, Progress, Confirm, Discard, Refused, File string
}

Paths are the segments of the routes under Prefix, so an application that speaks another language renames them without touching the package.

type Preview

type Preview struct {
	// Header says the first row of the file was a heading.
	Header bool `json:"header"`
	// Columns are the columns of the definition and what the file calls each.
	Columns []MappedColumn `json:"columns"`
	// Ignored are the headings of the file no column answers to.
	Ignored []string `json:"ignored,omitempty"`
	// Total is the rows read, Valid those that passed the check and Invalid
	// those that did not.
	Total   int `json:"total"`
	Valid   int `json:"valid"`
	Invalid int `json:"invalid"`
	// Sample is the first rows, and Problems the first rows refused.
	Sample   []Line `json:"sample,omitempty"`
	Problems []Line `json:"problems,omitempty"`
}

Preview is what the screen shows before the confirmation.

type Refusal

type Refusal struct{ Reason string }

Refusal is the error that refuses one row and nothing else: its Reason is what the report tells the person, beside the row's number. Any other error a Definition returns is not about the row, and stops the import.

func (*Refusal) Error

func (r *Refusal) Error() string

Error is the reason.

type Resource

type Resource struct {
	// contains filtered or unexported fields
}

Resource is the list of fields one Run is allowed to answer with. A declared list rather than the entity itself: an encoder handed the entity answers with whatever it happens to have, and TenantID and the keys of the files are what belong in no response.

func NewResource

func NewResource(r Run) Resource

NewResource snapshots a run for a response.

func (Resource) ToArray

func (r Resource) ToArray() map[string]any

ToArray returns the fields that may leave, by name.

func (Resource) With

func (r Resource) With() map[string]any

With returns what goes beside the fields: nothing does.

type Row

type Row map[string]string

Row is one line of a file as the definition sees it: the value of each column by the column's Key, trimmed. A column the file does not carry, or leaves empty, is the empty string.

type Run

type Run struct {
	model.Model

	// ID is the identifier, generated by the application.
	ID string `db:"id"`
	// TenantID is the space the run belongs to, written from the Grant.
	TenantID string `db:"tenant_id"`
	// Definition is the key of the Definition.
	Definition string `db:"definition"`
	// Kind is KindImport or KindExport.
	Kind string `db:"kind"`
	// Status is where the run is.
	Status string `db:"status"`
	// Format is the format of the file: of the upload, or of the export.
	Format string `db:"format"`
	// FileName is what the person called the upload, or what the export is
	// called when it downloads.
	FileName string `db:"file_name"`
	// FileKey is where the upload or the export is on the Disk, generated by the
	// server. Empty once the clean deleted it, or before an export wrote it.
	FileKey string `db:"file_key"`
	// FileSize is its size in bytes.
	FileSize int64 `db:"file_size"`
	// ReportKey is where the refused rows are, as a CSV; empty when none were.
	ReportKey string `db:"report_key"`
	// Total is the rows in the file, and Processed how many the job has read.
	Total     int64 `db:"total"`
	Processed int64 `db:"processed"`
	// Created, Updated and Unchanged say what writing the rows did, and Refused
	// how many were turned away.
	Created   int64 `db:"created"`
	Updated   int64 `db:"updated"`
	Unchanged int64 `db:"unchanged"`
	Refused   int64 `db:"refused"`
	// Preview is what the screen shows before the confirmation, as JSON.
	Preview string `db:"preview"`
	// Sample is the first refused rows, as JSON, so the screen needs no file.
	Sample string `db:"sample"`
	// Failure is why a run failed, in a sentence the person can read.
	Failure string `db:"failure"`
	// ActorID is who started the run.
	ActorID string `db:"actor_id"`
	// CreatedAt and UpdatedAt are in UTC.
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
	// StartedAt and FinishedAt are when the worker began and ended.
	StartedAt  *time.Time `db:"started_at"`
	FinishedAt *time.Time `db:"finished_at"`
	// ExpiresAt is when the clean deletes the files of the run.
	ExpiresAt time.Time `db:"expires_at"`
}

Run is one import or export: the file, the state of the work and what it did.

It embeds the model, so a row returned by a query carries the connection and can be saved again. Build a new one with Runs(db).New().

A run keeps counts and never the content of a row: the rows are in the file, which the clean deletes, and in the application's own tables.

func (Run) CanConfirm

func (r Run) CanConfirm() bool

CanConfirm says the run is an upload waiting for the person's yes.

func (Run) CanDiscard

func (r Run) CanDiscard() bool

CanDiscard says the same: only an upload nobody confirmed can be thrown away.

func (Run) Current

func (r Run) Current() Status

Current is the status as a Status.

func (Run) Finished

func (r Run) Finished() bool

Finished says the work ended, well or not.

func (*Run) Fresh

func (row *Run) Fresh(ctx context.Context, g auth.Grant, with ...string) (*Run, error)

Fresh reads the row again, as a new Run; it shadows the untyped one the embedded model promotes.

func (Run) HasFile

func (r Run) HasFile() bool

HasFile says the file the run produced or read is still on the Disk.

func (Run) HasReport

func (r Run) HasReport() bool

HasReport says the refused rows are still on the Disk.

func (Run) IsExport

func (r Run) IsExport() bool

func (Run) IsImport

func (r Run) IsImport() bool

IsImport and IsExport say which kind of run it is.

func (Run) Percent

func (r Run) Percent() int

Percent is how far along the job is, from 0 to 100.

func (*Run) Replicate

func (row *Run) Replicate(except ...string) (*Run, error)

Replicate returns an unsaved copy of the row without its key; it shadows the untyped one the embedded model promotes.

type RunCollection

type RunCollection []*Run

RunCollection is rows of Run, as a query returns them.

func RunCollectionOf

func RunCollectionOf(rows model.Rows) RunCollection

RunCollectionOf converts rows of the table of Run. A nil input is a nil collection.

func (RunCollection) Contains

func (c RunCollection) Contains(key any) bool

func (RunCollection) Except

func (c RunCollection) Except(keys ...any) RunCollection

func (RunCollection) Find

func (c RunCollection) Find(key any) *Run

func (RunCollection) FindOrFail

func (c RunCollection) FindOrFail(key any) (*Run, error)

func (RunCollection) Fresh

func (c RunCollection) Fresh(ctx context.Context, g auth.Grant, with ...string) (RunCollection, error)

func (RunCollection) Load

func (c RunCollection) Load(ctx context.Context, g auth.Grant, relations ...string) error

func (RunCollection) LoadCount

func (c RunCollection) LoadCount(ctx context.Context, g auth.Grant, relations ...string) error

func (RunCollection) LoadMissing

func (c RunCollection) LoadMissing(ctx context.Context, g auth.Grant, relations ...string) error

func (RunCollection) ModelKeys

func (c RunCollection) ModelKeys() []any

func (RunCollection) Only

func (c RunCollection) Only(keys ...any) RunCollection

func (RunCollection) Pluck

func (c RunCollection) Pluck(column string) []any

func (RunCollection) Rows

func (c RunCollection) Rows() model.Rows

Rows converts the collection back to the core's rows.

func (RunCollection) ToArray

func (c RunCollection) ToArray() []map[string]any

func (RunCollection) Unique

func (c RunCollection) Unique() RunCollection

type RunPolicy

type RunPolicy struct {
	// contains filtered or unexported fields
}

RunPolicy decides about the runs of the Definitions it was built over.

func NewRunPolicy

func NewRunPolicy(defs []Definition) RunPolicy

NewRunPolicy builds the policy over the actions of the definitions: an action no Definition names is refused, whoever holds it.

func (RunPolicy) Can

func (p RunPolicy) Can(_ context.Context, s security.Subject, a security.Action, record Run) error

Can decides. It denies by default and has one rule that allows: the subject holds the Definition's action and the run, when there is one, is in the subject's own tenant.

func (RunPolicy) Writes

func (RunPolicy) Writes(a security.Action) bool

Writes reports whether an action changes anything: every one but the read forms.

type RunQuery

type RunQuery struct {
	// contains filtered or unexported fields
}

RunQuery is a query on the table of Run.

The methods that build return the query, so they chain. The methods that run take a context and an auth.Grant, and return Run rows or a value; there is no way to run one without a Grant.

func Runs

func Runs(db model.DB) *RunQuery

Runs starts a query on the table of Run, through db.

Global scopes are on and, unless the table is Global, every statement is filtered by the tenant of the Grant its terminal takes.

func (*RunQuery) AddSelect

func (q *RunQuery) AddSelect(columns ...any) *RunQuery

func (*RunQuery) Avg

func (q *RunQuery) Avg(ctx context.Context, g auth.Grant, column any) (any, error)

func (*RunQuery) Base

func (q *RunQuery) Base() *model.Builder

Base returns the core builder the query forwards to. It is for code in this package -- a local scope in the custom block of model.go -- and for the factory, which builds rows through it.

func (*RunQuery) Chunk

func (q *RunQuery) Chunk(ctx context.Context, g auth.Grant, count int, callback func(RunCollection, int) (bool, error)) error

func (*RunQuery) Clone

func (q *RunQuery) Clone() *RunQuery

Clone returns an independent copy of the query.

func (*RunQuery) Count

func (q *RunQuery) Count(ctx context.Context, g auth.Grant, columns ...any) (int64, error)

func (*RunQuery) Create

func (q *RunQuery) Create(ctx context.Context, g auth.Grant, attributes map[string]any) (*Run, error)

func (*RunQuery) Cursor

func (q *RunQuery) Cursor(ctx context.Context, g auth.Grant) iter.Seq2[*Run, error]

func (*RunQuery) CursorPaginate

func (q *RunQuery) CursorPaginate(ctx context.Context, g auth.Grant, perPage int, cursor *pagination.Cursor, opts pagination.Options, columns ...any) (RunCollection, *pagination.CursorPage, error)

func (*RunQuery) Decrement

func (q *RunQuery) Decrement(ctx context.Context, g auth.Grant, column string, amount any, extra map[string]any) (int64, error)

func (*RunQuery) Delete

func (q *RunQuery) Delete(ctx context.Context, g auth.Grant) (int64, error)

func (*RunQuery) DoesntExist

func (q *RunQuery) DoesntExist(ctx context.Context, g auth.Grant) (bool, error)

func (*RunQuery) DoesntHave

func (q *RunQuery) DoesntHave(relation, boolean string, callback func(*query.Builder)) *RunQuery

func (*RunQuery) Each

func (q *RunQuery) Each(ctx context.Context, g auth.Grant, count int, callback func(*Run, int) (bool, error)) error

func (*RunQuery) Exists

func (q *RunQuery) Exists(ctx context.Context, g auth.Grant) (bool, error)

func (*RunQuery) Find

func (q *RunQuery) Find(ctx context.Context, g auth.Grant, id any, columns ...any) (*Run, error)

func (*RunQuery) FindMany

func (q *RunQuery) FindMany(ctx context.Context, g auth.Grant, ids []any, columns ...any) (RunCollection, error)

func (*RunQuery) FindOrFail

func (q *RunQuery) FindOrFail(ctx context.Context, g auth.Grant, id any, columns ...any) (*Run, error)

func (*RunQuery) FindOrNew

func (q *RunQuery) FindOrNew(ctx context.Context, g auth.Grant, id any, columns ...any) (*Run, error)

func (*RunQuery) First

func (q *RunQuery) First(ctx context.Context, g auth.Grant, columns ...any) (*Run, error)

First returns the first row, or nil and no error when there is none.

func (*RunQuery) FirstOrCreate

func (q *RunQuery) FirstOrCreate(ctx context.Context, g auth.Grant, attributes, values map[string]any) (*Run, error)

func (*RunQuery) FirstOrFail

func (q *RunQuery) FirstOrFail(ctx context.Context, g auth.Grant, columns ...any) (*Run, error)

func (*RunQuery) FirstOrNew

func (q *RunQuery) FirstOrNew(ctx context.Context, g auth.Grant, attributes, values map[string]any) (*Run, error)

func (*RunQuery) FirstWhere

func (q *RunQuery) FirstWhere(ctx context.Context, g auth.Grant, column any, args ...any) (*Run, error)

func (*RunQuery) ForPage

func (q *RunQuery) ForPage(page, perPage int) *RunQuery

func (*RunQuery) ForceCreate

func (q *RunQuery) ForceCreate(ctx context.Context, g auth.Grant, attributes map[string]any) (*Run, error)

func (*RunQuery) Get

func (q *RunQuery) Get(ctx context.Context, g auth.Grant, columns ...any) (RunCollection, error)

Get runs the query and returns the rows.

func (*RunQuery) GroupBy

func (q *RunQuery) GroupBy(groups ...any) *RunQuery

func (*RunQuery) Has

func (q *RunQuery) Has(relation, operator string, count int, boolean string, callback func(*query.Builder)) *RunQuery

func (*RunQuery) Increment

func (q *RunQuery) Increment(ctx context.Context, g auth.Grant, column string, amount any, extra map[string]any) (int64, error)

func (*RunQuery) Insert

func (q *RunQuery) Insert(ctx context.Context, g auth.Grant, values ...map[string]any) (bool, error)

func (*RunQuery) InsertOrIgnore

func (q *RunQuery) InsertOrIgnore(ctx context.Context, g auth.Grant, values ...map[string]any) (bool, error)

func (*RunQuery) Join

func (q *RunQuery) Join(table any, first any, args ...any) *RunQuery

func (*RunQuery) Latest

func (q *RunQuery) Latest(column ...string) *RunQuery

func (*RunQuery) Limit

func (q *RunQuery) Limit(value int) *RunQuery

func (*RunQuery) Max

func (q *RunQuery) Max(ctx context.Context, g auth.Grant, column any) (any, error)

func (*RunQuery) Min

func (q *RunQuery) Min(ctx context.Context, g auth.Grant, column any) (any, error)

func (*RunQuery) New

func (q *RunQuery) New() (*Run, error)

New returns an empty Run on the query's connection, not yet saved, filled with what WithAttributes recorded on the query.

func (*RunQuery) Offset

func (q *RunQuery) Offset(value int) *RunQuery

func (*RunQuery) Oldest

func (q *RunQuery) Oldest(column ...string) *RunQuery

func (*RunQuery) OrDoesntHave

func (q *RunQuery) OrDoesntHave(relation string) *RunQuery

func (*RunQuery) OrHas

func (q *RunQuery) OrHas(relation, operator string, count int) *RunQuery

func (*RunQuery) OrWhere

func (q *RunQuery) OrWhere(column any, args ...any) *RunQuery

OrWhere adds an or-where clause; a func(*RunQuery) in place of the column adds a group.

func (*RunQuery) OrWhereDoesntHave

func (q *RunQuery) OrWhereDoesntHave(relation string, callback func(*query.Builder)) *RunQuery

func (*RunQuery) OrWhereHas

func (q *RunQuery) OrWhereHas(relation string, callback func(*query.Builder)) *RunQuery

func (*RunQuery) OrWhereNot

func (q *RunQuery) OrWhereNot(column any, args ...any) *RunQuery

OrWhereNot adds an or-joined negated where clause, or a negated group.

func (*RunQuery) OrWhereRelation

func (q *RunQuery) OrWhereRelation(relation string, column any, args ...any) *RunQuery

func (*RunQuery) OrderBy

func (q *RunQuery) OrderBy(column any, direction ...string) *RunQuery

func (*RunQuery) OrderByDesc

func (q *RunQuery) OrderByDesc(column any) *RunQuery

func (*RunQuery) Paginate

func (q *RunQuery) Paginate(ctx context.Context, g auth.Grant, perPage, page int, opts pagination.Options, columns ...any) (RunCollection, *pagination.LengthAwarePage, error)

func (*RunQuery) Pluck

func (q *RunQuery) Pluck(ctx context.Context, g auth.Grant, column string) ([]any, error)

func (*RunQuery) Select

func (q *RunQuery) Select(columns ...any) *RunQuery

func (*RunQuery) SelectRaw

func (q *RunQuery) SelectRaw(expression string, bindings ...any) *RunQuery

func (*RunQuery) SimplePaginate

func (q *RunQuery) SimplePaginate(ctx context.Context, g auth.Grant, perPage, page int, opts pagination.Options, columns ...any) (RunCollection, *pagination.Page, error)

func (*RunQuery) Sole

func (q *RunQuery) Sole(ctx context.Context, g auth.Grant, columns ...any) (*Run, error)

func (*RunQuery) SoleValue

func (q *RunQuery) SoleValue(ctx context.Context, g auth.Grant, column string) (any, error)

func (*RunQuery) Sum

func (q *RunQuery) Sum(ctx context.Context, g auth.Grant, column any) (any, error)

func (*RunQuery) Touch

func (q *RunQuery) Touch(ctx context.Context, g auth.Grant, column ...string) (int64, error)

func (*RunQuery) Update

func (q *RunQuery) Update(ctx context.Context, g auth.Grant, values map[string]any) (int64, error)

func (*RunQuery) UpdateOrCreate

func (q *RunQuery) UpdateOrCreate(ctx context.Context, g auth.Grant, attributes, values map[string]any) (*Run, error)

func (*RunQuery) Upsert

func (q *RunQuery) Upsert(ctx context.Context, g auth.Grant, values []map[string]any, uniqueBy, update []string) (int64, error)

func (*RunQuery) Value

func (q *RunQuery) Value(ctx context.Context, g auth.Grant, column string) (any, error)

func (*RunQuery) ValueOrFail

func (q *RunQuery) ValueOrFail(ctx context.Context, g auth.Grant, column string) (any, error)

func (*RunQuery) Where

func (q *RunQuery) Where(column any, args ...any) *RunQuery

Where adds a where clause; a func(*RunQuery) in place of the column adds a group.

func (*RunQuery) WhereBelongsTo

func (q *RunQuery) WhereBelongsTo(relationshipName string, related ...model.Entity) *RunQuery

func (*RunQuery) WhereBetween

func (q *RunQuery) WhereBetween(column any, from, to any) *RunQuery

func (*RunQuery) WhereColumn

func (q *RunQuery) WhereColumn(first any, args ...any) *RunQuery

func (*RunQuery) WhereDoesntHave

func (q *RunQuery) WhereDoesntHave(relation string, callback func(*query.Builder)) *RunQuery

func (*RunQuery) WhereExists

func (q *RunQuery) WhereExists(callback func(*query.Builder)) *RunQuery

func (*RunQuery) WhereHas

func (q *RunQuery) WhereHas(relation string, callback func(*query.Builder)) *RunQuery

func (*RunQuery) WhereHasCount

func (q *RunQuery) WhereHasCount(relation string, callback func(*query.Builder), operator string, count int) *RunQuery

func (*RunQuery) WhereIn

func (q *RunQuery) WhereIn(column any, values []any) *RunQuery

func (*RunQuery) WhereKey

func (q *RunQuery) WhereKey(id any) *RunQuery

func (*RunQuery) WhereNot

func (q *RunQuery) WhereNot(column any, args ...any) *RunQuery

WhereNot adds a negated where clause, or a negated group.

func (*RunQuery) WhereNotExists

func (q *RunQuery) WhereNotExists(callback func(*query.Builder)) *RunQuery

func (*RunQuery) WhereNotIn

func (q *RunQuery) WhereNotIn(column any, values []any) *RunQuery

func (*RunQuery) WhereNotNull

func (q *RunQuery) WhereNotNull(columns ...any) *RunQuery

func (*RunQuery) WhereNull

func (q *RunQuery) WhereNull(columns ...any) *RunQuery

func (*RunQuery) WhereRelation

func (q *RunQuery) WhereRelation(relation string, column any, args ...any) *RunQuery

func (*RunQuery) With

func (q *RunQuery) With(relations ...string) *RunQuery

func (*RunQuery) WithAvg

func (q *RunQuery) WithAvg(relation, column string) *RunQuery

func (*RunQuery) WithCount

func (q *RunQuery) WithCount(relations ...string) *RunQuery

func (*RunQuery) WithExists

func (q *RunQuery) WithExists(relation string) *RunQuery

func (*RunQuery) WithMax

func (q *RunQuery) WithMax(relation, column string) *RunQuery

func (*RunQuery) WithMin

func (q *RunQuery) WithMin(relation, column string) *RunQuery

func (*RunQuery) WithSum

func (q *RunQuery) WithSum(relation, column string) *RunQuery

func (*RunQuery) Without

func (q *RunQuery) Without(relations ...string) *RunQuery

func (*RunQuery) WithoutGlobalScope

func (q *RunQuery) WithoutGlobalScope(identifier string) *RunQuery

func (*RunQuery) WithoutGlobalScopes

func (q *RunQuery) WithoutGlobalScopes(identifiers ...string) *RunQuery

type RunRow

type RunRow struct {
	ID, Definition, DefinitionLabel string
	Kind, KindLabel                 string
	Status, StatusKey               string
	StatusVariant                   string
	FileName, Format, Size          string
	Total, Processed                int64
	Created, Updated                int64
	Unchanged, Refused              int64
	Percent                         int
	Result                          string
	When, WhenISO                   string
	Finished                        string
	URL                             string
	Active, IsFinished              bool
	IsImport, IsExport              bool
}

RunRow is a run as a screen shows it.

type Screen

type Screen struct {
	hview.Page
	Labels Labels
	// URLs are the addresses of the package's routes.
	URLs URLs
	// Location is the time zone instants are written in.
	Location *time.Location
	// contains filtered or unexported fields
}

Screen is what every view of the package starts from: the page, the labels, the addresses and the application's frame.

It embeds the Hesape page, so a layout reads the title, the CSRF token and the flash from it as it does from any other page, and it carries what the application's Chrome answered, for the layout to draw the sidebar from.

func (Screen) All

func (s Screen) All() []string

All is one line per rejected message, in the screen's language.

func (Screen) Chrome

func (s Screen) Chrome() any

Chrome is what Config.Chrome answered for this screen; nil when Config.Chrome is nil.

func (Screen) ErrorSummary

func (s Screen) ErrorSummary() []string

ErrorSummary is All under the name the framework's layout contract uses.

func (Screen) FieldErrors

func (s Screen) FieldErrors(name string) []string

FieldErrors are the messages of one field.

func (Screen) Form

func (s Screen) Form() FormState

Form is the page as a form's components read it.

func (Screen) HasErrors

func (s Screen) HasErrors() bool

HasErrors says whether the attempt that landed here was rejected.

func (Screen) LogValue

func (s Screen) LogValue() slog.Value

LogValue keeps a page out of the logs but for its public fields.

func (Screen) MarshalJSON

func (s Screen) MarshalJSON() ([]byte, error)

MarshalJSON hands the framework's page a JSON of its own.

func (Screen) SearchLabel

func (s Screen) SearchLabel(table string) string

SearchLabel is the accessible name of a table's search box.

func (Screen) StatusOptions

func (s Screen) StatusOptions() []components.SelectOption

StatusOptions are the statuses a history can be filtered by.

func (Screen) TableLines

func (s Screen) TableLines() components.DataTableLines

TableLines are the sentences of a data table, in the screen's language.

func (Screen) When

func (s Screen) When(t time.Time) string

When writes an instant in the screen's zone.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service is every use case of the package: it validates the request, asks the policy for a Grant, and reaches the database only with it.

func NewService

func NewService(db *data.DB, cfg Config) *Service

NewService builds the service over a database handle and a configuration.

func (*Service) Clean

func (s *Service) Clean(ctx context.Context) error

Clean is Module.Clean over the service.

func (*Service) Confirm

func (s *Service) Confirm(ctx context.Context, actor security.Subject, key, id string) (*Run, error)

Confirm queues the import of an upload. The status moves from uploaded to queued and the job is pushed in the same transaction, so a job never refers to a run that is not there and a run is never queued without its job. Confirming twice is the same as confirming once.

func (*Service) Discard

func (s *Service) Discard(ctx context.Context, actor security.Subject, key, id string) error

Discard throws away an upload nobody confirmed, and its file.

func (*Service) Find

func (s *Service) Find(ctx context.Context, actor security.Subject, key, id string) (*Run, error)

Find returns one run for a subject that may see it: the one that holds the action of the run's kind.

func (*Service) List

func (s *Service) List(ctx context.Context, actor security.Subject, q ListQuery) (Page, error)

List returns a page of the history of the space: the runs of the definitions and the sides the subject holds an action for, newest first.

func (*Service) Open

func (s *Service) Open(ctx context.Context, actor security.Subject, key, id string, report bool) (*Run, Download, error)

Open returns a file of a run: the report of the refused rows, or the file the run read or wrote. The read is authorized, and written to the activity log.

func (*Service) RequestExport

func (s *Service) RequestExport(ctx context.Context, actor security.Subject, key, format string) (*Run, error)

RequestExport queues the export of a definition's records in a format. The file is written in the background and the run says when it is ready.

func (*Service) Template

func (s *Service) Template(ctx context.Context, actor security.Subject, key, format string, w io.Writer) (name, kind string, err error)

Template writes the template of a definition in a format: the headings, and the example rows, so the person sees what a row looks like. It returns the name the file downloads as and its content type.

func (*Service) Upload

func (s *Service) Upload(ctx context.Context, actor security.Subject, key string, in Upload) (*Run, error)

Upload reads a file, checks every row of it against the definition and keeps it for the person to look at before confirming. Nothing is written to the application's tables yet.

What is wrong with the file as a whole -- empty, too large, not a table, a required column missing, too many rows -- is returned as validation errors on the "file" field. A row that is wrong is not an error: it is in the preview.

type ShowPageData

type ShowPageData struct {
	Screen
	Run        RunRow
	Definition DefinitionView
	BackURL    string

	// ProgressURL is where the fragment asks again, and Polling says it should.
	ProgressURL string
	Polling     bool

	ConfirmURL, DiscardURL, RefusedURL, FileURL string
	CanConfirm                                  bool
	HasFile, HasReport                          bool

	// Heads are the labels of the definition's columns, in order, and Mapped
	// what the file called each.
	Heads  []string
	Mapped []MappedRow
	// Ignored is the headings of the file no column answers to.
	Ignored string
	// Header says the first row was read as a heading.
	Header bool

	// Valid and Invalid are the counts of the preview.
	Valid, Invalid int
	// Sample is the first rows of the file; Problems the first refused ones, in
	// the preview and in the report.
	Sample      []LineRow
	Problems    []LineRow
	MoreProblem int

	// Failure is why a run failed, in words.
	Failure string
	// Retention is how many days the files are kept.
	Retention int
	// Progress is what the bar needs.
	Indeterminate bool
}

ShowPageData is what the page of a run, and the fragment of its state, are drawn from. They share the data: the fragment is the part of the page that changes while the run goes, and the page asks for it again until it is over.

func (ShowPageData) ExportDoneLead

func (d ShowPageData) ExportDoneLead() string

ExportDoneLead says what a finished export holds and how long it is kept.

func (ShowPageData) HeaderNote

func (d ShowPageData) HeaderNote() string

HeaderNote says how the first row of the file was read.

func (ShowPageData) IgnoredText

func (d ShowPageData) IgnoredText() string

IgnoredText names the headings of the file nothing answered to.

func (ShowPageData) KeepNote

func (d ShowPageData) KeepNote() string

KeepNote says how long the file is kept.

func (ShowPageData) MoreText

func (d ShowPageData) MoreText(report bool) string

MoreText says how many refused rows the list does not show, and where the rest is.

func (ShowPageData) ProgressLead

func (d ShowPageData) ProgressLead() string

func (ShowPageData) ProgressTitle

func (d ShowPageData) ProgressTitle() string

ProgressTitle and ProgressLead are what the card of a run that is going says.

func (ShowPageData) StartedText

func (d ShowPageData) StartedText() string

StartedText says when the run was sent, and when it ended.

type Status

type Status string

Status is where a run is. An import is Uploaded until somebody confirms it; an export starts Queued.

const (
	// StatusUploaded is a file that was read and checked and waits for the
	// person to look at the preview and confirm.
	StatusUploaded Status = "uploaded"
	// StatusQueued is work waiting for the worker.
	StatusQueued Status = "queued"
	// StatusRunning is work the worker is doing.
	StatusRunning Status = "running"
	// StatusDone is work that ended; an import may still have refused rows.
	StatusDone Status = "done"
	// StatusFailed is work that stopped on something that was not a row.
	StatusFailed Status = "failed"
	// StatusDiscarded is an upload nobody confirmed: the person threw it away,
	// or the clean did.
	StatusDiscarded Status = "discarded"
)

func (Status) Active

func (s Status) Active() bool

Active says the worker is, or will be, working on the run: the screen keeps asking how far along it is.

func (Status) Valid

func (s Status) Valid() bool

Valid says s is one of them.

type Summary

type Summary struct {
	// Definition is the key of the definition.
	Definition string
	// Total is the rows read, and Created, Updated, Unchanged and Refused what
	// became of them; the four add up to Total.
	Total, Created, Updated, Unchanged, Refused int
}

Summary is what a finished import says, as a definition's Finish is told.

type URLs

type URLs struct {
	// contains filtered or unexported fields
}

URLs builds the addresses of the package from the configuration, so a link to an import is written once, here, and never spelled out a second time.

func NewURLs

func NewURLs(prefix string, paths Paths) URLs

NewURLs builds the addresses for a prefix and paths without a module: a view test, or a menu built before the module is.

func (URLs) Confirm

func (u URLs) Confirm(key, id string) string

Confirm and Discard are where an upload's preview posts.

func (URLs) Definition

func (u URLs) Definition(key string) string

Definition is the page of one definition: the upload, the template, the export and its history. The upload form posts here.

func (URLs) Discard

func (u URLs) Discard(key, id string) string

func (URLs) Export

func (u URLs) Export(key string) string

Export is where a request for an export is posted.

func (URLs) File

func (u URLs) File(key, id string) string

func (URLs) Index

func (u URLs) Index() string

Index is the page that lists every definition and the history of the space.

func (URLs) Progress

func (u URLs) Progress(key, id string) string

Progress is the fragment of a run's state, which the page asks for again while the run is going.

func (URLs) Refused

func (u URLs) Refused(key, id string) string

Refused is the CSV of the rows an import turned away, and File the file the run read or wrote.

func (URLs) Run

func (u URLs) Run(key, id string) string

Run is the page of one run.

func (URLs) Template

func (u URLs) Template(key, format string) string

Template is the address of the template of a definition in a format.

type Upload

type Upload struct {
	// Name is what the person's computer called it.
	Name string
	// Body is its content.
	Body io.Reader
}

Upload is a file as it arrived.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL