build

package
v1.1.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 15, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package build — alertmanager.go: Build-pipeline integration for the Alertmanager routing config, with ɳSentry-aware receiver + route generation.

`nself build` must produce a complete monitoring/alertmanager.yml that includes a dedicated `nsentry` receiver plus one routing rule per installed ɳSentry plugin. This file owns the build-time stitching — the base AlertmanagerConfig template lives in internal/compose/monitoring/alerts.go and is reused here so we never fork the YAML schema.

Behavior:

  • WriteAlertmanagerConfig renders alertmanager.yml + appends ɳSentry routing stanzas atomically into <workdir>/monitoring/alertmanager.yml.
  • When zero ɳSentry plugins are installed, no nsentry receiver or route blocks are emitted; the base config stands alone.
  • Idempotent: identical inputs → byte-identical output, so Docker bind-mounts don't churn on rebuild.

Package build — grafana_nsentry.go: Auto-provisions Grafana datasource + dashboards for the ɳSentry plugin bundle.

When the ɳSentry bundle is licensed and any of its plugins are installed, `nself build` must emit:

  • monitoring/grafana/provisioning/datasources/nsentry.yaml — datasource entry referencing the existing Prometheus datasource so panels resolve.
  • monitoring/grafana/provisioning/dashboards/nsentry.yaml — dashboard provisioning entry pointing at the dashboard directory.
  • monitoring/grafana/dashboards/nsentry/<slug>.json — one dashboard JSON per installed ɳSentry plugin (uptime, status, incident, alert-router, slo, synthetic, rum, and any expansion plugins).

T04 wires the `nself-status-page` plugin's status summary endpoint (HTTP GET <host>:3832/status) into the status-page dashboard via a Grafana text panel that renders a status-summary link, so operators land directly on the public status page from the bundle overview.

Behavior:

  • Detection reuses the canonical ɳSentry plugin list from prometheus.go (nsentryPlugins) — single source of truth for which plugins exist.
  • Zero ɳSentry plugins installed → no files emitted, no directories created. Caller invokes WriteNSentryGrafanaProvisioning regardless; it is a no-op when nothing is installed.
  • Idempotent: re-running with the same set of installed plugins yields byte-identical files (deterministic JSON marshaling + stable key order).
  • Adding a new ɳSentry plugin: append to nsentryPlugins in prometheus.go and add a dashboard template entry below (dashboardTitles map).

Package build — loki.go: Programmatic Loki + Promtail YAML rendering for the monitoring bundle.

When the monitoring bundle is enabled, `nself build` must emit monitoring/loki.yml + monitoring/promtail.yml with a configurable retention period (default 30d / 720h), optional S3-backed chunk storage, and per-project tenant labels. This file owns the build-pipeline integration — the actual YAML templates live in internal/compose/monitoring/loki.go and are reused here.

Behavior:

  • LokiBuildOptions wraps the monitoring.LokiConfig + monitoring.PromtailConfig pair with build-time overrides (retention, S3 storage, project name).
  • WriteLokiConfigs renders both files into <workdir>/monitoring/ idempotently (atomic temp-file + rename, 0644 permissions, parent dir 0755).
  • Re-running the build with the same options yields byte-identical files so downstream Docker volume hashes don't churn.
  • Returns the count of files written (always 0 or 2 when monitoring on).

np_plugins_init.go — np_plugins seed SQL generator.

Writes an idempotent SQL seed script that ensures one row exists in np_plugins for every plugin installed under pluginDir. The script is named 05-np-plugins-seed.sql so it runs strictly AFTER 04-np-plugins.sql (which CREATE TABLE IF NOT EXISTS np_plugins ... ) emitted by internal/postgres/generator.go.

Idempotency is guaranteed by INSERT ... ON CONFLICT (name) DO NOTHING. Re-running `nself build` produces the same script with the same content; re-running `nself start` (or, equivalently, re-applying init scripts on a rebuilt postgres volume) inserts no duplicate rows.

Why a "seed empty row" matters: downstream consumers (admin UI, doctor, plugin loader) join np_plugins by name. When a plugin is installed on disk but no row exists yet, those joins drop the plugin from views. Seeding an empty row at build time keeps every installed plugin discoverable from SQL, even before its first runtime touch.

Package build — prometheus.go: Auto-generates Prometheus scrape config for the ɳSentry plugin bundle.

When the ɳSentry bundle is licensed and any of its 7 baseline plugins are installed, `nself build` must emit scrape-target blocks for each one in monitoring/prometheus.yml so Prometheus can pull /metrics from each plugin. This file owns ɳSentry detection + scrape-target generation; the actual prometheus.yml render lives in internal/compose/monitoring/prometheus.go.

Behavior:

  • Detects which of the 7 ɳSentry plugins (uptime-monitor, status-page, incident-mgmt, alert-router, slo-tracker, synthetic-monitor, rum) are installed by checking <pluginDir>/<plugin-name>/plugin.json.
  • Returns one ScrapeTarget per installed plugin with bundle="nsentry" and plugin="<slug>" labels.
  • Idempotent: re-running yields the same target list (no duplicates).
  • Zero installed → returns nil (no nsentry stanza emitted upstream).

Index

Constants

View Source 
const ComposeGeneratedHeader = "# GENERATED BY nself build - DO NOT HAND EDIT\n" +
	"# Run 'nself build' to regenerate this file.\n"

ComposeGeneratedHeader is the canonical marker prepended to every generated docker-compose.yml. Pre-commit hooks and auditors grep for this exact line to confirm a compose file was produced by `nself build` and not hand-edited. See S32-T12 and the PPI nSelf-First Doctrine.

Variables

This section is empty.

Functions

func AppendNSentryTargets added in v1.1.0 

func AppendNSentryTargets(cfg *monitoring.PrometheusConfig, pluginDir string) int

AppendNSentryTargets merges NSentryScrapeTargets(pluginDir) into cfg.Targets, skipping any target whose JobName already exists in cfg.Targets. This is the idempotency guard for re-running `nself build` on a project whose prometheus.yml was previously generated — duplicate stanzas are never emitted.

cfg must be non-nil. Returns the count of targets added (0 when none installed or all already present).

func CheckGoPluginDockerfiles added in v1.0.3 

func CheckGoPluginDockerfiles(pluginDir string) []string

CheckGoPluginDockerfiles scans pluginDir for Go plugins and verifies each has a Dockerfile containing a HEALTHCHECK instruction. Returns a list of warning strings for plugins where the check fails. Never returns an error — missing Dockerfiles or unreadable files produce warnings, not hard failures.

func ComputePluginEnvVars 

func ComputePluginEnvVars(workdir, pluginDir string) map[string]string

ComputePluginEnvVars returns environment variables needed by plugin compose files. These are written to .env.computed so that `docker compose` can interpolate them in plugin compose fragments.

Variables returned:

  • NSELF_PLUGIN_DIR: absolute path to the global plugin directory
  • PLUGIN_{NAME}_INTERNAL_URL: http://plugin-{name}:{port} for every declared dependency (required + optional) of every installed plugin. Only wired when the dependency plugin is also installed.

func DefaultPluginDir 

func DefaultPluginDir() string

DefaultPluginDir returns the default global plugin installation directory (~/.nself/plugins). Falls back to /tmp/.nself/plugins when the home directory cannot be determined.

func DetectServices 

func DetectServices(cfg *config.Config) []string

DetectServices returns the list of Docker service names that should be generated in docker-compose.yml based on the current configuration.

Core services (4): postgres, hasura, auth, nginx Optional services (6): redis, minio, mailpit (email), search, functions, admin Custom services: CS_1..CS_10

Monitoring and MLflow are free plugins — not generated by core.

Redis auto-enable: when any BullMQ-backed plugin (ai, claw, mux, cron, notify) is installed in DefaultPluginDir() and cfg.Redis.Enabled is false, "redis" is appended automatically. The compose generator is driven from cfg, so callers (e.g. Build) must also set cfg.Redis.Enabled = true before passing cfg to compose.NewGenerator to keep both outputs consistent.

func DiscoverPluginComposeFiles 

func DiscoverPluginComposeFiles(workdir, pluginDir string) ([]string, error)

DiscoverPluginComposeFiles scans pluginDir for installed plugins that contain a docker-compose.plugin.yml file. It returns absolute paths to each discovered compose file, sorted by plugin directory name for deterministic ordering. Plugins without a compose file are silently skipped (they are background-process plugins, not compose plugins).

func GenerateNpPluginsSeed added in v1.0.13 

func GenerateNpPluginsSeed(workdir, pluginDir string) (string, error)

GenerateNpPluginsSeed scans pluginDir for installed plugins (each is a subdir with a plugin.json manifest), determines a tier ('free' / 'pro') for each, and writes postgres/init/05-np-plugins-seed.sql. The script uses INSERT ... ON CONFLICT (name) DO NOTHING so re-runs are no-ops.

If pluginDir is missing or empty, the function still writes a header-only file (a single comment + no INSERTs). That keeps the path stable across builds — operators inspecting postgres/init/ won't see the file vanish when they uninstall every plugin.

Returns the absolute path of the file that was written.

func InjectPluginNginxRoutes 

func InjectPluginNginxRoutes(workdir, pluginDir string, cfg *config.Config) (int, error)

InjectPluginNginxRoutes scans pluginDir for installed plugins that ship nginx route configs (in a nginx/ subdirectory) and copies them into the project's nginx/sites/ directory after templating config variables.

Returns the number of config files injected. Plugins without a nginx/ directory are silently skipped.

func MergeOllamaSidecar added in v1.0.9 

func MergeOllamaSidecar(composeYAML []byte, ollamaEnv map[string]string) ([]byte, bool, error)

MergeOllamaSidecar injects an Ollama service into composeYAML when AI_OLLAMA_ENABLED=true is present in ollamaEnv. The injected service runs the official Ollama image on localhost:11434, which plugin-ai uses for local LLM inference without an external API key.

func NSentryScrapeTargets added in v1.1.0 

func NSentryScrapeTargets(pluginDir string) []monitoring.ScrapeTarget

NSentryScrapeTargets returns one Prometheus ScrapeTarget per installed ɳSentry plugin under pluginDir. A plugin is considered installed when <pluginDir>/<plugin-name>/plugin.json exists (matching the convention used by ListInstalled / DiscoverPluginComposeFiles).

Returns nil (not empty slice) when no ɳSentry plugin is installed so callers can cleanly omit the bundle stanza upstream. Output is stable-sorted by job name so repeated builds produce byte-identical prometheus.yml.

func NeedsRebuild 

func NeedsRebuild(workdir string) (bool, error)

NeedsRebuild reports whether the build outputs are stale and a rebuild is required. It returns true when any of the following hold:

  • .env or docker-compose.yml does not exist
  • .env is newer than docker-compose.yml
  • The CLI version that produced the last build differs from the running version
  • The build-version file is missing or unreadable

It returns false only when docker-compose.yml is at least as new as .env and the recorded build version matches the current CLI version.

func ReadComposeManifest 

func ReadComposeManifest(workdir string) ([]string, error)

ReadComposeManifest reads .nself/compose-files.txt and returns the ordered list of compose file paths. If the manifest does not exist, it returns a single-element slice with "docker-compose.yml" (relative, base only) so callers always get a usable default. Lines pointing to files that no longer exist on disk (e.g. a plugin was uninstalled) are silently skipped.

func RenderAlertmanagerBundle added in v1.1.0 

func RenderAlertmanagerBundle(opts AlertmanagerBuildOptions) ([]byte, error)

RenderAlertmanagerBundle returns the rendered alertmanager.yml bytes. The output is the base alertmanager template (from internal/compose/monitoring) with ɳSentry routing rules spliced in just before the `inhibit_rules:` section. When no ɳSentry plugin is installed, the splice is a no-op and the output equals the base render.

Pure function — no filesystem side effects. Unit tests assert on the returned bytes directly.

func RenderLokiBundle added in v1.1.0 

func RenderLokiBundle(opts LokiBuildOptions) (lokiYAML []byte, promtailYAML []byte, err error)

RenderLokiBundle returns the (lokiYAML, promtailYAML) pair for opts. It is pure — no filesystem side effects — so unit tests can assert on the rendered bytes without touching disk. The orchestrator calls WriteLokiConfigs which wraps this in atomic file writes.

func RenderNSentryDashboard added in v1.1.0 

func RenderNSentryDashboard(slug string) ([]byte, error)

RenderNSentryDashboard returns the dashboard JSON bytes for a single ɳSentry plugin slug. The dashboard contains three panels with PromQL queries filtered by bundle="nsentry" and plugin="<slug>":

  1. Request rate (rate(http_requests_total{...}[5m]))
  2. Error rate (rate(http_requests_total{status=~"5.."}[5m]))
  3. p95 latency (histogram_quantile(0.95, ...))

When slug == "status-page", a fourth text panel (T04) is appended that links operators directly to the public status summary endpoint exposed by the nself-status-page plugin (GET <host>:3832/status). The link text uses a Grafana variable so the host substitutes correctly per environment.

func RenderNSentryDashboardProvisioning added in v1.1.0 

func RenderNSentryDashboardProvisioning(pluginDir string) []byte

RenderNSentryDashboardProvisioning returns the YAML pointing Grafana's dashboard provisioner at the per-plugin dashboard JSON files emitted by RenderNSentryDashboard. Returns nil when no ɳSentry plugin is installed.

func RenderNSentryDatasourceYAML added in v1.1.0 

func RenderNSentryDatasourceYAML(pluginDir string) []byte

RenderNSentryDatasourceYAML returns the bytes for grafana/provisioning/datasources/nsentry.yaml. The datasource simply references the existing Prometheus datasource (provisioned by the core monitoring bundle) — the ɳSentry dashboards consume it via the ${DS_PROMETHEUS} variable. Emitting the file under the ɳSentry namespace makes provisioning per-bundle: uninstalling the bundle removes only the nsentry.yaml, leaving the core Prometheus datasource untouched.

Returns nil when no ɳSentry plugin is installed under pluginDir so callers can skip writing the file cleanly.

func RenderPluginConfigs 

func RenderPluginConfigs(workdir, pluginDir string, cfg *config.Config) (int, error)

RenderPluginConfigs walks installed plugins' configs/ directories and renders templates into the project workdir. For each installed plugin that has a configs/ directory, every file inside it is read, ${VAR} substitutions are replaced with project values, and the result is written to {workdir}/{relative_path} (stripping the configs/ prefix).

Returns the number of files rendered, or an error if any file operation fails.

func ShouldAutoEnableRedis 

func ShouldAutoEnableRedis(pluginDir string) bool

ShouldAutoEnableRedis reports whether any BullMQ-backed plugin that requires Redis (ai, claw, mux, cron, notify) is installed in pluginDir. Detection is based on the presence of a plugin.json manifest file in the plugin's sub-directory.

Use DefaultPluginDir() when no override is needed.

func WriteAlertmanagerConfig added in v1.1.0 

func WriteAlertmanagerConfig(workdir string, opts AlertmanagerBuildOptions) (int, error)

WriteAlertmanagerConfig renders the alertmanager.yml and writes it into <workdir>/monitoring/alertmanager.yml atomically. Returns 1 on success (one file written), 0 on validate failure. Uses the same atomicWrite pattern as the Loki + Prometheus pipelines so concurrent Docker readers never see partial files.

func WriteComposeManifest 

func WriteComposeManifest(workdir string, baseCompose string, pluginFiles []string) error

WriteComposeManifest writes .nself/compose-files.txt with one compose file path per line. The first line is always the base docker-compose.yml (absolute path). Subsequent lines are absolute paths to plugin compose files. The .nself/ directory is created if it does not exist.

func WriteLokiConfigs added in v1.1.0 

func WriteLokiConfigs(workdir string, opts LokiBuildOptions) (int, error)

WriteLokiConfigs renders the Loki + Promtail YAML and writes both into <workdir>/monitoring/ idempotently. Returns the number of files written (always 2 on success). The temp-file + rename pattern guarantees readers (Docker bind mounts) never see a partially-written file.

func WriteNSentryGrafanaProvisioning added in v1.1.0 

func WriteNSentryGrafanaProvisioning(workdir, pluginDir string) (int, error)

WriteNSentryGrafanaProvisioning emits the full Grafana auto-provisioning tree for the installed ɳSentry plugins under pluginDir. Writes: 

<workdir>/monitoring/grafana/provisioning/datasources/nsentry.yaml
<workdir>/monitoring/grafana/provisioning/dashboards/nsentry.yaml
<workdir>/monitoring/grafana/dashboards/nsentry/<slug>.json   (one per installed plugin)

Returns the count of files written (0 when no ɳSentry plugin installed). Idempotent: a second call with the same on-disk state produces byte-identical output (no .tmp leftovers, atomic rename pattern reused from loki.go's atomicWrite).

Types

type AlertmanagerBuildOptions added in v1.1.0 

type AlertmanagerBuildOptions struct {
	// ProjectName is the nSelf project slug. Required.
	ProjectName string
	// PluginDir is the directory under which ɳSentry plugin.json manifests
	// live (typically <workdir>/.nself/plugins). Empty disables ɳSentry
	// detection entirely (no nsentry stanzas emitted).
	PluginDir string
	// OncallEmail overrides the default ALERTMANAGER_ONCALL_EMAIL stub for
	// the critical receiver. Empty falls back to the env-var default in
	// monitoring.DefaultAlertmanagerConfig.
	OncallEmail string
	// NSentryEmail is the destination address for ɳSentry-routed alerts
	// when at least one ɳSentry plugin is installed. Empty defaults to
	// the same env-var fallback as the oncall receiver.
	NSentryEmail string
}

AlertmanagerBuildOptions captures the build-time inputs that shape the rendered alertmanager.yml. ProjectName is required so the receiver name can be scoped per nSelf project (avoids collisions when a single Alertmanager fronts multiple deploys).

type BuildOptions 

type BuildOptions struct {
	// Force rebuilds everything regardless of cache freshness.
	Force bool
	// Verbose enables detailed build progress output.
	Verbose bool
	// Check validates configuration and exits without generating files.
	Check bool
	// SecurityReport prints a detailed security audit after validation.
	SecurityReport bool
	// NoAutoRedis disables automatic Redis enablement when a BullMQ-backed
	// plugin (ai, claw, mux, cron, notify, push) is detected. Pass
	// --no-auto-redis from the CLI to opt out of this behaviour.
	NoAutoRedis bool
}

BuildOptions controls build behavior via CLI flags.

type BuildResult 

type BuildResult struct {
	// ProjectName is the sanitized project name from config.
	ProjectName string
	// ComposeFile is the path to the generated docker-compose.yml.
	ComposeFile string
	// NginxConfig is the path to the generated nginx/nginx.conf.
	NginxConfig string
	// SSLCerts is the number of SSL certificate sets generated.
	SSLCerts int
	// Duration is the wall-clock time the build took.
	Duration time.Duration
	// FilesGenerated is the total number of files written.
	FilesGenerated int
	// PluginComposeFiles lists the absolute paths to plugin compose files
	// discovered during build. Empty when no plugins with compose files
	// are installed.
	PluginComposeFiles []string
	// CAInstalled is true when the mkcert CA is trusted by the OS.
	CAInstalled bool
	// CAManualCmd is non-empty when the user must manually trust the CA.
	CAManualCmd string
	// HostsAdded is the number of new /etc/hosts entries written.
	HostsAdded int
	// HostsManualNote is non-empty when /etc/hosts could not be updated automatically.
	HostsManualNote string
}

BuildResult summarizes what the build produced.

func Build 

func Build(workdir string, opts BuildOptions) (*BuildResult, error)

Build orchestrates the full nself build pipeline.

The sequence follows BUILD_SPEC.md:

  1. Load config via env cascade
  2. Validate config (security, passwords, ports, CORS)
  3. If --check: return after validation
  4. Check cache (skip rebuild if not --force and cache fresh)
  5. Create required directories
  6. Generate SSL certificates
  7. Generate nginx configuration files
  8. Generate docker-compose.yml
  9. Write docker-compose.yml with 0600 permissions
  10. Write .env.computed (DATABASE_URL + DOCKER_NETWORK)
  11. Save build version to .nself/build-version
  12. Return BuildResult with summary

type LokiBuildOptions added in v1.1.0 

type LokiBuildOptions struct {
	// ProjectName is attached as a static label on every shipped log line so
	// operators can distinguish multiple projects on one host. Required.
	ProjectName string
	// RetentionPeriod overrides DefaultLokiConfig().RetentionPeriod. Loki
	// duration syntax (e.g. "720h", "30d", "168h"). Empty = use default (30d).
	// Per S9.T10 OPS-LOKI-RETENTION the doctor enforces a >= 720h floor.
	RetentionPeriod string
	// S3Bucket, when non-empty, switches Loki + Promtail chunk storage from
	// the default filesystem driver to s3 (via Cloudflare R2 / AWS S3). The
	// bucket must exist; credentials live in the runtime env. Empty disables.
	S3Bucket string
	// S3Region is the bucket's region (e.g. "auto" for R2, "us-east-1" for AWS).
	// Required when S3Bucket is non-empty.
	S3Region string
	// MultiTenant turns on Loki multi-tenant mode. Each nSelf project becomes
	// a tenant via the X-Scope-OrgID header set on Promtail pushes.
	MultiTenant bool
	// TenantID is the Promtail tenant header. Required when MultiTenant is true.
	TenantID string
}

LokiBuildOptions captures the build-time overrides applied on top of monitoring.DefaultLokiConfig + DefaultPromtailConfig. Zero-value fields fall back to the package defaults.

type PostValidateResult 

type PostValidateResult struct {
	ComposeValid bool
	NginxValid   bool
	PortsUnique  bool
	NamesUnique  bool
	Errors       []string
	Warnings     []string
}

PostValidateResult contains the results of all post-build checks.

func PostValidate 

func PostValidate(composePath, nginxConfDir string) PostValidateResult

PostValidate runs all post-build checks on the generated files.

composePath: path to docker-compose.yml nginxConfDir: path to nginx/sites/ directory (used to locate the parent nginx.conf)

Returns result with all findings. Never returns a Go error — all findings go in the result Errors/Warnings slices.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.