plugin

Documentation

Index

• Constants
• func DBOptionsFromConfig(cfg *Config) db.Options
• type Config
  • func MakeConfig(setDefaults bool, configFile string, parseEnv bool) (*Config, error)
  • func MakeConfigReader(setDefaults bool, configYaml io.Reader, parseEnv bool) (*Config, error)
• type HistogramRequest
• type Plugin
  • func Init(config *Config) (*Plugin, error)
  • func InitWithDB(config *Config, d *db.DB) (*Plugin, error)
  • func (p *Plugin) Close()
  • func (p *Plugin) Listen() error
  • func (p *Plugin) RotateCPUProfile() (string, error)
  • func (p *Plugin) Shutdown()
  • func (p *Plugin) StartCPUProfile() error
  • func (p *Plugin) StopCPUProfile() string

Constants

const ClusterNameLabel = "ClusterName"

ClusterNameLabel is the well-known metadata key that ingest writes for every metric row carrying a cluster-name label. It shows up as both a row in the labels set (p.cache.metadata["ClusterName"]) and as a column on the metric sets that filter by cluster. Hoisting it into a constant keeps the histogram handler and any other per-cluster lookup from drifting if we ever rename the pattern output (a quiet rename would otherwise break histograms without any test failure).

Functions

func DBOptionsFromConfig

func DBOptionsFromConfig(cfg *Config) db.Options

DBOptionsFromConfig translates the plugin Config's DB sub-struct into a db.Options. Callers that share a DB handle across packages (see cmdAgiExecService) use this to build the same options without duplicating field-by-field mapping. Mirrors ingest.DBOptionsFromConfig: start from db.DefaultOptions() and override only the explicitly-set numeric fields so the two packages stay in lockstep if db.DefaultOptions() changes.

Numeric fields use 0 as a sentinel for "not set". Bool fields (EnableWAL, SyncWrites) cannot do that — Go bools are tri-stateless — so they ARE operator-authoritative: yaml false beats DefaultOptions true. That mirrors the user expectation ("if I write enableWAL: false, WAL is off") and matches what cmdAgiExecService does explicitly via --no-force-wal when service mode wants to override.

Types

type Config

type Config struct {
	Service struct {
		ListenAddress string `yaml:"listenAddress" default:"127.0.0.1" envconfig:"PLUGIN_LISTEN_ADDR"`
		ListenPort    int    `yaml:"listenPort" default:"8851" envconfig:"PLUGIN_LISTEN_PORT"`
	} `yaml:"service"`
	AddNoneToLabels            []string `yaml:"addNoneToLabels"`
	TimeseriesLegendSeparator  string   `yaml:"timeseriesLegendSeparator" default:" : " envconfig:"PLUGIN_SEPARATOR"`
	TimeseriesDisplayNameFirst bool     `yaml:"timeseriesDisplayNameFirst" default:"false" envconfig:"PLUGIN_DISPLAYNAME_FIRST"` // should the display name come first in the legend
	MaxSeriesPerGraph          int      `yaml:"maxSeriesPerGraph" default:"1000" envconfig:"PLUGIN_MAX_SERIES"`
	MaxDataPointsReceived      int      `yaml:"maxDataPointsReceived" default:"34560000" envconfig:"PLUGIN_MAX_DP_RECV"` // 8640000 is about 1 GiB for concurrent 4 graphs, covering 1000 series in each graph, for a day; default max 4 GiB before reduction
	// Defaults bumped post-Pebble migration: snapshot-isolated reads
	// no longer share an in-memory primary index, so fanning out is
	// almost free. Operators on tiny hosts can still pin these back
	// to 4/4 in plugin.yaml; cmdAgiCreate writes 16/8 (cloud) or
	// 4/4 (Docker) into the deployed yaml on instance creation.
	MaxConcurrentRequests int           `yaml:"maxConcurrentRequests" default:"16" envconfig:"PLUGIN_MAX_REQUESTS"`
	MaxConcurrentJobs     int           `yaml:"maxConcurrentJobs" default:"8" envconfig:"PLUGIN_MAX_JOBS"`
	CacheRefreshInterval  time.Duration `yaml:"cacheRefreshInterval" default:"30s" envconfig:"PLUGIN_CACHE_REFRESH"`
	LabelsSetName         string        `yaml:"labelsSetName" default:"labels" envconfig:"PLUGIN_LABELS_SETNAME"`
	LogLevel              int           `yaml:"logLevel" default:"4" envconfig:"PLUGIN_LOGLEVEL"` // 0=NO_LOGGING 1=CRITICAL, 2=ERROR, 3=WARNING, 4=INFO, 5=DEBUG, 6=DETAIL
	DB                    struct {
		// Path is the on-disk directory Pebble writes to. The default
		// is db.DefaultPath; keep ingest and plugin in lockstep or
		// they will open separate stores and never see each other's
		// data.
		Path string `yaml:"path" default:"/opt/agi/db" envconfig:"PLUGIN_DB_PATH"`
		// CacheBytes/MemTableSizeBytes/MemTableStopWritesThreshold/
		// MaxConcurrentCompactions default to 0 so the unset path
		// falls through to db.DefaultOptions(); operators that
		// previously pinned the legacy 64 MiB / 512 MiB sizes via
		// yaml will keep getting those values (override still wins).
		CacheBytes                  int64  `yaml:"cacheBytes" default:"0" envconfig:"PLUGIN_DB_CACHE_BYTES"`
		MemTableSizeBytes           uint64 `yaml:"memTableSizeBytes" default:"0" envconfig:"PLUGIN_DB_MEMTABLE_BYTES"`
		MemTableStopWritesThreshold int    `yaml:"memTableStopWritesThreshold" default:"0" envconfig:"PLUGIN_DB_MEMTABLE_STOP_THRESHOLD"`
		MaxConcurrentCompactions    int    `yaml:"maxConcurrentCompactions" default:"0" envconfig:"PLUGIN_DB_MAX_COMPACTIONS"`
		MaxOpenFiles                int    `yaml:"maxOpenFiles" default:"0" envconfig:"PLUGIN_DB_MAX_OPEN_FILES"`
		BlockSize                   int    `yaml:"blockSize" default:"0" envconfig:"PLUGIN_DB_BLOCK_SIZE"`
		Compression                 string `yaml:"compression" default:"" envconfig:"PLUGIN_DB_COMPRESSION"`
		// EFS / NFS-shape Pebble tuning knobs. See db.Options docs
		// for full semantics. 0 = leave Pebble's default; for
		// BytesPerSync a negative value (db.BytesPerSyncDisabled)
		// explicitly disables the periodic sync_file_range cadence
		// that becomes a NFS COMMIT round-trip on EFS.
		TargetFileSizeL0      int64         `yaml:"targetFileSizeL0" default:"0" envconfig:"PLUGIN_DB_TARGET_FILE_SIZE_L0"`
		BytesPerSync          int           `yaml:"bytesPerSync" default:"0" envconfig:"PLUGIN_DB_BYTES_PER_SYNC"`
		LBaseMaxBytes         int64         `yaml:"lBaseMaxBytes" default:"0" envconfig:"PLUGIN_DB_LBASE_MAX_BYTES"`
		L0StopWritesThreshold int           `yaml:"l0StopWritesThreshold" default:"0" envconfig:"PLUGIN_DB_L0_STOP_WRITES_THRESHOLD"`
		EnableBloomFilter     bool          `yaml:"enableBloomFilter" default:"false" envconfig:"PLUGIN_DB_ENABLE_BLOOM_FILTER"`
		EnableWAL             bool          `yaml:"enableWAL" default:"false" envconfig:"PLUGIN_DB_ENABLE_WAL"`
		SyncWrites            bool          `yaml:"syncWrites" default:"false" envconfig:"PLUGIN_DB_SYNC_WRITES"`
		ShutdownTimeout       time.Duration `yaml:"shutdownTimeout" default:"60s" envconfig:"PLUGIN_DB_SHUTDOWN_TIMEOUT"`
	} `yaml:"db"`
	TimestampBinName       string `yaml:"timestampBinName" default:"timestamp" envconfig:"PLUGIN_TIMESTAMP_BIN"`
	CPUProfilingOutputFile string `yaml:"cpuProfilingOutputFile" envconfig:"PLUGIN_CPUPROFILE_FILE"`
}

func MakeConfig

func MakeConfig(setDefaults bool, configFile string, parseEnv bool) (*Config, error)

func MakeConfigReader

func MakeConfigReader(setDefaults bool, configYaml io.Reader, parseEnv bool) (*Config, error)

type HistogramRequest

type HistogramRequest struct {
	Range struct {
		From time.Time `json:"from"`
		To   time.Time `json:"to"`
	} `json:"range"`
	Cluster string `json:"cluster"`
	Metric  struct {
		Target string `json:"target"`
		Set    string `json:"set"`
		Name   string `json:"name"`
	} `json:"metric"`
}

type Plugin

type Plugin struct {
	// contains filtered or unexported fields
}

func Init

func Init(config *Config) (*Plugin, error)

Init opens its own embedded db handle and initialises the plugin service. Use InitWithDB instead when plugin needs to share a handle with ingest in the same process — Init's exclusive Pebble lock would otherwise block the co-resident ingest from opening the same directory.

Init also auto-starts the configured CPU profile (if any) as part of finalizeInit. The standalone path owns the whole process, so capturing from t=0 is the right default. The merged-service path (InitWithDB) must NOT auto-start: ingest's own pprof would clash with it (pprof is process-global) and the operator wants plugin samples without ingest noise — see StartCPUProfile.

func InitWithDB

func InitWithDB(config *Config, d *db.DB) (*Plugin, error)

InitWithDB is like Init but uses an externally-owned db handle. The caller retains ownership: Close() on the returned Plugin will NOT close d. This is the entry-point used by the merged agi service (cmdAgiExecService) where ingest and plugin share a single Pebble store opened once at process start.

Unlike Init, InitWithDB does NOT auto-start CPU profiling even when CPUProfilingOutputFile is configured. The merged service runs ingest and plugin in the same process and Go's runtime/pprof CPU profiler is process-global; the orchestrator (cmdAgiExecService) calls StartCPUProfile explicitly once ingest has finished and released the profiler, which gives a clean plugin-only profile.

func (*Plugin) Close

func (p *Plugin) Close()

Close releases the resources owned by this Plugin. It is safe to call multiple times and from concurrent goroutines (e.g. SIGTERM racing the deferred call after Listen returns). Close only closes the underlying db handle when ownsDB=true (i.e. Init opened the handle itself); when the handle was injected via InitWithDB, the caller retains ownership and must close it.

func (*Plugin) Listen

func (p *Plugin) Listen() error

func (*Plugin) RotateCPUProfile

func (p *Plugin) RotateCPUProfile() (string, error)

RotateCPUProfile flushes the in-flight CPU profile to a timestamped sibling of the configured output path, then starts a fresh profile at that configured path. Use this from a SIGUSR1 handler to obtain a complete profile dump from a long-lived plugin without restarting the service: Go's runtime/pprof buffers the entire profile in memory until StopCPUProfile is called, so the only way to get bytes on disk short of process exit is to stop-and-restart.

On entry a profile may or may not already be running. If one was, it is flushed and the file is renamed to "<CPUProfilingOutputFile>.<UTC-stamp>" before a new profile is started at the original path. The "current" file therefore always sits at the configured path and is always 0 bytes (it is the live profile); each rotation produces a complete, parseable pprof file with a timestamp suffix.

Returns:

• rotated: the path of the just-flushed, timestamp-suffixed file. Empty when there was no prior profile to flush (e.g. the very first SIGUSR1 fires before the deferred coordinator goroutine in cmdAgiExecService got around to starting one). A non-empty value is suitable for `go tool pprof <rotated>`.
• err: a non-nil error means re-arming the next profile failed (or, much more rarely, the rename failed). The just-flushed file is still safely on disk in either case.

No-op (returns "", nil) when CPUProfilingOutputFile is unset or the plugin is nil.

func (*Plugin) Shutdown

func (p *Plugin) Shutdown()

Shutdown triggers a graceful shutdown of the plugin's HTTP server. It is safe to call multiple times and from any goroutine (e.g. from a SIGTERM handler in cmdAgiExecService). Shutdown waits for in-flight requests up to Config.DB.ShutdownTimeout before force-closing. Shutdown does NOT close the underlying db handle; call Plugin.Close for that.

func (*Plugin) StartCPUProfile

func (p *Plugin) StartCPUProfile() error

StartCPUProfile starts a runtime/pprof CPU profile and writes samples to the path configured in Config.CPUProfilingOutputFile. It is safe to call multiple times: the second and subsequent calls are no-ops.

Init() calls StartCPUProfile automatically because the standalone plugin owns the whole process. The merged-service path (InitWithDB) deliberately does NOT call it, because Go's CPU profiler is process-global and would clash with ingest's profile. cmdAgiExecService calls this explicitly once ingest has finished, so the resulting profile contains plugin work only.

Returns:

• error: nil on success; nil also when CPUProfilingOutputFile is empty, when the plugin is already shutting down, or when a profile is already running. Returns a wrapped error if the output file cannot be created or pprof.StartCPUProfile rejects it.

func (*Plugin) StopCPUProfile

func (p *Plugin) StopCPUProfile() string

StopCPUProfile stops the in-flight CPU profile (if any) and closes the output file, but does NOT tear down the rest of the plugin. After it returns, the file at the returned path is a complete, parseable pprof profile suitable for `go tool pprof`. Safe to call when no profile is running (no-op, returns ""). Safe to call concurrently with StartCPUProfile / RotateCPUProfile / Close — all four serialise on p.pprofMu.

Returns:

• string: the on-disk path of the file that was just flushed, or "" when there was nothing running to flush.

Note: StartCPUProfile creates the output file with O_TRUNC, so the file at this path will be re-truncated to 0 bytes the moment a subsequent StartCPUProfile runs. Callers that want to keep the flushed bytes around must rename the file before re-arming — RotateCPUProfile does exactly that.