worker

Documentation

Index

• func DefaultConfigPath() (string, error)
• type Config
  • func DefaultConfig() Config
  • func LoadConfig(path string) (Config, error)
  • func (c Config) Validate() error
• type FirecrackerConfig
• type ImageConfig
  • func (i ImageConfig) IdleTimeoutDur() (time.Duration, error)
• type PoolConfig
• type RuntimeConfig
• type SchedulerLink
• type TLSConfig
• type VMConfig
  • func (v VMConfig) IdleTimeoutDur() (time.Duration, error)
  • func (v VMConfig) MemoryBytes() int64
  • func (v VMConfig) SessionTimeoutDur() (time.Duration, error)
• type Worker
  • func NewDefaultWorker() *Worker
  • func NewWorker(cfg Config) (*Worker, error)
  • func (w *Worker) BeginCompile()
  • func (w *Worker) Compile(ctx context.Context, req *gen.CompileRequest) (*gen.CompileResponse, error)
  • func (w *Worker) EndCompile()
  • func (w *Worker) Run(ctx context.Context) error
  • func (w *Worker) SchedulerSigningKey() []byte
  • func (w *Worker) ValidateToken(req *gen.CompileRequest) error

Functions

func DefaultConfigPath

func DefaultConfigPath() (string, error)

DefaultConfigPath returns ~/.config/hpcc/worker.toml on Unix and the platform equivalent elsewhere.

Types

type Config

type Config struct {
	Listen     string        `toml:"listen"`      // gRPC listen addr for incoming Compile RPCs
	WorkerID   string        `toml:"worker_id"`   // empty → auto-generate at startup
	PublicAddr string        `toml:"public_addr"` // address advertised to the scheduler; clients dial this
	Paranoid   bool          `toml:"paranoid"`    // mirror of scheduler-side paranoid mode (§4.13)
	TLS        TLSConfig     `toml:"tls"`
	Scheduler  SchedulerLink `toml:"scheduler"`
	Runtime    RuntimeConfig `toml:"runtime"`
	VM         VMConfig      `toml:"vm"`
	Pool       PoolConfig    `toml:"pool"`
	Image      ImageConfig   `toml:"image"`

	// Caches is the worker-side cache backends. In paranoid mode (§4.13)
	// the worker is the only process that reads/writes the cache; in the
	// default mode this can be empty and clients carry their own caches.
	// Schema is shared with the client config (config.CacheConfig).
	Caches []config.CacheConfig `toml:"cache"`
}

func DefaultConfig

func DefaultConfig() Config

func LoadConfig

func LoadConfig(path string) (Config, error)

func (Config) Validate

func (c Config) Validate() error

type FirecrackerConfig

type FirecrackerConfig struct {
	FirecrackerBin string `toml:"firecracker_bin"`
	JailerBin      string `toml:"jailer_bin"`
	KernelImage    string `toml:"kernel_image"`
	RootfsDir      string `toml:"rootfs_dir"`
	RunDir         string `toml:"run_dir"`
	UID            int    `toml:"uid"`
	GID            int    `toml:"gid"`
	BootArgs       string `toml:"boot_args"`
}

FirecrackerConfig is the raw-Firecracker runtime's host-side knobs. Unused unless runtime.handler == "firecracker".

FirecrackerBin and JailerBin are the host paths to the upstream firecracker and jailer executables. KernelImage is the vmlinux that every microVM boots — hpcc owns the kernel, not the user image (plan §4.3). RootfsDir mirrors the rootfs.Store CacheDir so the runtime can resolve a prepared "<algo>-<hex>.ext4" by digest. RunDir is jailer's --chroot-base-dir; one chroot per VM lands underneath as <RunDir>/firecracker/<vm-id>/root/. UID/GID are the non-root credentials jailer drops to before exec'ing firecracker. BootArgs overrides the default kernel cmdline (sane defaults boot the rootfs read-only with the in-VM hpcc-agent as PID 1).

type ImageConfig

type ImageConfig struct {
	PauseLinuxAmd64   string   `toml:"pause_linux_amd64"`
	PauseLinuxArm64   string   `toml:"pause_linux_arm64"`
	PauseWindowsAmd64 string   `toml:"pause_windows_amd64"`
	AdvertisedDigests []string `toml:"advertised_digests"`
	IdleTimeout       string   `toml:"idle_timeout"` // e.g. "24h"; empty disables eviction
}

ImageConfig points at pre-built pause binaries on disk. Empty paths fall back to the binaries embedded in the worker (built from /pause).

AdvertisedDigests is a static list of image digests the worker reports in RegisterWorker / Heartbeat in addition to whatever the real ImageStore knows about. Useful when the runtime is the dev-mode "really_really_dangerous" handler (no containerd, no image store) or when an operator wants to manually pin which toolchains this worker accepts. AdvertisedDigests are exempt from idle-image eviction.

IdleTimeout is the maximum time an image entry may sit in the local catalogue without being used by a Compile before the eviction loop drops it (untagging the prepared image in the backing image store — containerd on Windows, hpcc's rootfs cache on Linux — and freeing its blobs at the next GC). Empty disables eviction.

func (ImageConfig) IdleTimeoutDur

func (i ImageConfig) IdleTimeoutDur() (time.Duration, error)

IdleTimeoutDur parses Image.IdleTimeout. An empty string returns 0 (eviction disabled).

type PoolConfig

type PoolConfig struct {
	MaxActive int `toml:"max_active"` // upper bound on concurrent per-tenant VMs
}

type RuntimeConfig

type RuntimeConfig struct {
	// Handler selects the worker runtime backend.
	//   "firecracker"             — raw Firecracker driver (Linux).
	//   "runhcs-wcow-hypervisor"  — containerd + hcsshim Hyper-V (Windows). Not implemented yet.
	//   "really_really_dangerous" — host-exec; dev only.
	Handler string `toml:"handler"`

	Firecracker FirecrackerConfig `toml:"firecracker"`
}

type SchedulerLink

type SchedulerLink struct {
	URL         string `toml:"url"`          // e.g. "scheduler.internal:9091"
	WorkerToken string `toml:"worker_token"` // shared static token; matches scheduler.auth.worker_token
	CAFile      string `toml:"ca_file"`      // optional: pin the scheduler's CA cert
}

type TLSConfig

type TLSConfig struct {
	CertFile string `toml:"cert_file"`
	KeyFile  string `toml:"key_file"`
}

type VMConfig

type VMConfig struct {
	Memory         string `toml:"memory"` // e.g. "2GB"
	VCPUs          int32  `toml:"vcpus"`
	IdleTimeout    string `toml:"idle_timeout"`    // e.g. "10m"
	SessionTimeout string `toml:"session_timeout"` // e.g. "8h"
}

func (VMConfig) IdleTimeoutDur

func (v VMConfig) IdleTimeoutDur() (time.Duration, error)

func (VMConfig) MemoryBytes

func (v VMConfig) MemoryBytes() int64

func (VMConfig) SessionTimeoutDur

func (v VMConfig) SessionTimeoutDur() (time.Duration, error)

type Worker

type Worker struct {
	Config Config
	// Images is the local image catalog: digest → *imageEntry. The
	// presence of an entry means "the prepared image for this digest
	// is locally available." Pulls add entries on success; advertised
	// and previously-prepared digests pre-populate at bootstrap.
	Images sync.Map
	// ImageStore prepares per-tenant images for the runtime. nil means
	// "no image store" — the dev-mode dangerous runtime takes that
	// path: ensureImage records every digest as locally-present
	// without I/O, and the eviction loop only drops catalogue
	// entries. Real deployments wire either a cdimage.Store
	// (containerd, Windows under hcsshim) or a rootfs.Store
	// (Linux under raw Firecracker).
	ImageStore image.Store
	Containers sync.Map // containerID -> runtime.Container

	gen.UnimplementedWorkerServiceServer
	// contains filtered or unexported fields
}

func NewDefaultWorker

func NewDefaultWorker() *Worker

func NewWorker

func NewWorker(cfg Config) (*Worker, error)

func (*Worker) BeginCompile

func (w *Worker) BeginCompile()

BeginCompile / EndCompile are the hooks the Compile RPC handler uses to keep `inflight` in sync; defer EndCompile right after BeginCompile to avoid drift on early returns.

func (*Worker) Compile

func (w *Worker) Compile(ctx context.Context, req *gen.CompileRequest) (*gen.CompileResponse, error)

func (*Worker) EndCompile

func (w *Worker) EndCompile()

func (*Worker) Run

func (w *Worker) Run(ctx context.Context) error

Run drives the worker's scheduler-side loop: dial the scheduler, authenticate, register, then heartbeat on a ticker until ctx is cancelled. On RPC failures it re-authenticates rather than retrying the same dead session.

func (*Worker) SchedulerSigningKey

func (w *Worker) SchedulerSigningKey() []byte

SchedulerSigningKey returns the scheduler's task-JWT signing pubkey learned during authentication. Compile RPCs use it to verify the JWT the client presents in gRPC metadata. Returns nil before first auth.

func (*Worker) ValidateToken

func (w *Worker) ValidateToken(req *gen.CompileRequest) error