placement

Documentation

Overview

Package placement is STABLE — deterministic filter, rank, and select for task placement. It is part of the stable operator path.

Index

• func ComputeFitScore(n models.NodeFacts, isLocal bool, st *state.ClusterState) int
• func ComputeTaskFitScore(n models.NodeFacts, isLocal bool, st *state.ClusterState, ...) int
• func ExplainPlacement(reqs models.TaskRequirements, nodes []models.NodeFacts, st *state.ClusterState) models.PlacementExplanation
• func ExtractModelName(description string) string
• func FilterCandidates(reqs models.TaskRequirements, nodes []models.NodeFacts, st *state.ClusterState) []models.NodeFacts
• func Headroom(n models.NodeFacts, nodes []models.NodeFacts, reqs models.TaskRequirements) int64
• func InferRequirements(desc string, opts ...workload.InferRequirementsOptions) models.TaskRequirements
• func MinFreeRAMForNode(reqs models.TaskRequirements, n models.NodeFacts) int64
• func ObservationScopeForRequirements(node string, reqs models.TaskRequirements, selectedTool string) models.ObservationScope
• func RankCandidates(candidates []models.NodeFacts, reqs models.TaskRequirements, ...) []models.NodeFacts
• func SelectBestNode(reqs models.TaskRequirements, nodes []models.NodeFacts, st *state.ClusterState) models.PlacementDecision

Functions

func ComputeFitScore

func ComputeFitScore(n models.NodeFacts, isLocal bool, st *state.ClusterState) int

ComputeFitScore returns 0-100 indicating small-model suitability. Scoring breakdown:

• Allocatable RAM: up to 30 pts (1 pt per 256MB, capped at 30)
• Pressure: up to 25 pts (none=25, low=20, medium=10, high=0)
• GPU present: +25 pts
• CPU cores: up to 10 pts (1 pt per core, capped at 10)
• Local node: +10 pts (no SSH hop = lower latency)
• TurboQuant-capable long-context backend: +15..25 pts for long-context asks
• Unified-memory topology bonus: +8..16 pts for mlx/long-context asks

Max: capped at 100

func ComputeTaskFitScore

func ComputeTaskFitScore(n models.NodeFacts, isLocal bool, st *state.ClusterState, reqs models.TaskRequirements) int

ComputeTaskFitScore returns 0-100 indicating task-specific placement fit.

func ExplainPlacement

func ExplainPlacement(reqs models.TaskRequirements, nodes []models.NodeFacts, st *state.ClusterState) models.PlacementExplanation

func ExtractModelName

func ExtractModelName(description string) string

ExtractModelName attempts to extract an inference model name from a task description or command string. Returns the first match found, or "" if no model name is identifiable.

Priority order:

1. Explicit flag forms: --model=X, -m=X, --model X, -m X
2. Ollama subcommand forms: ollama run X, ollama pull X
3. Bare model-tag heuristic: first token matching word:tag form

The function is intentionally conservative — it returns "" rather than guessing when the description is ambiguous prose without a recognisable model reference.

func FilterCandidates

func FilterCandidates(reqs models.TaskRequirements, nodes []models.NodeFacts, st *state.ClusterState) []models.NodeFacts

FilterCandidates returns nodes that meet all task requirements. Rules (all must pass):

• Status must be complete
• If MinFreeRAMMB > 0, node must have resources with enough free RAM
• If RequiredTools are set, node must satisfy all of them

func Headroom

func Headroom(n models.NodeFacts, nodes []models.NodeFacts, reqs models.TaskRequirements) int64

Headroom computes the effective free RAM headroom for a node after subtracting the task's minimum requirement and a cluster-pressure penalty. Returns -1 if the node cannot meet the minimum. Used by execution and explain surfaces.

func InferRequirements

func InferRequirements(desc string, opts ...workload.InferRequirementsOptions) models.TaskRequirements

InferRequirements derives TaskRequirements from a task description string. It delegates to the internal/workload package for structured profile matching.

An optional InferRequirementsOptions may be provided to inject a semantic Classifier (e.g. llmrouter.Engine). All existing call-sites that pass no options continue to use the legacy string-matcher path unchanged.

func MinFreeRAMForNode

func MinFreeRAMForNode(reqs models.TaskRequirements, n models.NodeFacts) int64

MinFreeRAMForNode exposes the effective placement floor used for a node. Guarded execution reuses this to keep placement and last-second safety checks aligned.

func ObservationScopeForRequirements

func ObservationScopeForRequirements(node string, reqs models.TaskRequirements, selectedTool string) models.ObservationScope

ObservationScopeForRequirements normalizes the exact empirical scope used by guarded execution recording and placement lookup.

ModelName is populated by extracting a model name from reqs.Description only for inference-related workload classes. Non-inference workloads leave ModelName empty — preserving backward compatibility and avoiding false positives from flags like -m in git commit messages.

func RankCandidates

func RankCandidates(candidates []models.NodeFacts, reqs models.TaskRequirements, st *state.ClusterState) []models.NodeFacts

RankCandidates sorts nodes deterministically. Priority order:

1. Highest allocatable RAM
2. Best exact-scope empirical observation (fresh only)
3. Resident model locality for the requested runtime
4. Preferred backend rank
5. GPU score
6. Highest effective headroom (free-with-state - requirement)
7. Highest unified-memory suitability / TurboQuant for matching asks
8. Lowest RAM pressure (soft tie-break after hard blockers)
9. Lowest reservation ratio and cluster reservation share

10. Node name ascending (stable tiebreak)

func SelectBestNode

func SelectBestNode(reqs models.TaskRequirements, nodes []models.NodeFacts, st *state.ClusterState) models.PlacementDecision

SelectBestNode runs the full placement pipeline: filter → rank → select. Reasoning is diagnostic: on failure it explains why each node was excluded; on success it explains fit score, locality, and runner-up comparison.