hardware

Documentation

Overview

Package hardware probes the local machine for the resources that decide which models it can run: the GPU and its memory, and the system RAM a CPU-only run draws on. It is a best-effort diagnostic, not part of a governed run: detection shells out to a vendor tool or reads a well-known OS source when present and reports nothing rather than guessing when it is absent, so a caller can fall back to an explicit budget. The parsing is pure and tested; only the probe itself touches the machine.

Index

• type Box
  • func Detect(ctx context.Context) Box
  • func (b Box) HasGPU() bool
  • func (b Box) HasRAM() bool
  • func (b Box) SupportsNVFP4() bool
• type ContainerSupport
  • func (c ContainerSupport) Available() bool
  • func (c ContainerSupport) GPUPassthrough() bool

Types

type Box

type Box struct {
	// GPUName is the detected accelerator's name, empty if none was found.
	GPUName string
	// VRAMBytes is the detected GPU memory in bytes, 0 if none was found.
	VRAMBytes int64
	// RAMBytes is the total system memory in bytes, 0 if it could not be read. A
	// CPU-only run is bounded by this, not by VRAM, so it decides fit on the common
	// machine that has no usable GPU.
	RAMBytes int64
	// ComputeCapability is the GPU's CUDA compute capability ("8.9", "12.0"), empty when
	// no NVIDIA GPU was detected. It is the precise signal for what a quantized format
	// needs: the 4-bit floating-point path is only available from Blackwell upward.
	ComputeCapability string
	// GPUArch is the architecture family derived from the compute capability ("Ada
	// Lovelace", "Hopper", "Blackwell"), empty when unknown. It is the human-facing label
	// and the key the runtime auto-selection reads.
	GPUArch string
	// DriverVersion is the NVIDIA driver version string ("550.54.14"), empty when unknown.
	// It bounds which CUDA runtime a container can use.
	DriverVersion string
	// CUDAVersion is the maximum CUDA version the installed driver supports ("12.4"),
	// empty when unknown. A runtime image built against a newer CUDA than this will not
	// run, so it gates which runtime build is viable.
	CUDAVersion string
	// Containers is what the machine can run a containerized runtime with: which OCI
	// runtimes are present and whether GPU passthrough is wired. A GPU runtime shipped as
	// an image needs this; without it that path is refused in favor of a native binary.
	Containers ContainerSupport
}

Box is what was detected about the machine. A zero field means "not detected", so a caller treats it as unknown rather than zero capacity.

func Detect

func Detect(ctx context.Context) Box

Detect probes the machine. It is best-effort: an absent or failing probe leaves the corresponding field zero, never an error, so callers degrade to an explicit budget instead of failing. The two probes are independent: a machine with no GPU still reports its RAM, which is what a CPU-only run is judged against. The context bounds each probe so a wedged tool cannot hang the caller.

func (Box) HasGPU

func (b Box) HasGPU() bool

HasGPU reports whether a GPU with known memory was detected.

func (Box) HasRAM

func (b Box) HasRAM() bool

HasRAM reports whether the system memory total was read.

func (Box) SupportsNVFP4

func (b Box) SupportsNVFP4() bool

SupportsNVFP4 reports whether the GPU can serve the 4-bit floating-point format, which is a Blackwell-and-up capability (compute capability major 10 or above). A model in that format is refused on an older GPU rather than served on a path it cannot run.

type ContainerSupport

type ContainerSupport struct {
	// Docker is true when a usable docker client was found.
	Docker bool
	// Podman is true when a usable podman client was found.
	Podman bool
	// NVIDIAToolkit is true when the NVIDIA Container Toolkit was found, the component
	// that lets a container see the host GPU. Without it a container runs CPU-only.
	NVIDIAToolkit bool
}

ContainerSupport is the OCI tooling detected on the machine, which decides whether a runtime shipped as a container image can be run, and whether it can reach the GPU.

func (ContainerSupport) Available

func (c ContainerSupport) Available() bool

Available reports whether any OCI runtime was found to run an image with.

func (ContainerSupport) GPUPassthrough

func (c ContainerSupport) GPUPassthrough() bool

GPUPassthrough reports whether a container can be given the host GPU: an OCI runtime plus the NVIDIA toolkit. A GPU runtime image needs both, so this gates that path.