README
¶
NVIDIA GPU Health Agent
Lightweight GPU health monitoring and reporting agent for NVIDIA GPU infrastructure building on top of leptonai/gpud
Overview
What It Monitors:
- GPU Metrics: Power, temperature, clocks, utilization, memory, Xid events
- System Metrics: CPU, memory, disk, network usage
- Infrastructure: NVIDIA drivers, CUDA runtime, InfiniBand, containers
Export Formats:
- HTTP API Server: Serves data via REST endpoints (JSON) and Prometheus metrics (
/metrics) - File Export (Offline Mode): Writes data to local files in CSV or JSON format
- Remote Export: Sends telemetry data to OpenTelemetry-compatible endpoints via OTLP over HTTP
Key Features:
- Lightweight: <100MB RAM, <1% CPU usage
- Non-intrusive: Read-only operations, no system modifications
- Production-ready: 24/7 datacenter operation
Documentation
- Installation - Installation, updating, and uninstalling
- Usage - Commands, HTTP API, integration, and troubleshooting
- Configuration - Environment variables and service configuration
- Development - Building from source and contributing
Contributing
See CONTRIBUTING.md for development setup and guidelines.
Related: leptonai/gpud (upstream dependency)
License
Apache License 2.0 - see LICENSE for details.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
gpuhealth
command
|
|
|
internal
|
|
|
exporter
Package healthexporter provides functionality to export health data from local SQLite to a global health endpoint for centralized monitoring and long-term storage using OTLP format.
|
Package healthexporter provides functionality to export health data from local SQLite to a global health endpoint for centralized monitoring and long-term storage using OTLP format. |
|
exporter/collector
Package collector handles health data collection from various sources
|
Package collector handles health data collection from various sources |
|
exporter/converter
Package converter handles conversion of health data to different formats
|
Package converter handles conversion of health data to different formats |
|
exporter/writer
Package writer handles writing health data to various outputs
|
Package writer handles writing health data to various outputs |
|
machineinfo
Package machineinfo provides a shim layer over gpud's machine-info package to customize version information for GPU Health.
|
Package machineinfo provides a shim layer over gpud's machine-info package to customize version information for GPU Health. |
|
server
Package healthserver provides a simplified HTTP server for GPU health metrics export.
|
Package healthserver provides a simplified HTTP server for GPU health metrics export. |
Click to show internal directories.
Click to hide internal directories.