accel

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 7, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package accel defines the acceleration runtime surface for Insyra.

It exposes configuration, sessions, device metadata, typed datasets, execution reports, and the small set of measured backend operations.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrShaderCompile   = errors.New("accel: shader compilation failed")
	ErrBufferTooLarge  = errors.New("accel: column exceeds device buffer limit")
	ErrReadbackTimeout = errors.New("accel: device readback timed out")
)

Sentinel errors a backend wraps so the runtime can map a failure onto a stable fallback reason without knowing anything about the backend.

View Source 
var ErrNativeProbeUnavailable = errors.New("accel: native probe unavailable")
View Source 
var ErrSDKProbeUnavailable = errors.New("accel: sdk probe unavailable")

ErrSDKProbeUnavailable signals that an SDK-backed probe could not run on this host (driver missing, library not loaded, unsupported platform, etc.). Discoverers treat this as a clean miss and fall back to native command probes followed by env stubs.

Functions

func DeviceMatMul 

func DeviceMatMul(a []float32, aRows, aCols int, b []float32, bRows, bCols int) ([]float32, error)

DeviceMatMul computes one 2-D float32 matrix product on the discovered device. A device failure is reported through Default and returned as an error so the caller can preserve its exact CPU fallback.

func NearestExactCPU 

func NearestExactCPU(dataset *Dataset, queries [][]float64, m int) ([]uint32, []float64, int, error)

NearestExactCPU is the reference: every distance, every query point, float64 throughout. It is what the accelerated path is asserted equal to.

func RegisterBackendExecutor 

func RegisterBackendExecutor(backend Backend, executor BackendExecutor) error

func RegisterDiscoverer 

func RegisterDiscoverer(d Discoverer)

func RegisterSDKProbe 

func RegisterSDKProbe(probe SDKProbe)

RegisterSDKProbe wires an SDK probe into the discoverer pipeline for its backend. Multiple probes per backend are tried in registration order; the first one that returns devices wins. The pipeline ordering is: SDK > native command > env stub.

func ResetBackendExecutorsForTest 

func ResetBackendExecutorsForTest()

ResetBackendExecutorsForTest clears every registered executor. Tests use this to keep one package's registration from leaking into another's.

func ResetDefaultForTest 

func ResetDefaultForTest()

ResetDefaultForTest drops the shared session so the next Default call rediscovers. Tests use this the way they use ResetDiscoverersForTest.

func ResetDiscoverersForTest 

func ResetDiscoverersForTest()

func ResetSDKProbesForTest 

func ResetSDKProbesForTest()

ResetSDKProbesForTest clears every registered SDK probe. Tests use this to isolate themselves from probes that may have been registered by init().

Types

type Backend 

type Backend string
const (
	BackendUnknown Backend = "unknown"
	BackendCPU     Backend = "cpu"
	BackendCUDA    Backend = "cuda"
	BackendMetal   Backend = "metal"
	BackendWebGPU  Backend = "webgpu"
)

type BackendExecutor 

type BackendExecutor interface {
	Name() string
	Execute(ctx context.Context, req ExecuteRequest) (ExecuteResponse, error)
}

BackendExecutor runs an operation on a real device. Registering one is how a backend module opts into the accel runtime.

type Buffer 

type Buffer struct {
	Name          string
	Type          DataType
	Values        any
	Nulls         []bool
	Validity      []byte
	StringOffsets []uint32
	StringData    []byte
	Len           int
}

type CacheDeviceUsage 

type CacheDeviceUsage struct {
	DeviceID        string
	ResidentBuffers int
	ResidentBytes   uint64
	BudgetBytes     uint64
}

type CacheEntry 

type CacheEntry struct {
	Key                 string
	DatasetName         string
	DatasetID           string
	Lineage             string
	BufferName          string
	Type                DataType
	Len                 int
	ResidentBytes       uint64
	DeviceIDs           []string
	DeviceResidentBytes map[string]uint64
	LastAccess          time.Time
	// contains filtered or unexported fields
}

type CacheSnapshot 

type CacheSnapshot struct {
	ResidentBuffers int
	ResidentBytes   uint64
	BudgetBytes     uint64
	EvictedBuffers  uint64
	EvictedBytes    uint64
	DeviceUsage     []CacheDeviceUsage
	Entries         []CacheEntry
}

type Config 

type Config struct {
	Mode              Mode
	ShardStrategy     ShardStrategy
	PreferredBackends []Backend
	MemoryBudget      MemoryBudgetPolicy
	Strict            bool
	EnableFallback    bool
	// Devices is a hard per-session allowlist. Entries may be stable device IDs
	// or zero-based discovery indices. An empty list allows every discovered
	// device that is not removed by INSYRA_ACCEL_DEVICES.
	Devices           []string
	PreferredDevices  []string
	ReportHistorySize int
	DiscoveryTimeout  time.Duration
}

func DefaultConfig 

func DefaultConfig() Config

type DataType 

type DataType string
const (
	DataTypeUnknown DataType = "unknown"
	DataTypeBool    DataType = "bool"
	DataTypeInt64   DataType = "int64"
	DataTypeFloat64 DataType = "float64"
	DataTypeString  DataType = "string"
	DataTypeAny     DataType = "any"
)

type Dataset 

type Dataset struct {
	Name        string
	Fingerprint string
	Lineage     string
	Rows        int
	Buffers     []Buffer
}

type Device 

type Device struct {
	ID                string
	Name              string
	Vendor            string
	Backend           Backend
	ProbeSource       ProbeSource
	Type              DeviceType
	MemoryClass       MemoryClass
	SharedMemory      bool
	BudgetBytes       uint64
	Score             float64
	CapabilitySummary map[string]bool
	DriverVersion     string
	ComputeCapability string
	PCIBusID          string
}

type DeviceType 

type DeviceType string
const (
	DeviceTypeUnknown    DeviceType = "unknown"
	DeviceTypeCPU        DeviceType = "cpu"
	DeviceTypeIntegrated DeviceType = "integrated"
	DeviceTypeDiscrete   DeviceType = "discrete"
	DeviceTypeVirtual    DeviceType = "virtual"
)

type Discoverer 

type Discoverer interface {
	Name() string
	Discover(cfg Config) ([]Device, error)
}

type ExactNearestResult 

type ExactNearestResult struct {
	ExecutionResult
	// Index and Distance are row-major and hold M entries per row, nearest
	// first: entry r*M+j is row r's j-th nearest query point.
	Index    []uint32
	Distance []float64
	Rows     int
	Queries  int
	M        int
	// Rechecked counts the rows whose device shortlist could not be trusted and
	// were recomputed against every query point. It is the number to watch on
	// real data: a shortlist too narrow for the data shows up here as a rising
	// count long before it shows up as a slowdown.
	Rechecked int
}

ExactNearestResult carries the nearest query points per row, computed to the same answer a pure float64 pass would give.

type ExecuteColumn 

type ExecuteColumn struct {
	Name   string
	Values []float32
}

ExecuteColumn is one projected column, already narrowed to float32 and with nulls replaced by the operation's identity, so a backend never has to know about precision policy or Insyra's null representation.

type ExecuteRequest 

type ExecuteRequest struct {
	Op        Op
	Device    Device
	Columns   []ExecuteColumn
	Precision Precision
	// Queries holds one point per entry, each carrying one value per column in
	// Columns order. Only OpSquaredDistance reads it.
	Queries [][]float32
	// Shortlist is how many candidates per row OpNearestShortlist keeps. Other
	// operations ignore it.
	Shortlist int
}

ExecuteRequest is one operation over one dataset on one device. Columns keep their dataset order, because a kernel that reads across columns cares about position in a way a map would lose.

type ExecuteResponse 

type ExecuteResponse struct {
	Reductions map[string]float64
	// Distances is query-major for OpSquaredDistance: entry q*rows+r is the
	// distance from row r to query q. For OpNearestQuery it holds one distance
	// per row, the smallest one.
	Distances []float32
	// NearestIndex holds the closest query point per row. Only OpNearestQuery
	// fills it.
	NearestIndex []uint32
	// ShortlistIndex and ShortlistDistance are row-major, holding Shortlist
	// entries per row: entry r*Shortlist+j is row r's j-th nearest.
	// ShortlistBoundary holds one value per row, the distance of the best
	// candidate that did not make the list. Only OpNearestShortlist fills them.
	ShortlistIndex    []uint32
	ShortlistDistance []float32
	ShortlistBoundary []float32

	Transfer      time.Duration
	Dispatch      time.Duration
	Readback      time.Duration
	BytesUploaded uint64
}

ExecuteResponse carries the computed results and what the submission cost. The durations describe the whole submission rather than any one column — transfer, dispatch and readback are properties of the submission, and a per-column split would be invented. They are host-observed: Metal and GLES do not implement GPU timestamp queries.

type ExecutionResult 

type ExecutionResult struct {
	Accelerated    bool
	FallbackReason FallbackReason
	MergePolicy    MergePolicy
	Executor       string
	ExecutorKind   ExecutorKind
	Assignments    []ShardAssignment
	DeviceIDs      []string
	// Chunks is the number of sequential device submissions used by an
	// execution. It is zero when no device submission ran.
	Chunks int

	// Op and Precision describe what actually ran, not what was asked for.
	Op        Op
	Precision Precision

	// Reductions holds one value per buffer, keyed by buffer name. It is only
	// populated when Accelerated is true.
	Reductions map[string]float64
	// Counts holds the number of non-null values folded into each reduction.
	Counts map[string]int

	// Measured cost. These are host-observed durations: Metal and GLES return
	// ErrTimestampsNotSupported for GPU timestamp queries, so only Vulkan and
	// DX12 could report device-side timing. Zero when nothing ran on a device.
	Transfer      time.Duration
	Dispatch      time.Duration
	Readback      time.Duration
	BytesUploaded uint64
}

type ExecutorKind 

type ExecutorKind string
const (
	ExecutorKindUnknown    ExecutorKind = "unknown"
	ExecutorKindNone       ExecutorKind = "none"
	ExecutorKindRegistered ExecutorKind = "registered"
)

type FallbackReason 

type FallbackReason string
const (
	FallbackReasonNone                  FallbackReason = "none"
	FallbackReasonNoAccelerator         FallbackReason = "no-accelerator"
	FallbackReasonCPUOnly               FallbackReason = "cpu-only-mode"
	FallbackReasonDiscoveryError        FallbackReason = "discovery-error"
	FallbackReasonStrictGPUUnavailable  FallbackReason = "strict-gpu-unavailable"
	FallbackReasonWorkloadUnsupported   FallbackReason = "workload-unsupported"
	FallbackReasonWorkloadNotProfitable FallbackReason = "workload-not-profitable"
	FallbackReasonNoBackendExecutor     FallbackReason = "no-backend-executor"
	FallbackReasonDeviceSelectionEmpty  FallbackReason = "device-selection-empty"
	FallbackReasonPrecisionNotAccepted  FallbackReason = "precision-not-accepted"
	FallbackReasonDTypeNotEligible      FallbackReason = "dtype-not-eligible"
	FallbackReasonShaderCompileFailed   FallbackReason = "shader-compile-failed"
	FallbackReasonBufferTooLarge        FallbackReason = "buffer-too-large"
	FallbackReasonReadbackTimeout       FallbackReason = "readback-timeout"
	FallbackReasonExecutionFailed       FallbackReason = "execution-failed"
)

type MemoryBudgetPolicy 

type MemoryBudgetPolicy struct {
	DeviceFraction float64
	SharedFraction float64
}

type MemoryClass 

type MemoryClass string
const (
	MemoryClassUnknown MemoryClass = "unknown"
	MemoryClassShared  MemoryClass = "shared"
	MemoryClassDevice  MemoryClass = "device-local"
)

type MergePolicy 

type MergePolicy string
const (
	MergePolicyUnknown       MergePolicy = "unknown"
	MergePolicyCPU           MergePolicy = "cpu"
	MergePolicyBackendNative MergePolicy = "backend-native"
)

type Mode 

type Mode string
const (
	ModeAuto      Mode = "auto"
	ModeCPU       Mode = "cpu"
	ModeGPU       Mode = "gpu"
	ModeStrictGPU Mode = "strict-gpu"
)

type Op 

type Op string

Op names the operation a backend is asked to perform. The runtime ships one operation; adding a second is a spec change, not a signature change. 

const (
	OpUnknown Op = "unknown"
	// OpNearestShortlist returns the several nearest query points per row rather
	// than only the nearest, plus the distance of the best rejected one. It is
	// how an exact float64 answer is reached through an f32 device: the device
	// narrows the field, the host settles the ranking.
	//
	// It is the only device operation. Three others existed and were removed once
	// measured: a column sum at 0.7x, a distance matrix whose readback grew with
	// the answer, and a single-precision nearest query no float64 caller could
	// use. Nothing is added back without a measurement against a host using every
	// core it has.
	OpNearestShortlist Op = "nearest-shortlist"
)

type Precision 

type Precision string

Precision states what the caller will accept from device execution. The default refuses anything the device cannot compute at the column's own precision, because narrowing a column silently would change the numbers a data-analysis library returns. WGSL has no f64 and Apple GPUs have no double-precision hardware, so float64 columns need an explicit opt-in. 

const (
	PrecisionExact   Precision = "exact"
	PrecisionFloat32 Precision = "float32"
)

type ProbeSource 

type ProbeSource string
const (
	ProbeSourceUnknown ProbeSource = "unknown"
	ProbeSourceSDK     ProbeSource = "sdk"
	ProbeSourceNative  ProbeSource = "native"
	ProbeSourceEnvStub ProbeSource = "env-stub"
)

type Report 

type Report struct {
	Mode                     Mode
	Accelerated              bool
	SelectedBackend          Backend
	DiscoveredDeviceIDs      []string
	SelectedDeviceIDs        []string
	SelectedDevices          []string
	UnmatchedDeviceSelectors []UnmatchedDeviceSelector
	FallbackReason           FallbackReason
	StartedAt                time.Time
	FinishedAt               time.Time
	GeneratedAt              time.Time
	Metrics                  map[string]float64
}

func (Report) Duration 

func (r Report) Duration() time.Duration

type SDKProbe 

type SDKProbe interface {
	Name() string
	Backend() Backend
	Probe(cfg Config) ([]Device, error)
}

SDKProbe is a backend-specific probe that talks directly to the vendor SDK (NVML for CUDA, Metal API for Apple, wgpu-native for WebGPU, …) instead of shelling out to a host command. Probes that succeed should set Device.ProbeSource to ProbeSourceSDK; the discoverer normalizes the field if the probe leaves it blank.

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session is safe for concurrent use. mu guards every field below it; methods suffixed Locked assume the caller already holds it, which is how the public methods call one another without re-entering a non-reentrant mutex.

func Default 

func Default() *Session

Default returns the session shared by this process, creating it on first use.

Construction is lazy on purpose: importing accel must not open a GPU device, because most programs reach this package transitively through allpkgs without ever asking for acceleration.

A host with no usable device still gets a session. Discovery failure is reported through the session's report and fallback reason, which is the runtime's normal way of saying "no acceleration here" — returning nil would push a nil check into every call site to express the same thing.

The returned session is shared, so Close on it does nothing. Callers that need their own lifetime should use Open.

func NewSession 

func NewSession(cfgs ...Config) *Session

func Open 

func Open(cfg Config) (*Session, error)

func (*Session) CacheSnapshot 

func (s *Session) CacheSnapshot() CacheSnapshot

func (*Session) Close 

func (s *Session) Close() error

Close releases the session. On the process-shared session from Default it is a no-op: library code holding it cannot know it is shared, so failing would turn a reasonable defensive call into a spurious error, and actually closing would let one caller disable acceleration for the whole process.

func (*Session) Closed 

func (s *Session) Closed() bool

func (*Session) Config 

func (s *Session) Config() Config

func (*Session) Devices 

func (s *Session) Devices() []Device

func (*Session) Discover 

func (s *Session) Discover() error

func (*Session) ExecuteNearestExact 

func (s *Session) ExecuteNearestExact(dataset *Dataset, queries [][]float64, m int, workload WorkloadEstimate) (ExactNearestResult, error)

ExecuteNearestExact reports the M nearest query points for every row, and returns what a float64 computation over every query point would return.

The device is used to narrow the field, never to decide. It ranks the query points in single precision and returns a shortlist per row; the host then recomputes that shortlist in float64 and picks from it. When the shortlist's cut is too close to call in single precision, that row is recomputed against every query point instead. So the answer does not depend on the device being right, only on it being close, and it does not depend on there being a device at all.

Unlike the single-precision operations, this one needs no precision opt-in. Narrowing happens inside, where it cannot reach the result.

func (*Session) LastReport 

func (s *Session) LastReport() *Report

func (*Session) PlanShardable 

func (s *Session) PlanShardable() ShardPlan

func (*Session) PlanShardableWorkload 

func (s *Session) PlanShardableWorkload(workload WorkloadEstimate) ShardPlan

func (*Session) ProjectDataList 

func (s *Session) ProjectDataList(dl *insyra.DataList) (*Dataset, error)

func (*Session) ProjectDataTable 

func (s *Session) ProjectDataTable(dt *insyra.DataTable) (*Dataset, error)

func (*Session) RecordReport 

func (s *Session) RecordReport(report Report) error

func (*Session) RegisterDevice 

func (s *Session) RegisterDevice(device Device) error

func (*Session) Report 

func (s *Session) Report() Report

func (*Session) Reports 

func (s *Session) Reports() []Report

type ShardAssignment 

type ShardAssignment struct {
	DeviceID       string
	Backend        Backend
	Weight         float64
	SharePercent   float64
	Rows           int
	Bytes          uint64
	BudgetBytes    uint64
	RowStart       int
	RowEnd         int
	WallTime       time.Duration
	FallbackReason FallbackReason
	Chunks         int
}

type ShardPlan 

type ShardPlan struct {
	Accelerated      bool
	Backend          Backend
	DeviceIDs        []string
	Assignments      []ShardAssignment
	TotalBudgetBytes uint64
	Heterogeneous    bool
	MergePolicy      MergePolicy
	FallbackReason   FallbackReason
}

type ShardStrategy 

type ShardStrategy string

ShardStrategy controls how a shardable workload consumes its eligible devices. Auto is the default and only shards above the recorded saturation floor; forced is useful when a caller is measuring its own hardware. 

const (
	ShardStrategySingle ShardStrategy = "single"
	ShardStrategyAuto   ShardStrategy = "auto"
	ShardStrategyForced ShardStrategy = "forced"
)

type UnmatchedDeviceSelector 

type UnmatchedDeviceSelector struct {
	Bound    string
	Selector string
}

UnmatchedDeviceSelector records a device bound entry that matched neither a discovered device ID nor a zero-based discovery index.

type WorkloadClass 

type WorkloadClass string
const (
	WorkloadClassUnknown  WorkloadClass = "unknown"
	WorkloadClassColumnar WorkloadClass = "columnar"
)

type WorkloadEstimate 

type WorkloadEstimate struct {
	Class WorkloadClass
	Rows  int
	Bytes uint64
	// Dimensions identifies the measured per-row shape used by auto sharding.
	// Zero means the conservative 32-dimensional class.
	Dimensions int
	// Op is the operation to execute on the device. Empty means OpSum.
	Op Op
	// Precision is what the caller will accept. Empty means PrecisionExact.
	Precision Precision
}

Source Files

View all Source files

Directories

Path Synopsis
internal
wgpu
Package wgpu runs numeric kernels on a GPU through the pure-Go WebGPU implementation in github.com/gogpu/wgpu.
 Package wgpu runs numeric kernels on a GPU through the pure-Go WebGPU implementation in github.com/gogpu/wgpu.
Package knnbridge plugs the accelerator into stats' KNN device socket.
 Package knnbridge plugs the accelerator into stats' KNN device socket.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.