options

Documentation

Index

• type GoMLXOptions
• type Options
  • func Defaults() *Options
• type OrtOptions
• type WithOption
  • func WithCPUMemArena(enable bool) WithOption
  • func WithCoreML(flags map[string]string) WithOption
  • func WithCuda(options map[string]string) WithOption
  • func WithDirectML(deviceID int) WithOption
  • func WithExecutionMode(parallel bool) WithOption
  • func WithInterOpNumThreads(numThreads int) WithOption
  • func WithInterOpSpinning(spinning bool) WithOption
  • func WithIntraOpNumThreads(numThreads int) WithOption
  • func WithIntraOpSpinning(spinning bool) WithOption
  • func WithMemPattern(enable bool) WithOption
  • func WithOnnxLibraryPath(ortLibraryPath string) WithOption
  • func WithOpenVINO(options map[string]string) WithOption
  • func WithTelemetry() WithOption
  • func WithTensorRT(options map[string]string) WithOption

Types

type GoMLXOptions

type GoMLXOptions struct {
	Cuda bool
	XLA  bool
}

type Options

type Options struct {
	BackendOptions any
	ORTOptions     *OrtOptions
	GoMLXOptions   *GoMLXOptions
	Destroy        func() error
	Backend        string
}

func Defaults

func Defaults() *Options

type OrtOptions

type OrtOptions struct {
	LibraryPath           *string
	Telemetry             *bool
	IntraOpNumThreads     *int
	InterOpNumThreads     *int
	CPUMemArena           *bool
	MemPattern            *bool
	ParallelExecutionMode *bool
	IntraOpSpinning       *bool
	InterOpSpinning       *bool
	CudaOptions           map[string]string
	CoreMLOptions         map[string]string
	DirectMLOptions       *int
	OpenVINOOptions       map[string]string
	TensorRTOptions       map[string]string
}

type WithOption

type WithOption func(o *Options) error

WithOption is the interface for all option functions.

func WithCPUMemArena

func WithCPUMemArena(enable bool) WithOption

WithCPUMemArena (ORT only) Enable/Disable the usage of the memory arena on CPU. Arena may pre-allocate memory for future usage. Default is true.

func WithCoreML

func WithCoreML(flags map[string]string) WithOption

WithCoreML (ORT only) Use this function to set the CoreML options flags for the ONNX backend configuration. The `flags` parameter represents the CoreML options flags. The `o.CoreMLOptions` field in `OrtOptions` struct will be set to the provided flags parameter.

func WithCuda

func WithCuda(options map[string]string) WithOption

WithCuda Use this function to set the options for CUDA provider. It takes a map of CUDA parameters as input. The options will be applied to the OrtOptions or GoMLXOptions struct, depending on your current backend.

func WithDirectML

func WithDirectML(deviceID int) WithOption

WithDirectML (ORT only) Use this function to set the DirectML device ID for the onnxbackend. By default, this option is not set.

func WithExecutionMode

func WithExecutionMode(parallel bool) WithOption

WithExecutionMode sets the parallel execution mode for the ORT backend. Returns an error if the backend is not ORT.

func WithInterOpNumThreads

func WithInterOpNumThreads(numThreads int) WithOption

WithInterOpNumThreads (ORT only) Sets the number of threads used to parallelize execution across separate onnxbackend graph nodes. If unspecified, onnxbackend uses the number of physical CPU cores.

func WithInterOpSpinning

func WithInterOpSpinning(spinning bool) WithOption

WithInterOpSpinning sets the spinning behavior for inter-op threads when the backend is ORT. It returns an error if used with a backend other than ORT.

func WithIntraOpNumThreads

func WithIntraOpNumThreads(numThreads int) WithOption

WithIntraOpNumThreads (ORT only) Sets the number of threads used to parallelize execution within onnxbackend graph nodes. If unspecified, onnxbackend uses the number of physical CPU cores.

func WithIntraOpSpinning

func WithIntraOpSpinning(spinning bool) WithOption

WithIntraOpSpinning configures whether intra-op spinning is enabled for the ORT backend. It returns an error if used with a backend other than ORT.

func WithMemPattern

func WithMemPattern(enable bool) WithOption

WithMemPattern (ORT only) Enable/Disable the memory pattern optimization. If this is enabled memory is preallocated if all shapes are known. Default is true.

func WithOnnxLibraryPath

func WithOnnxLibraryPath(ortLibraryPath string) WithOption

WithOnnxLibraryPath (ORT only) Use this function to set the path to the "onnxbackend.so" or "onnxbackend.dll" function. By default, it will be set to "onnxbackend.so" on non-Windows systems, and "onnxbackend.dll" on Windows.

func WithOpenVINO

func WithOpenVINO(options map[string]string) WithOption

WithOpenVINO (ORT only) Use this function to set the OpenVINO options for the OpenVINO execution provider. The options parameter should be a map of string keys and string values, representing the configuration options. For each key-value pair in the map, the specified option will be set in the OpenVINO execution provider. Example usage: WithOpenVINO(map[string]string{"device_type": "CPU", "num_threads": "4"}) This will configure the OpenVINO execution provider to use CPU as the device type and set the number of threads to 4.

func WithTelemetry

func WithTelemetry() WithOption

WithTelemetry (ORT only) Enables telemetry events for the onnxbackend environment. Default is off.

func WithTensorRT

func WithTensorRT(options map[string]string) WithOption

WithTensorRT (ORT only) Use this function to set the options for the TensorRT provider. The options parameter should be a pointer to an instance of TensorRTProviderOptions. By default, the options will be nil and the TensorRT provider will not be used. Example usage:

options := &onnxbackend_go.TensorRTProviderOptions{
    DeviceID: 0,
}
WithTensorRT(options)

Note: For the TensorRT provider to work, the onnxbackend library must be built with TensorRT support.