onnxruntime_go

package module
v0.0.0-...-66b3851 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 9, 2026 License: MIT Imports: 17 Imported by: 0

README

onnxruntime_go Cross-Platform ONNX Runtime Wrapper for Go with Encrypted Model Support

Go Reference

Overview

This library provides a Go interface for loading and executing ONNX neural networks using Microsoft's onnxruntime library. It extends the original yalue/onnxruntime_go with:

  • Encrypted Model Support - AES-256-GCM encryption for model protection
  • Authorization Integration - Integration with machineid/cert for enterprise licensing
  • Machine Binding - Derive encryption keys from machine ID for hardware-locked models

Features

  • Cross-platform support (Windows, Linux, macOS)
  • Multiple execution providers (CUDA, TensorRT, CoreML, DirectML, OpenVINO)
  • Generic tensor support with Go generics
  • Dynamic and static session modes
  • Model encryption and decryption
  • Authorization-based model access control

Installation

go get cnb.cool/svn/onnxruntime

You'll also need the ONNX Runtime shared library for your platform. Download from onnxruntime releases.

Quick Start

Basic Inference
package main

import (
    "fmt"
    ort "cnb.cool/svn/onnxruntime"
)

func main() {
    // Set library path
    ort.SetSharedLibraryPath("/path/to/libonnxruntime.so")

    // Initialize environment
    if err := ort.InitializeEnvironment(); err != nil {
        panic(err)
    }
    defer ort.DestroyEnvironment()

    // Create input tensor
    inputShape := ort.NewShape(1, 3, 224, 224)
    inputData := make([]float32, inputShape.FlattenedSize())
    inputTensor, _ := ort.NewTensor(inputShape, inputData)
    defer inputTensor.Destroy()

    // Create output tensor
    outputShape := ort.NewShape(1, 1000)
    outputTensor, _ := ort.NewEmptyTensor[float32](outputShape)
    defer outputTensor.Destroy()

    // Create session and run
    session, _ := ort.NewAdvancedSession("model.onnx",
        []string{"input"}, []string{"output"},
        []ort.Value{inputTensor}, []ort.Value{outputTensor}, nil)
    defer session.Destroy()

    session.Run()

    result := outputTensor.GetData()
    fmt.Println("Result:", result[:10])
}
Encrypted Model Inference
package main

import (
    ort "cnb.cool/svn/onnxruntime"
)

func main() {
    ort.SetSharedLibraryPath("/path/to/libonnxruntime.so")
    ort.InitializeEnvironment()
    defer ort.DestroyEnvironment()

    // Encryption key (32 bytes for AES-256)
    key := []byte("your-32-byte-secret-key-here!!")

    // Encrypt model (one-time operation)
    ort.EncryptModel("model.onnx", "model.onnx.enc", key)

    // Get model info from encrypted file
    inputs, outputs, _ := ort.GetInputOutputInfoFromEncryptedFile("model.onnx.enc", key)

    // Create session from encrypted model
    session, _ := ort.NewDynamicAdvancedSessionFromEncryptedFile(
        "model.onnx.enc", key,
        []string{inputs[0].Name}, []string{outputs[0].Name}, nil)
    defer session.Destroy()

    // Run inference...
}
Machine-Bound Encrypted Model
package main

import (
    "github.com/darkit/machineid"
    ort "cnb.cool/svn/onnxruntime"
)

func main() {
    ort.SetSharedLibraryPath("/path/to/libonnxruntime.so")
    ort.InitializeEnvironment()
    defer ort.DestroyEnvironment()

    // Get machine ID
    machineID, _ := machineid.ID()
    moduleName := "ai.model.classifier"
    salt := []byte("app-specific-salt")

    // Encrypt model for this machine
    ort.EncryptModelForMachine("model.onnx", "model.onnx.enc",
        machineID, moduleName, salt)

    // Derive key and create session
    key := ort.DeriveModelKey(machineID, moduleName, salt)
    session, _ := ort.NewDynamicAdvancedSessionFromEncryptedFile(
        "model.onnx.enc", key, inputNames, outputNames, nil)
    defer session.Destroy()
}

API Reference

Environment Management
SetSharedLibraryPath
func SetSharedLibraryPath(path string)

Sets the path to the ONNX Runtime shared library. Must be called before InitializeEnvironment().

InitializeEnvironment
func InitializeEnvironment(opts ...EnvironmentOption) error

Initializes the ONNX Runtime environment. Must be called before creating any sessions.

Options:

  • WithLogLevelVerbose() - Enable verbose logging
  • WithLogLevelInfo() - Enable info logging
  • WithLogLevelWarning() - Enable warning logging (default)
  • WithLogLevelError() - Enable error logging only
  • WithLogLevelFatal() - Enable fatal logging only
DestroyEnvironment
func DestroyEnvironment() error

Cleans up the ONNX Runtime environment. Should be called when done using the library.

IsInitialized
func IsInitialized() bool

Returns true if the environment has been initialized.

GetVersion
func GetVersion() string

Returns the ONNX Runtime version string.

DisableTelemetry / EnableTelemetry
func DisableTelemetry() error
func EnableTelemetry() error

Controls ONNX Runtime telemetry collection.

SetEnvironmentLogLevel
func SetEnvironmentLogLevel(level LoggingLevel) error

Sets the logging level for the environment.

Shape
NewShape
func NewShape(dimensions ...int64) Shape

Creates a new tensor shape from the given dimensions.

Shape Methods
func (s Shape) FlattenedSize() int64    // Total number of elements
func (s Shape) Validate() error          // Validates dimensions are positive
func (s Shape) Clone() Shape             // Creates a copy
func (s Shape) String() string           // String representation
func (s Shape) Equals(other Shape) bool  // Compares two shapes
Tensor Types
Tensor[T]

Generic tensor for numeric data types.

func NewTensor[T TensorData](s Shape, data []T) (*Tensor[T], error)
func NewEmptyTensor[T TensorData](s Shape) (*Tensor[T], error)

Supported types (TensorData):

  • float32, float64
  • int8, int16, int32, int64
  • uint8, uint16, uint32, uint64

Methods:

func (t *Tensor[T]) GetData() []T                    // Get underlying data slice
func (t *Tensor[T]) GetShape() Shape                 // Get tensor shape
func (t *Tensor[T]) Clone() (*Tensor[T], error)      // Create a copy
func (t *Tensor[T]) ZeroContents()                   // Zero all elements
func (t *Tensor[T]) Destroy() error                  // Release resources
func (t *Tensor[T]) GetONNXType() ONNXType           // Returns ONNXTypeTensor
func (t *Tensor[T]) DataType() ONNXTensorElementDataType
Scalar[T]

Single-value tensor (0-dimensional).

func NewScalar[T TensorData](data T) (*Scalar[T], error)
func NewEmptyScalar[T TensorData]() (*Scalar[T], error)

Methods:

func (s *Scalar[T]) GetData() T      // Get the scalar value
func (s *Scalar[T]) Set(value T)     // Set the scalar value
func (s *Scalar[T]) Destroy() error
StringTensor

Tensor containing string data.

func NewStringTensor(shape Shape) (*StringTensor, error)

Methods:

func (t *StringTensor) SetContents(contents []string) error
func (t *StringTensor) GetContents() ([]string, error)
func (t *StringTensor) SetElement(index int64, s string) error
func (t *StringTensor) GetElement(index int64) (string, error)
func (t *StringTensor) Destroy() error
CustomDataTensor

Tensor with custom binary data and element type.

func NewCustomDataTensor(s Shape, data []byte, dataType TensorElementDataType) (*CustomDataTensor, error)

Methods:

func (t *CustomDataTensor) GetData() []byte
func (t *CustomDataTensor) Destroy() error
Container Types
Sequence

ONNX sequence container.

func NewSequence(contents []Value) (*Sequence, error)

Methods:

func (s *Sequence) GetValues() ([]Value, error)
func (s *Sequence) Destroy() error
Map

ONNX map container.

func NewMap(keys, values Value) (*Map, error)
func NewMapFromGoMap[K, V TensorData](m map[K]V) (*Map, error)

Methods:

func (m *Map) GetKeysAndValues() (Value, Value, error)
func (m *Map) Destroy() error
Sessions
AdvancedSession

Static session with pre-bound input/output tensors.

func NewAdvancedSession(onnxFilePath string, inputNames, outputNames []string,
    inputs, outputs []Value, options *SessionOptions) (*AdvancedSession, error)

func NewAdvancedSessionWithONNXData(onnxData []byte, inputNames, outputNames []string,
    inputs, outputs []Value, options *SessionOptions) (*AdvancedSession, error)

Methods:

func (s *AdvancedSession) Run() error
func (s *AdvancedSession) RunWithOptions(opts *RunOptions) error
func (s *AdvancedSession) GetModelMetadata() (*ModelMetadata, error)
func (s *AdvancedSession) Destroy() error
DynamicAdvancedSession

Dynamic session where inputs/outputs are specified at runtime.

func NewDynamicAdvancedSession(onnxFilePath string, inputNames, outputNames []string,
    options *SessionOptions) (*DynamicAdvancedSession, error)

func NewDynamicAdvancedSessionWithONNXData(onnxData []byte, inputNames, outputNames []string,
    options *SessionOptions) (*DynamicAdvancedSession, error)

Methods:

func (s *DynamicAdvancedSession) Run(inputs, outputs []Value) error
func (s *DynamicAdvancedSession) RunWithOptions(inputs, outputs []Value, opts *RunOptions) error
func (s *DynamicAdvancedSession) RunWithBinding(b *IoBinding) error
func (s *DynamicAdvancedSession) CreateIoBinding() (*IoBinding, error)
func (s *DynamicAdvancedSession) GetModelMetadata() (*ModelMetadata, error)
func (s *DynamicAdvancedSession) Destroy() error
IoBinding

I/O binding for optimized memory management.

func (s *DynamicAdvancedSession) CreateIoBinding() (*IoBinding, error)

Methods:

func (b *IoBinding) BindInput(name string, value Value) error
func (b *IoBinding) BindOutput(name string, value Value) error
func (b *IoBinding) GetBoundOutputNames() ([]string, error)
func (b *IoBinding) GetBoundOutputValues() ([]Value, error)
func (b *IoBinding) ClearBoundInputs()
func (b *IoBinding) ClearBoundOutputs()
func (b *IoBinding) Destroy() error
Session Options
SessionOptions
func NewSessionOptions() (*SessionOptions, error)

Methods:

// Execution configuration
func (o *SessionOptions) SetExecutionMode(mode ExecutionMode) error
func (o *SessionOptions) SetGraphOptimizationLevel(level GraphOptimizationLevel) error
func (o *SessionOptions) SetLogSeverityLevel(level LoggingLevel) error
func (o *SessionOptions) SetIntraOpNumThreads(n int) error
func (o *SessionOptions) SetInterOpNumThreads(n int) error
func (o *SessionOptions) SetCpuMemArena(isEnabled bool) error
func (o *SessionOptions) SetMemPattern(isEnabled bool) error

// Session config entries
func (o *SessionOptions) HasSessionConfigEntry(key string) (bool, error)
func (o *SessionOptions) GetSessionConfigEntry(key string) (string, error)
func (o *SessionOptions) AddSessionConfigEntry(key, value string) error

// Execution providers
func (o *SessionOptions) AppendExecutionProviderCUDA(cudaOptions *CUDAProviderOptions) error
func (o *SessionOptions) AppendExecutionProviderTensorRT(tensorrtOptions *TensorRTProviderOptions) error
func (o *SessionOptions) AppendExecutionProviderCoreML(flags uint32) error
func (o *SessionOptions) AppendExecutionProviderCoreMLV2(options map[string]string) error
func (o *SessionOptions) AppendExecutionProviderDirectML(deviceID int) error
func (o *SessionOptions) AppendExecutionProviderOpenVINO(options map[string]string) error
func (o *SessionOptions) AppendExecutionProvider(providerName string, options map[string]string) error

func (o *SessionOptions) Destroy() error

ExecutionMode:

  • ExecutionModeSequential - Sequential execution
  • ExecutionModeParallel - Parallel execution

GraphOptimizationLevel:

  • GraphOptLevelDisableAll - No optimization
  • GraphOptLevelBasic - Basic optimizations
  • GraphOptLevelExtended - Extended optimizations
  • GraphOptLevelAll - All optimizations
RunOptions
func NewRunOptions() (*RunOptions, error)

Methods:

func (o *RunOptions) Terminate() error      // Request termination
func (o *RunOptions) UnsetTerminate() error // Clear termination flag
func (o *RunOptions) Destroy() error
Execution Providers
CUDAProviderOptions
func NewCUDAProviderOptions() (*CUDAProviderOptions, error)

Methods:

func (o *CUDAProviderOptions) Update(options map[string]string) error
func (o *CUDAProviderOptions) Destroy() error

Common options:

  • "device_id" - GPU device ID
  • "gpu_mem_limit" - Memory limit in bytes
  • "arena_extend_strategy" - Memory allocation strategy
TensorRTProviderOptions
func NewTensorRTProviderOptions() (*TensorRTProviderOptions, error)

Methods:

func (o *TensorRTProviderOptions) Update(options map[string]string) error
func (o *TensorRTProviderOptions) Destroy() error

Common options:

  • "device_id" - GPU device ID
  • "trt_max_workspace_size" - Maximum workspace size
  • "trt_fp16_enable" - Enable FP16 precision
  • "trt_int8_enable" - Enable INT8 precision
Model Information
GetInputOutputInfo
func GetInputOutputInfo(path string) ([]InputOutputInfo, []InputOutputInfo, error)
func GetInputOutputInfoWithOptions(path string, options *SessionOptions) ([]InputOutputInfo, []InputOutputInfo, error)
func GetInputOutputInfoWithONNXData(data []byte) ([]InputOutputInfo, []InputOutputInfo, error)

Returns input and output tensor information for a model.

InputOutputInfo
type InputOutputInfo struct {
    Name        string
    DataType    TensorElementDataType
    Dimensions  []int64
}

func (n *InputOutputInfo) String() string
GetModelMetadata
func GetModelMetadata(path string) (*ModelMetadata, error)
ModelMetadata Methods
func (m *ModelMetadata) GetProducerName() (string, error)
func (m *ModelMetadata) GetGraphName() (string, error)
func (m *ModelMetadata) GetDomain() (string, error)
func (m *ModelMetadata) GetDescription() (string, error)
func (m *ModelMetadata) GetVersion() (int64, error)
func (m *ModelMetadata) GetCustomMetadataMapKeys() ([]string, error)
func (m *ModelMetadata) LookupCustomMetadataMap(key string) (string, bool, error)
func (m *ModelMetadata) Destroy() error
Model Encryption
Basic Encryption
// Encrypt model file
func EncryptModel(inputPath, outputPath string, key []byte) error

// Decrypt model file
func DecryptModel(inputPath, outputPath string, key []byte) error

// Encrypt model data in memory
func EncryptModelData(plaintext, key []byte) ([]byte, error)

// Decrypt model data in memory
func DecryptModelData(data, key []byte) ([]byte, error)

// Generate random 32-byte encryption key
func GenerateEncryptionKey() ([]byte, error)

Encryption format: AES-256-GCM with magic header ORTENC01

Encrypted Sessions
// Create session from encrypted file
func NewAdvancedSessionFromEncryptedFile(encryptedPath string, key []byte,
    inputNames, outputNames []string, inputs, outputs []Value,
    options *SessionOptions) (*AdvancedSession, error)

func NewDynamicAdvancedSessionFromEncryptedFile(encryptedPath string, key []byte,
    inputNames, outputNames []string,
    options *SessionOptions) (*DynamicAdvancedSession, error)

// Create session from encrypted data
func NewAdvancedSessionFromEncryptedData(encryptedData, key []byte,
    inputNames, outputNames []string, inputs, outputs []Value,
    options *SessionOptions) (*AdvancedSession, error)

func NewDynamicAdvancedSessionFromEncryptedData(encryptedData, key []byte,
    inputNames, outputNames []string,
    options *SessionOptions) (*DynamicAdvancedSession, error)

// Get model info from encrypted file
func GetInputOutputInfoFromEncryptedFile(encryptedPath string, key []byte) (
    []InputOutputInfo, []InputOutputInfo, error)
Machine-Bound Encryption
// Derive encryption key from machine ID
func DeriveModelKey(machineID, moduleName string, salt []byte) []byte

// Derive key from Authorization object
func DeriveModelKeyFromAuth(auth Authorization, moduleName string, salt []byte) ([]byte, error)

// Encrypt model for specific machine
func EncryptModelForMachine(inputPath, outputPath, machineID, moduleName string, salt []byte) error

// Encrypt model data for specific machine
func EncryptModelDataForMachine(plaintext []byte, machineID, moduleName string, salt []byte) ([]byte, error)
Authorization Integration

Integration with machineid/cert package for enterprise licensing.

Authorization Interface
type Authorization interface {
    Validate(machineID string) error
    HasModule(name string) bool
    GetModuleQuota(name string) int
    ValidateModule(name string) error
    ExpiresAt() time.Time
    MachineIDs() []string
}
SecurityChecker Interface
type SecurityChecker interface {
    Check() error  // Anti-debugging, VM detection, etc.
}
AuthorizedModelConfig
type AuthorizedModelConfig struct {
    ModuleName         string           // Module name for authorization
    MachineID          string           // Current machine ID
    Authorization      Authorization    // Authorization object
    SecurityChecker    SecurityChecker  // Optional security checker
    KeyDerivationSalt  []byte           // Salt for key derivation
    ValidateOnEveryRun bool             // Validate before each inference
    QuotaTracker       QuotaTracker     // Optional quota tracking
}
AuthorizedSession
func NewAuthorizedSession(encryptedPath string, config AuthorizedModelConfig,
    inputNames, outputNames []string, options *SessionOptions) (*AuthorizedSession, error)

func NewAuthorizedSessionFromData(encryptedData []byte, config AuthorizedModelConfig,
    inputNames, outputNames []string, options *SessionOptions) (*AuthorizedSession, error)

Methods:

func (s *AuthorizedSession) Run(inputs, outputs []Value) error
func (s *AuthorizedSession) GetSession() *DynamicAdvancedSession
func (s *AuthorizedSession) Destroy() error
QuotaTracker Interface
type QuotaTracker interface {
    Increment(moduleName string) (int, error)
    GetCount(moduleName string) int
    Reset(moduleName string)
}

// Built-in implementation
func NewInMemoryQuotaTracker() *InMemoryQuotaTracker
ModelAuthorizationInfo
type ModelAuthorizationInfo struct {
    ModuleName  string   `json:"module_name"`
    Salt        []byte   `json:"salt"`
    ModelHash   []byte   `json:"model_hash,omitempty"`
    InputNames  []string `json:"input_names"`
    OutputNames []string `json:"output_names"`
    Description string   `json:"description,omitempty"`
    Version     string   `json:"version,omitempty"`
}

func (info *ModelAuthorizationInfo) ValidateModelHash(encryptedData []byte) bool
Constants and Types
TensorElementDataType
const (
    TensorElementDataTypeUndefined TensorElementDataType = iota
    TensorElementDataTypeFloat
    TensorElementDataTypeUint8
    TensorElementDataTypeInt8
    TensorElementDataTypeUint16
    TensorElementDataTypeInt16
    TensorElementDataTypeInt32
    TensorElementDataTypeInt64
    TensorElementDataTypeString
    TensorElementDataTypeBool
    TensorElementDataTypeFloat16
    TensorElementDataTypeDouble
    TensorElementDataTypeUint32
    TensorElementDataTypeUint64
    TensorElementDataTypeComplex64
    TensorElementDataTypeComplex128
    TensorElementDataTypeBFloat16
)
ONNXType
const (
    ONNXTypeUnknown ONNXType = iota
    ONNXTypeTensor
    ONNXTypeSequence
    ONNXTypeMap
    ONNXTypeOpaque
    ONNXTypeSparseTensor
    ONNXTypeOptional
)
LoggingLevel
const (
    LoggingLevelVerbose LoggingLevel = iota
    LoggingLevelInfo
    LoggingLevelWarning
    LoggingLevelError
    LoggingLevelFatal
)
Encryption Constants
const (
    EncryptedModelMagic = "ORTENC01"  // Magic header for encrypted files
    AESKeySize          = 32          // AES-256 key size
    GCMNonceSize        = 12          // GCM nonce size
)
Deprecated APIs

The following APIs are deprecated but maintained for backward compatibility:

// Use AdvancedSession instead
type Session[T TensorData] struct{}
type DynamicSession[In, Out TensorData] struct{}

func NewSession[T TensorData](...) (*Session[T], error)
func NewDynamicSession[In, Out TensorData](...) (*DynamicSession[In, Out], error)
func NewSessionWithONNXData[T TensorData](...) (*Session[T], error)
func NewDynamicSessionWithONNXData[In, Out TensorData](...) (*DynamicSession[In, Out], error)

// Training API (deprecated in onnxruntime 1.20+)
type TrainingSession struct{}
func NewTrainingSession(...) (*TrainingSession, error)
func IsTrainingSupported() bool  // Always returns false

Examples

CUDA Acceleration
cudaOpts, _ := ort.NewCUDAProviderOptions()
defer cudaOpts.Destroy()
cudaOpts.Update(map[string]string{
    "device_id": "0",
})

sessionOpts, _ := ort.NewSessionOptions()
defer sessionOpts.Destroy()
sessionOpts.AppendExecutionProviderCUDA(cudaOpts)

session, _ := ort.NewAdvancedSession("model.onnx",
    inputNames, outputNames, inputs, outputs, sessionOpts)
TensorRT Acceleration
trtOpts, _ := ort.NewTensorRTProviderOptions()
defer trtOpts.Destroy()
trtOpts.Update(map[string]string{
    "device_id":              "0",
    "trt_fp16_enable":        "1",
    "trt_max_workspace_size": "2147483648",
})

sessionOpts, _ := ort.NewSessionOptions()
defer sessionOpts.Destroy()
sessionOpts.AppendExecutionProviderTensorRT(trtOpts)

session, _ := ort.NewDynamicAdvancedSession("model.onnx",
    inputNames, outputNames, sessionOpts)
Authorization with machineid/cert
import (
    "github.com/darkit/machineid"
    "github.com/darkit/machineid/cert"
    ort "cnb.cool/svn/onnxruntime"
)

func main() {
    // Get machine ID
    machineID, _ := machineid.ID()

    // Create authorizer and load certificate
    authorizer, _ := cert.NewAuthorizer().
        WithCA(caPEM, caKeyPEM).
        Build()

    certAuth, _ := cert.NewCertAuthorization(certPEM, authorizer)

    // Create security checker
    securityMgr := cert.NewSecurityManager(cert.SecurityLevelAdvanced)

    // Configure authorized session
    config := ort.AuthorizedModelConfig{
        ModuleName:         "ai.model.classifier",
        MachineID:          machineID,
        Authorization:      certAuth,  // Implements ort.Authorization
        SecurityChecker:    securityMgr,
        KeyDerivationSalt:  []byte("app-salt"),
        ValidateOnEveryRun: true,
        QuotaTracker:       ort.NewInMemoryQuotaTracker(),
    }

    session, _ := ort.NewAuthorizedSession("model.onnx.enc", config,
        inputNames, outputNames, nil)
    defer session.Destroy()

    // Run with authorization checks
    session.Run(inputs, outputs)
}

Testing

# Run all tests
go test -v

# Run with benchmarks
go test -v -bench=.

# Use custom onnxruntime library
ONNXRUNTIME_SHARED_LIBRARY_PATH=/path/to/libonnxruntime.so go test -v

Version Compatibility

This library uses ONNX Runtime C API version 1.23.2. To use a different version:

  1. Replace onnxruntime_c_api.h and onnxruntime_ep_c_api.h with your version
  2. Replace the shared library in test_data/
  3. Verify DirectML API compatibility if needed

License

See the original yalue/onnxruntime_go repository for license information.

Static Library Build

By default, this library loads ONNX Runtime dynamically at runtime. To use a statically linked ONNX Runtime library, use the static build tag.

Build with Static Library
# Set include and library paths
export CGO_CFLAGS="-I/opt/onnxruntime/include"
export CGO_LDFLAGS="-L/opt/onnxruntime/lib -l:libonnxruntime.a -lstdc++ -lm -lpthread -ldl"

# Build with static tag
go build -tags static ./...
Platform-Specific Examples

Linux (CPU only):

export CGO_CFLAGS="-I/opt/onnxruntime/include"
export CGO_LDFLAGS="-L/opt/onnxruntime/lib -l:libonnxruntime.a -lstdc++ -lm -lpthread -ldl"
go build -tags static ./...

Linux (with CUDA):

export CGO_CFLAGS="-I/opt/onnxruntime/include -I/usr/local/cuda/include"
export CGO_LDFLAGS="-L/opt/onnxruntime/lib -L/usr/local/cuda/lib64 \
    -l:libonnxruntime.a -lcudart -lcublas -lcublasLt -lcudnn \
    -lstdc++ -lm -lpthread -ldl"
go build -tags static ./...

Linux (with TensorRT):

export CGO_CFLAGS="-I/opt/onnxruntime/include -I/usr/local/cuda/include -I/opt/TensorRT/include"
export CGO_LDFLAGS="-L/opt/onnxruntime/lib -L/usr/local/cuda/lib64 -L/opt/TensorRT/lib \
    -l:libonnxruntime.a -lcudart -lcublas -lcublasLt -lcudnn \
    -lnvinfer -lnvinfer_plugin -lnvonnxparser \
    -lstdc++ -lm -lpthread -ldl"
go build -tags static ./...

macOS:

export CGO_CFLAGS="-I/opt/onnxruntime/include"
export CGO_LDFLAGS="-L/opt/onnxruntime/lib -lonnxruntime -lc++ \
    -framework Foundation -framework CoreML"
go build -tags static ./...

Windows (MinGW):

set CGO_CFLAGS=-I/opt/onnxruntime/include
set CGO_LDFLAGS=-L/opt/onnxruntime/lib -lonnxruntime -lstdc++ -lm -lpthread
go build -tags static ./...
Code Differences

When using static build, you don't need to call SetSharedLibraryPath():

package main

import (
    ort "github.com/yalue/onnxruntime_go"
)

func main() {
    // Static build: No need to set library path
    // ort.SetSharedLibraryPath() is ignored in static mode

    // Check if using static build
    if ort.IsStaticBuild() {
        println("Using static library")
    }

    // Initialize as usual
    if err := ort.InitializeEnvironment(); err != nil {
        panic(err)
    }
    defer ort.DestroyEnvironment()

    // ... rest of your code
}
Checking Build Mode
// IsStaticBuild returns true if compiled with -tags static
func IsStaticBuild() bool
Hybrid Linking (Static + Dynamic)

You can mix static and dynamic linking for optimal deployment:

Linker Flags
# -l:libxxx.a      Force static linking for specific library
# -lxxx            Prefer dynamic, fallback to static
# -Wl,-Bstatic     Static linking for subsequent libraries
# -Wl,-Bdynamic    Dynamic linking for subsequent libraries

export CGO_CFLAGS="-I/opt/onnxruntime/include -I/usr/local/cuda/include"
export CGO_LDFLAGS="\
    -L/opt/onnxruntime/lib \
    -L/usr/local/cuda/lib64 \
    -Wl,-Bstatic -lonnxruntime \
    -Wl,-Bdynamic -lcudart -lcublas -lcublasLt -lcudnn \
    -lstdc++ -lm -lpthread -ldl"

go build -tags static ./...
Example: ONNX Runtime Static + TensorRT Dynamic
export CGO_LDFLAGS="\
    -L/opt/onnxruntime/lib \
    -L/usr/local/cuda/lib64 \
    -L/opt/TensorRT/lib \
    -Wl,-Bstatic -lonnxruntime \
    -Wl,-Bdynamic -lcudart -lcublas -lcudnn \
    -lnvinfer -lnvinfer_plugin \
    -lstdc++ -lm -lpthread -ldl"
Example: Maximum Static (Core Static, System Dynamic)
export CGO_LDFLAGS="\
    -L/opt/onnxruntime/lib \
    -L/usr/local/cuda/lib64/stubs \
    -Wl,-Bstatic \
    -lonnxruntime \
    -lcudart_static -lcublas_static -lcublasLt_static \
    -Wl,-Bdynamic \
    -ldl -lpthread -lrt -lm"
Verify Linking
# Linux - Check dynamic dependencies
ldd ./myapp

# Check all symbols (including static)
nm ./myapp | grep -i onnx

# macOS
otool -L ./myapp
Hybrid Linking Trade-offs
Pros Cons
Core functionality portable More complex configuration
GPU libs can update with system Need to understand linker behavior
Smaller executable than full static Some libs still need deployment
Balance portability and flexibility Version matching during debug 
# onnxruntime static (portable core)
# CUDA/cuDNN dynamic (follow driver version)
# System libs dynamic (libc, libm, libpthread)

CGO_LDFLAGS="\
    -Wl,-Bstatic -l:libonnxruntime.a \
    -Wl,-Bdynamic -lcudart -lcudnn -lcublas \
    -lstdc++ -lm -lpthread -ldl"

This produces a binary that:

  • Does not require distributing libonnxruntime.so
  • Requires matching CUDA driver on target machine
  • Uses target machine's system libs for compatibility

Documentation

Overview

This library wraps the C "onnxruntime" library maintained at https://github.com/microsoft/onnxruntime. It seeks to provide as simple an interface as possible to load and run ONNX-format neural networks from Go code.

Index

Constants

View Source 
const (
	// 加密文件魔数，用于识别加密格式
	EncryptedModelMagic = "ORTENC01"
	// AES-256 密钥长度
	AESKeySize = 32
	// GCM nonce 长度
	GCMNonceSize = 12
	// DefaultPBKDF2Iterations DeriveKeyFromPassword 的默认迭代次数
	DefaultPBKDF2Iterations = 600000
)

加密相关常量

View Source 
const (
	LoggingLevelVerbose = C.ORT_LOGGING_LEVEL_VERBOSE
	LoggingLevelInfo    = C.ORT_LOGGING_LEVEL_INFO
	LoggingLevelWarning = C.ORT_LOGGING_LEVEL_WARNING
	LoggingLevelError   = C.ORT_LOGGING_LEVEL_ERROR
	LoggingLevelFatal   = C.ORT_LOGGING_LEVEL_FATAL
)
View Source 
const (
	TensorElementDataTypeUndefined  = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UNDEFINED
	TensorElementDataTypeFloat      = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT  // maps to c type float
	TensorElementDataTypeUint8      = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8  // maps to c type uint8_t
	TensorElementDataTypeInt8       = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8   // maps to c type int8_t
	TensorElementDataTypeUint16     = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT16 // maps to c type uint16_t
	TensorElementDataTypeInt16      = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT16  // maps to c type int16_t
	TensorElementDataTypeInt32      = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32  // maps to c type int32_t
	TensorElementDataTypeInt64      = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64  // maps to c type int64_t
	TensorElementDataTypeString     = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING // maps to c++ type std::string
	TensorElementDataTypeBool       = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL
	TensorElementDataTypeFloat16    = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16
	TensorElementDataTypeDouble     = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE     // maps to c type double
	TensorElementDataTypeUint32     = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT32     // maps to c type uint32_t
	TensorElementDataTypeUint64     = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT64     // maps to c type uint64_t
	TensorElementDataTypeComplex64  = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX64  // complex with float32 real and imaginary components
	TensorElementDataTypeComplex128 = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX128 // complex with float64 real and imaginary components
	TensorElementDataTypeBFloat16   = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_BFLOAT16   // Non-IEEE floating-point format based on IEEE754 single-precision
	// float 8 types were introduced in onnx 1.14, see https://onnx.ai/onnx/technical/float8.html
	TensorElementDataTypeFloat8E4M3FN   = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FN   // Non-IEEE floating-point format based on IEEE754 single-precision
	TensorElementDataTypeFloat8E4M3FNUZ = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FNUZ // Non-IEEE floating-point format based on IEEE754 single-precision
	TensorElementDataTypeFloat8E5M2     = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2     // Non-IEEE floating-point format based on IEEE754 single-precision
	TensorElementDataTypeFloat8E5M2FNUZ = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2FNUZ // Non-IEEE floating-point format based on IEEE754 single-precision
	// Int4 types were introduced in ONNX 1.16. See https://onnx.ai/onnx/technical/int4.html
	TensorElementDataTypeUint4 = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT4 // maps to a pair of packed uint4 values (size == 1 byte)
	TensorElementDataTypeInt4  = C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT4  // maps to a pair of packed int4 values (size == 1 byte)
)
View Source 
const (
	GraphOptimizationLevelDisableAll     = C.ORT_DISABLE_ALL
	GraphOptimizationLevelEnableBasic    = C.ORT_ENABLE_BASIC
	GraphOptimizationLevelEnableExtended = C.ORT_ENABLE_EXTENDED
	GraphOptimizationLevelEnableAll      = C.ORT_ENABLE_ALL
)
View Source 
const (
	ONNXTypeUnknown      = C.ONNX_TYPE_UNKNOWN
	ONNXTypeTensor       = C.ONNX_TYPE_TENSOR
	ONNXTypeSequence     = C.ONNX_TYPE_SEQUENCE
	ONNXTypeMap          = C.ONNX_TYPE_MAP
	ONNXTypeOpaque       = C.ONNX_TYPE_OPAQUE
	ONNXTypeSparseTensor = C.ONNX_TYPE_SPARSETENSOR
	ONNXTypeOptional     = C.ONNX_TYPE_OPTIONAL
)
View Source 
const (
	ExecutionModeSequential = C.ORT_SEQUENTIAL
	ExecutionModeParallel   = C.ORT_PARALLEL
)

Variables

View Source 
var ErrSessionDestroyed = errors.New("authorized session already destroyed")

ErrSessionDestroyed 表示会话已被销毁，不能再使用。

View Source 
var NotInitializedError error = fmt.Errorf("InitializeRuntime() has either " +
	"not yet been called, or did not return successfully")
View Source 
var ShapeOverflowError error = fmt.Errorf("The shape's flattened size " +
	"overflows an int64")
View Source 
var TrainingAPIRemovedError error = fmt.Errorf("Support for the training " +
	"API has been removed from onnxruntime_go following its deprecation in " +
	"onnxruntime versions 1.19.2 and later. The last revision of " +
	"onnxruntime_go supporting the training API is version v1.12.1")
View Source 
var ZeroShapeLengthError error = fmt.Errorf("The shape has no dimensions")

Functions

func DecryptModel 

func DecryptModel(inputPath, outputPath string, key []byte) error

DecryptModel 解密 ONNX 模型文件 inputPath: 加密模型路径 outputPath: 解密后的输出路径 key: 解密密钥

func DecryptModelData 

func DecryptModelData(data, key []byte) ([]byte, error)

DecryptModelData 解密模型数据（内存中）

func DeriveKeyFromPassword 

func DeriveKeyFromPassword(password string, salt []byte, iterations int) ([]byte, error)

DeriveKeyFromPassword 用 PBKDF2-HMAC-SHA256 从口令派生 32 字节 AES-256 密钥。

相比对口令直接做单次 SHA-256，PBKDF2 的多轮迭代显著抬高暴力破解成本， 适合基于人类口令的模型加密场景。salt 应为每个模型唯一的随机值 （可用 GenerateEncryptionKey 生成并随加密产物一同保存）；iterations<=0 时 使用 DefaultPBKDF2Iterations。派生结果可直接传给 EncryptModelData/DecryptModelData。

func DeriveModelKey 

func DeriveModelKey(machineID, moduleName string, salt []byte) []byte

DeriveModelKey 从授权信息派生模型加密密钥 将机器ID、模块名称和盐值结合，生成唯一的加密密钥

func DeriveModelKeyFromAuth 

func DeriveModelKeyFromAuth(auth Authorization, moduleName string, salt []byte) ([]byte, error)

DeriveModelKeyFromAuth 从授权对象派生模型加密密钥

func DestroyEnvironment 

func DestroyEnvironment() error

Call this function to cleanup the internal onnxruntime environment when it is no longer needed.

func DisableTelemetry 

func DisableTelemetry() error

Disables telemetry events for the onnxruntime environment. Must be called after initializing the environment using InitializeEnvironment(). It is unclear from the onnxruntime docs whether this will cause an error or silently return if telemetry is already disabled.

func EnableTelemetry 

func EnableTelemetry() error

Enables telemetry events for the onnxruntime environment. Must be called after initializing the environment using InitializeEnvironment(). It is unclear from the onnxruntime docs whether this will cause an error or silently return if telemetry is already enabled.

func EncryptModel 

func EncryptModel(inputPath, outputPath string, key []byte) error

EncryptModel 加密 ONNX 模型文件 inputPath: 原始模型路径 outputPath: 加密后的输出路径 key: 加密密钥（任意长度，会通过 SHA-256 派生为 32 字节）

func EncryptModelData 

func EncryptModelData(plaintext, key []byte) ([]byte, error)

EncryptModelData 加密模型数据（内存中）

func EncryptModelDataForMachine 

func EncryptModelDataForMachine(plaintext []byte, machineID, moduleName string, salt []byte) ([]byte, error)

EncryptModelDataForMachine 为特定机器加密模型数据

func EncryptModelForMachine 

func EncryptModelForMachine(inputPath, outputPath, machineID, moduleName string, salt []byte) error

EncryptModelForMachine 为特定机器加密模型 使用机器ID和模块名称派生密钥

func GenerateEncryptionKey 

func GenerateEncryptionKey() ([]byte, error)

GenerateEncryptionKey 生成随机加密密钥

func GetInputOutputInfo 

func GetInputOutputInfo(path string) ([]InputOutputInfo, []InputOutputInfo,
	error)

Takes a path to a .onnx file, and returns a list of inputs and a list of outputs, respectively. Will open, read, and close the .onnx file to get the information. InitializeEnvironment() must have been called prior to using this function. Warning: this function requires loading the .onnx file into a temporary onnxruntime session, which may be an expensive operation.

For now, this may fail if the network has any non-tensor inputs or inputs that don't have a concrete shape and type. In the future, a new API may be added to support cases requiring more advanced usage of the C.OrtTypeInfo struct.

func GetInputOutputInfoFromEncryptedFile 

func GetInputOutputInfoFromEncryptedFile(encryptedPath string, key []byte) (
	[]InputOutputInfo, []InputOutputInfo, error)

GetInputOutputInfoFromEncryptedFile 从加密文件获取模型输入输出信息

func GetInputOutputInfoWithONNXData 

func GetInputOutputInfoWithONNXData(data []byte) ([]InputOutputInfo,
	[]InputOutputInfo, error)

Identical in behavior to GetInputOutputInfo, but takes a slice of bytes containing the .onnx network rather than a file path.

func GetInputOutputInfoWithOptions 

func GetInputOutputInfoWithOptions(path string,
	options *SessionOptions) ([]InputOutputInfo, []InputOutputInfo, error)

Identical in behavior to GetInputOutputInfo, but addtionally takes session options to handle models that require options to load.

func GetTensorElementDataType 

func GetTensorElementDataType[T TensorData]() C.ONNXTensorElementDataType

Returns the ONNX enum value used to indicate TensorData type T.

func GetVersion 

func GetVersion() string

GetVersion return version of the Onnxruntime library for logging.

func InitializeEnvironment 

func InitializeEnvironment(opts ...EnvironmentOption) error

Call this function, optionally with one or more EnvironmentOption, to initialize the internal onnxruntime environment. If this doesn't return an error, the caller will be responsible for calling DestroyEnvironment to free the onnxruntime state when no longer needed.

func IsInitialized 

func IsInitialized() bool

Returns false if the onnxruntime package is not initialized. Called internally by several functions, to avoid segfaulting if InitializeEnvironment hasn't been called yet.

func IsStaticBuild 

func IsStaticBuild() bool

IsStaticBuild 返回是否使用静态库构建

func IsTrainingSupported 

func IsTrainingSupported() bool

Always returns false.

func SetEnvironmentLogLevel 

func SetEnvironmentLogLevel(level LoggingLevel) error

Sets the environmnent-wide log severity level. The argument must be one of LoggingLevelVerbose, LoggingLevelInfo, LoggingLevelWarning, LoggingLevelError, or LoggingLevelFatal. Must only be used after the environment has been initialized.

func SetSharedLibraryPath 

func SetSharedLibraryPath(path string)

Use this function to set the path to the "onnxruntime.so" or "onnxruntime.dll" function. By default, it will be set to "onnxruntime.so" on non-Windows systems, and "onnxruntime.dll" on Windows. Users wishing to specify a particular location of this library must call this function prior to calling onnxruntime.InitializeEnvironment().

Types

type AdvancedSession 

type AdvancedSession struct {
	// contains filtered or unexported fields
}

A wrapper around the OrtSession C struct. Requires the user to maintain all input and output tensors, and to use the same data type for input and output tensors. Created using NewAdvancedSession(...) or NewAdvancedSessionWithONNXData(...). The caller is responsible for calling the Destroy() function on each session when it is no longer needed.

func NewAdvancedSession 

func NewAdvancedSession(onnxFilePath string, inputNames, outputNames []string,
	inputs, outputs []Value,
	options *SessionOptions) (*AdvancedSession, error)

Loads the ONNX network at the given path, and initializes an AdvancedSession instance. If this returns successfully, the caller must call Destroy() on the returned session when it is no longer needed. We require the user to provide the input and output tensors and names at this point, in order to not need to re-allocate them every time Run() is called. The user instead can just update or access the input/output tensor data after calling Run(). The input and output tensors MUST outlive this session, and calling session.Destroy() will not destroy the input or output tensors. If the provided SessionOptions pointer is nil, then the new session will use default options.

func NewAdvancedSessionFromEncryptedData 

func NewAdvancedSessionFromEncryptedData(encryptedData, key []byte,
	inputNames, outputNames []string, inputs, outputs []Value,
	options *SessionOptions) (*AdvancedSession, error)

NewAdvancedSessionFromEncryptedData 从加密数据创建 Session

func NewAdvancedSessionFromEncryptedFile 

func NewAdvancedSessionFromEncryptedFile(encryptedPath string, key []byte,
	inputNames, outputNames []string, inputs, outputs []Value,
	options *SessionOptions) (*AdvancedSession, error)

NewAdvancedSessionFromEncryptedFile 从加密文件创建 Session

func NewAdvancedSessionWithONNXData 

func NewAdvancedSessionWithONNXData(onnxData []byte, inputNames,
	outputNames []string, inputs, outputs []Value,
	options *SessionOptions) (*AdvancedSession, error)

The same as NewAdvancedSession, but takes a slice of bytes containing the .onnx network rather than a file path.

func (*AdvancedSession) Destroy 

func (s *AdvancedSession) Destroy() error

func (*AdvancedSession) GetModelMetadata 

func (s *AdvancedSession) GetModelMetadata() (*ModelMetadata, error)

Creates and returns a ModelMetadata instance for this session's model. The returned metadata must be freed using its Destroy() function when no longer needed.

func (*AdvancedSession) Run 

func (s *AdvancedSession) Run() error

Runs the session, updating the contents of the output tensors on success.

func (*AdvancedSession) RunWithOptions 

func (s *AdvancedSession) RunWithOptions(opts *RunOptions) error

RunWithOptions runs the session using the provided RunOptions.

type ArbitraryTensor 

type ArbitraryTensor = Value

This type alias is included to avoid breaking older code, where the inputs and outputs to session.Run() were ArbitraryTensors rather than Values.

type Authorization 

type Authorization interface {
	// Validate 验证授权（包含机器码验证）
	Validate(machineID string) error

	// HasModule 检查是否有模块权限
	HasModule(name string) bool

	// GetModuleQuota 获取模块配额（0=无限制）
	GetModuleQuota(name string) int

	// ValidateModule 验证模块权限（权限+时间）
	ValidateModule(name string) error

	// ExpiresAt 返回过期时间
	ExpiresAt() time.Time

	// MachineIDs 返回授权的机器码列表
	MachineIDs() []string
}

Authorization 授权接口（与 machineid/cert 兼容） 这是一个适配器接口，允许与 cert.Authorization 集成

type AuthorizedModelConfig 

type AuthorizedModelConfig struct {
	// ModuleName 模块名称（用于授权检查）
	// 每个 AI 模型可以对应一个授权模块
	ModuleName string

	// MachineID 当前机器ID
	MachineID string

	// Authorization 授权对象（来自 cert 包）
	Authorization Authorization

	// SecurityChecker 安全检查器（可选）
	SecurityChecker SecurityChecker

	// KeyDerivationSalt 密钥派生盐值（增强安全性）
	KeyDerivationSalt []byte

	// ValidateOnEveryRun 每次推理前验证授权
	ValidateOnEveryRun bool

	// QuotaTracker 配额追踪器（可选）
	QuotaTracker QuotaTracker
}

AuthorizedModelConfig 授权模型配置

type AuthorizedSession 

type AuthorizedSession struct {
	// contains filtered or unexported fields
}

AuthorizedSession 授权模型会话 结合授权验证和加密模型加载

func NewAuthorizedSession 

func NewAuthorizedSession(encryptedPath string, config AuthorizedModelConfig,
	inputNames, outputNames []string, options *SessionOptions) (*AuthorizedSession, error)

NewAuthorizedSession 创建授权模型会话

func NewAuthorizedSessionFromData 

func NewAuthorizedSessionFromData(encryptedData []byte, config AuthorizedModelConfig,
	inputNames, outputNames []string, options *SessionOptions) (*AuthorizedSession, error)

NewAuthorizedSessionFromData 从加密数据创建授权模型会话

func (*AuthorizedSession) Destroy 

func (s *AuthorizedSession) Destroy() error

Destroy 销毁会话。重复调用是幂等的，不会二次释放底层 C 会话。

func (*AuthorizedSession) GetSession 

func (s *AuthorizedSession) GetSession() *DynamicAdvancedSession

GetSession 获取底层会话（谨慎使用）

func (*AuthorizedSession) Run 

func (s *AuthorizedSession) Run(inputs, outputs []Value) error

Run 执行推理（带授权检查）

type BadShapeDimensionError 

type BadShapeDimensionError struct {
	DimensionIndex int
	DimensionSize  int64
}

This type of error is returned when we attempt to validate a tensor that has a negative or 0 dimension.

func (*BadShapeDimensionError) Error 

func (e *BadShapeDimensionError) Error() string

type CUDAProviderOptions 

type CUDAProviderOptions struct {
	// contains filtered or unexported fields
}

Holds options required when enabling the CUDA backend for a session. This struct wraps C onnxruntime types; users must create instances of this using the NewCUDAProviderOptions() function. So, to enable CUDA for a session, follow these steps:

  1. Call NewSessionOptions() to create a SessionOptions struct.
  2. Call NewCUDAProviderOptions() to obtain a CUDAProviderOptions struct.
  3. Call the CUDAProviderOptions struct's Update(...) function to pass a list of settings to CUDA. (See the comment on the Update() function.)
  4. Pass the CUDA options struct pointer to the SessionOptions.AppendExecutionProviderCUDA(...) function.
  5. Call the Destroy() function on the CUDA provider options.
  6. Call NewAdvancedSession(...), passing the SessionOptions struct to it.
  7. Call the Destroy() function on the SessionOptions struct.

Admittedly, this is a bit of a mess, but that's how it's handled by the C API internally. (The onnxruntime python API hides a bunch of this complexity using getter and setter functions, for which Go does not have a terse equivalent.)

func NewCUDAProviderOptions 

func NewCUDAProviderOptions() (*CUDAProviderOptions, error)

Initializes and returns a CUDAProviderOptions struct, used when enabling CUDA in a SessionOptions instance. (i.e., a CUDAProviderOptions must be configured, then passed to SessionOptions.AppendExecutionProviderCUDA.) The caller must call the Destroy() function on the returned struct when it's no longer needed.

func (*CUDAProviderOptions) Destroy 

func (o *CUDAProviderOptions) Destroy() error

Must be called when the CUDAProviderOptions struct is no longer needed; frees internal C-allocated state. Note that the CUDAProviderOptions struct can be destroyed as soon as options.AppendExecutionProviderCUDA has been called.

func (*CUDAProviderOptions) Update 

func (o *CUDAProviderOptions) Update(options map[string]string) error

Wraps the call to the UpdateCUDAProviderOptions in the onnxruntime C API. Requires a map of string keys to values for configuring the CUDA backend. For example, set the key "device_id" to "1" to use GPU 1 rather than 0.

The onnxruntime headers refer users to https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#configuration-options for a full list of available keys and values.

type CustomDataTensor 

type CustomDataTensor struct {
	// contains filtered or unexported fields
}

This satisfies the Value interface, but is intended to allow users to provide tensors of types that may not be supported by the generic typed Tensor[T] struct. Instead, CustomDataTensors are backed by a slice of bytes, using a user-provided shape and type from the ONNXTensorElementDataType enum.

func NewCustomDataTensor 

func NewCustomDataTensor(s Shape, data []byte,
	dataType TensorElementDataType) (*CustomDataTensor, error)

Creates and returns a new CustomDataTensor using the given bytes as the underlying data slice. Apart from ensuring that the provided data slice is non-empty, this function mostly delegates validation of the provided data to the C onnxruntime library. For example, it is the caller's responsibility to ensure that the provided dataType and data slice are valid and correctly sized for the specified shape. If this returns successfully, the caller must call the returned tensor's Destroy() function to free it when no longer in use.

func (*CustomDataTensor) DataType 

func (t *CustomDataTensor) DataType() C.ONNXTensorElementDataType

func (*CustomDataTensor) Destroy 

func (t *CustomDataTensor) Destroy() error

func (*CustomDataTensor) GetData 

func (t *CustomDataTensor) GetData() []byte

Returns the same slice that was passed to NewCustomDataTensor.

func (*CustomDataTensor) GetInternals 

func (t *CustomDataTensor) GetInternals() *ValueInternalData

func (*CustomDataTensor) GetONNXType 

func (t *CustomDataTensor) GetONNXType() ONNXType

Always returns ONNXTypeTensor, even if the CustomDataTensor is invalid for some reason.

func (*CustomDataTensor) GetShape 

func (t *CustomDataTensor) GetShape() Shape

func (*CustomDataTensor) ZeroContents 

func (t *CustomDataTensor) ZeroContents()

Sets all bytes in the data slice to 0.

type DynamicAdvancedSession 

type DynamicAdvancedSession struct {
	// contains filtered or unexported fields
}

This type of session does not require specifying input and output tensors ahead of time, but allows users to pass the list of input and output tensors when calling Run(). As with AdvancedSession, users must still call Destroy() on an DynamicAdvancedSession that is no longer needed.

func NewDynamicAdvancedSession 

func NewDynamicAdvancedSession(onnxFilePath string, inputNames,
	outputNames []string, options *SessionOptions) (*DynamicAdvancedSession,
	error)

Like NewAdvancedSession, but does not require specifying input and output tensors. Input and output names can be nil or empty, but _only if_ this session will only be used via RunWithBinding, which manages names separately.

func NewDynamicAdvancedSessionFromEncryptedData 

func NewDynamicAdvancedSessionFromEncryptedData(encryptedData, key []byte,
	inputNames, outputNames []string,
	options *SessionOptions) (*DynamicAdvancedSession, error)

NewDynamicAdvancedSessionFromEncryptedData 从加密数据创建动态 Session

func NewDynamicAdvancedSessionFromEncryptedFile 

func NewDynamicAdvancedSessionFromEncryptedFile(encryptedPath string, key []byte,
	inputNames, outputNames []string,
	options *SessionOptions) (*DynamicAdvancedSession, error)

NewDynamicAdvancedSessionFromEncryptedFile 从加密文件创建动态 Session

func NewDynamicAdvancedSessionWithONNXData 

func NewDynamicAdvancedSessionWithONNXData(onnxData []byte,
	inputNames, outputNames []string,
	options *SessionOptions) (*DynamicAdvancedSession, error)

Like NewAdvancedSessionWithONNXData, but does not require specifying input and output tensors.

func (*DynamicAdvancedSession) CreateIoBinding 

func (s *DynamicAdvancedSession) CreateIoBinding() (*IoBinding, error)

Creates and returns an IoBinding instance associated with the session. The I/O binding can be used to avoid unecessary copies to or from device memory, for sessions on different devices. The returned IoBinding must be freed using Destroy() when it is no longer needed.

func (*DynamicAdvancedSession) Destroy 

func (s *DynamicAdvancedSession) Destroy() error

func (*DynamicAdvancedSession) GetModelMetadata 

func (s *DynamicAdvancedSession) GetModelMetadata() (*ModelMetadata, error)

Creates and returns a ModelMetadata instance for this session's model. The returned metadata must be freed using its Destroy() function when no longer needed.

func (*DynamicAdvancedSession) Run 

func (s *DynamicAdvancedSession) Run(inputs, outputs []Value) error

Runs the network on the given input and output tensors. The number of input and output tensors must match the number (and order) of the input and output names specified to NewDynamicAdvancedSession. If a given output is nil, it will be allocated and the slice will be modified to include the new Value. Any new Value allocated in this way must be freed by calling Destroy on it.

func (*DynamicAdvancedSession) RunWithBinding 

func (s *DynamicAdvancedSession) RunWithBinding(b *IoBinding) error

Runs the session using the given IoBinding instance. The IoBinding must have been created from this session's CreateIoBinding() function.

func (*DynamicAdvancedSession) RunWithOptions 

func (s *DynamicAdvancedSession) RunWithOptions(inputs, outputs []Value, opts *RunOptions) error

type DynamicSession 

type DynamicSession[In TensorData, Out TensorData] struct {
	// contains filtered or unexported fields
}

DEPRECATED: See the notes on Session[T]. Use DynamicAdvancedSession instead.

func NewDynamicSession 

func NewDynamicSession[in TensorData, out TensorData](onnxFilePath string,
	inputNames, outputNames []string) (*DynamicSession[in, out], error)

DEPRECATED: See the notes on Session[T]. Use NewDynamicAdvancedSession instead.

func NewDynamicSessionWithONNXData 

func NewDynamicSessionWithONNXData[in TensorData, out TensorData](onnxData []byte,
	inputNames, outputNames []string) (*DynamicSession[in, out], error)

DEPRECATED: See the notes on Session[T]. Use NewDynamicAdvancedSessionWithONNXData instead.

func (*DynamicSession[_, _]) Destroy 

func (s *DynamicSession[_, _]) Destroy() error

func (*DynamicSession[in, out]) Run 

func (s *DynamicSession[in, out]) Run(inputs []*Tensor[in],
	outputs []*Tensor[out]) error

type EnvironmentOption 

type EnvironmentOption func(*C.OrtEnv) *C.OrtStatus

EnvironmentOption is a functional option that can be provided during initialization of an ORT Environment.

func WithLogLevelError 

func WithLogLevelError() EnvironmentOption

WithLogLevelError is an EnvironmentOption that will set the ORT Environment logging to emit error messages along with all messages of greater severity. This is the default logging level.

func WithLogLevelFatal 

func WithLogLevelFatal() EnvironmentOption

WithLogLevelFatal is an EnvironmentOption that will set the ORT Environment logging to emit only fatal error messages (most severe).

func WithLogLevelInfo 

func WithLogLevelInfo() EnvironmentOption

WithLogLevelInfo is an EnvironmentOption that will set the ORT Environment logging to emit informational messages along with all messages of greater severity.

func WithLogLevelVerbose 

func WithLogLevelVerbose() EnvironmentOption

WithLogLevelVerbose is an EnvironmentOption that will set the ORT Environment logging to emit verbose informational messages (least severe) along with all messages of greater severity.

func WithLogLevelWarning 

func WithLogLevelWarning() EnvironmentOption

WithLogLevelWarning is an EnvironmentOption that will set the ORT Environment logging to emit warning messages along with all messages of greater severity.

type ExecutionMode 

type ExecutionMode int

Wraps the ExecutionMode enum in C.

func (ExecutionMode) String 

func (m ExecutionMode) String() string

type FloatData 

type FloatData interface {
	~float32 | ~float64
}

type GraphOptimizationLevel 

type GraphOptimizationLevel int

Wraps the GraphOptimizationLevel enum in C.

func (GraphOptimizationLevel) String 

func (l GraphOptimizationLevel) String() string

type InMemoryQuotaTracker 

type InMemoryQuotaTracker struct {
	// contains filtered or unexported fields
}

InMemoryQuotaTracker 内存配额追踪器

func NewInMemoryQuotaTracker 

func NewInMemoryQuotaTracker() *InMemoryQuotaTracker

NewInMemoryQuotaTracker 创建内存配额追踪器

func (*InMemoryQuotaTracker) GetCount 

func (t *InMemoryQuotaTracker) GetCount(moduleName string) int

func (*InMemoryQuotaTracker) Increment 

func (t *InMemoryQuotaTracker) Increment(moduleName string) (int, error)

func (*InMemoryQuotaTracker) Reset 

func (t *InMemoryQuotaTracker) Reset(moduleName string)

type InputOutputInfo 

type InputOutputInfo struct {
	// The name of the input or output
	Name string
	// The higher-level "type" of the output; whether it's a tensor, sequence,
	// map, etc.
	OrtValueType ONNXType
	// The input or output's dimensions, if it's a tensor. This should be
	// ignored for non-tensor types.
	Dimensions Shape
	// The type of element in the input or output, if it's a tensor. This
	// should be ignored for non-tensor types.
	DataType TensorElementDataType
}

Holds information about the name, shape, and type of an input or output to a ONNX network.

func (*InputOutputInfo) String 

func (n *InputOutputInfo) String() string

type IntData 

type IntData interface {
	~int8 | ~uint8 | ~int16 | ~uint16 | ~int32 | ~uint32 | ~int64 | ~uint64
}

type IoBinding 

type IoBinding struct {
	// contains filtered or unexported fields
}

Wraps the OrtIoBinding instance. Must be created using DynamicAdvancedSession's CreateIoBinding method and Destroy'ed when no longer needed. (Only DynamicAdvancedSession is supported for this, since a regular AdvancedSession requires specifying input and output tensors at session creation time.)

func (*IoBinding) BindInput 

func (b *IoBinding) BindInput(name string, value Value) error

Binds a value to the named input, to be used when RunWithBinding is called.

func (*IoBinding) BindOutput 

func (b *IoBinding) BindOutput(name string, value Value) error

Binds a value to the named output, to be used when RunWithBinding is called.

func (*IoBinding) ClearBoundInputs 

func (b *IoBinding) ClearBoundInputs()

Clears any previously set inputs. Can't cause errors in the ORT C API.

func (*IoBinding) ClearBoundOutputs 

func (b *IoBinding) ClearBoundOutputs()

Clears any previously set outputs. Can't cause errors in the ORT C API.

func (*IoBinding) Destroy 

func (b *IoBinding) Destroy() error

Must be called to free resources associated with the IoBinding once it's no longer needed.

func (*IoBinding) GetBoundOutputNames 

func (b *IoBinding) GetBoundOutputNames() ([]string, error)

Returns a list of bound output names, which will be returned in the same order that outputs will be returned when GetBoundOutputValues is called.

func (*IoBinding) GetBoundOutputValues 

func (b *IoBinding) GetBoundOutputValues() ([]Value, error)

Returns a list of Values containing results of a model run using RunWithBinding. The returned slice contains the same number of values as the number of names returned by GetOutputNames, and/or in the same order as they were bound using IoBinding.BindOutput. IMPORTANT: Each Value returned by this function must be freed by the caller; they are _copies_.

Note: Using this function will cause Tensor contents to be copied from C-managed to Go-managed memory to avoid leaks (this is similar to behavior when DynamicAdvancedSession.Run is allowed to automatically allocate output tensors). Note that this may be expensive for larger tensors.

type LoggingLevel 

type LoggingLevel int

Wraps the OrtLoggingLevel enum

func (LoggingLevel) String 

func (l LoggingLevel) String() string

type Map 

type Map struct {
	// contains filtered or unexported fields
}

This wraps an ONNX_TYPE_MAP OrtValue. Satisfies the Value interface, though Tensor-related functions such as ZeroContents() may be no-ops.

func NewMap 

func NewMap(keys, values Value) (*Map, error)

Creates a new ONNX map that maps the given keys tensor to the given values tensor. Destroying the Map created by this function does _not_ destroy these keys and values tensors; the caller is still responsible for destroying them.

Internally, creating a Map requires two tensors of the same length, and with constraints on type. For example, keys are not allowed to be floats (at least currently). (At the time of writing, this has only been confirmed to work with int64 keys.) There may be many other constraints enforced by the underlying C API.

func NewMapFromGoMap 

func NewMapFromGoMap[K, V TensorData](m map[K]V) (*Map, error)

Wraps the creation of an ONNX map from a Go map. K is the key type, and V is the value type. Be aware that constraints on these types exist based on what ONNX supports. See the comment on NewMap.

func (*Map) DataType 

func (m *Map) DataType() C.ONNXTensorElementDataType

As with a Sequence, this always returns the undefined data type and is only present for compatibility with the Value interface.

func (*Map) Destroy 

func (m *Map) Destroy() error

func (*Map) GetInternals 

func (m *Map) GetInternals() *ValueInternalData

func (*Map) GetKeysAndValues 

func (m *Map) GetKeysAndValues() (Value, Value, error)

Returns two Tensors containing the keys and values, respectively. These tensors should _not_ be Destroyed by users; they will be automatically cleaned up when m.Destroy() is called. These are _not_ the same Value instances that were passed to NewMap, and these should not be modified by users.

func (*Map) GetONNXType 

func (m *Map) GetONNXType() ONNXType

Always returns ONNXTypeMap

func (*Map) GetShape 

func (m *Map) GetShape() Shape

Returns the shape of the map's keys Tensor. Essentially, this can be used to determine the number of key/value pairs in the map.

func (*Map) ZeroContents 

func (m *Map) ZeroContents()

As with Sequence.ZeroContents(), this is a no-op (at least for now), and is only present for compatibility with the Value interface.

type ModelAuthorizationInfo 

type ModelAuthorizationInfo struct {
	// ModuleName 模块名称
	ModuleName string `json:"module_name"`

	// Salt 密钥派生盐值（Base64编码）
	Salt []byte `json:"salt"`

	// ModelHash 模型文件哈希（用于完整性验证）
	ModelHash []byte `json:"model_hash,omitempty"`

	// InputNames 输入名称列表
	InputNames []string `json:"input_names"`

	// OutputNames 输出名称列表
	OutputNames []string `json:"output_names"`

	// Description 模型描述
	Description string `json:"description,omitempty"`

	// Version 模型版本
	Version string `json:"version,omitempty"`
}

ModelAuthorizationInfo 模型授权信息（用于分发）

func (*ModelAuthorizationInfo) ValidateModelHash 

func (info *ModelAuthorizationInfo) ValidateModelHash(encryptedData []byte) bool

ValidateModelHash 验证模型文件哈希

type ModelMetadata 

type ModelMetadata struct {
	// contains filtered or unexported fields
}

A wrapper around the OrtModelMetadata C struct. Must be freed by calling Destroy() on it when it's no longer needed.

func GetModelMetadata 

func GetModelMetadata(path string) (*ModelMetadata, error)

Takes a path to a .onnx file and returns the ModelMetadata associated with it. The returned metadata must be freed using its Destroy() function when it's no longer needed. InitializeEnvironment() must be called before using this function.

Warning: This function loads the onnx content into a temporary onnxruntime session, so it may be computationally expensive.

func GetModelMetadataWithONNXData 

func GetModelMetadataWithONNXData(data []byte) (*ModelMetadata, error)

Identical in behavior to GetModelMetadata, but takes a slice of bytes containing the .onnx network rather than a file path.

func GetModelMetadataWithOptions 

func GetModelMetadataWithOptions(path string,
	options *SessionOptions) (*ModelMetadata, error)

Identical in behavior to GetModelMetadata, but addtionally takes session options to handle models that require options to load.

func (*ModelMetadata) Destroy 

func (m *ModelMetadata) Destroy() error

Frees internal state required by the model metadata. Users are responsible for calling this on any ModelMetadata instance after it's no longer needed.

func (*ModelMetadata) GetCustomMetadataMapKeys 

func (m *ModelMetadata) GetCustomMetadataMapKeys() ([]string, error)

Returns a list of keys that are present in the custom metadata map. Returns an empty slice or nil if no keys are in the map.

NOTE: It is unclear from the docs whether an empty custom metadata map will cause the underlying C function to return an error along with a NULL list, or whether it will only return a NULL list with no error.

func (*ModelMetadata) GetDescription 

func (m *ModelMetadata) GetDescription() (string, error)

Returns the description associated with the model metadata, or an error if the description can't be obtained.

func (*ModelMetadata) GetDomain 

func (m *ModelMetadata) GetDomain() (string, error)

Returns the domain associated with the model metadata, or an error if the domain can't be obtained.

func (*ModelMetadata) GetGraphName 

func (m *ModelMetadata) GetGraphName() (string, error)

Returns the graph name associated with the model metadata, or an error if the name can't be obtained.

func (*ModelMetadata) GetProducerName 

func (m *ModelMetadata) GetProducerName() (string, error)

Returns the producer name associated with the model metadata, or an error if the name can't be obtained.

func (*ModelMetadata) GetVersion 

func (m *ModelMetadata) GetVersion() (int64, error)

Returns the version number in the model metadata, or an error if one occurs.

func (*ModelMetadata) LookupCustomMetadataMap 

func (m *ModelMetadata) LookupCustomMetadataMap(key string) (string, bool, error)

Looks up and returns the string associated with the given key in the custom metadata map. Returns a blank string and 'false' if the key isn't in the map. (A key that's in the map but set to a blank string will return "" and true instead.)

NOTE: It is unclear from the onnxruntime documentation for this function whether an error will be returned if the key isn't present. At the time of writing (1.17.1) the docs only state that no value is returned, not whether an error occurs.

type ONNXType 

type ONNXType int

Wraps the ONNXType enum in C.

func (ONNXType) String 

func (t ONNXType) String() string

type QuotaTracker 

type QuotaTracker interface {
	// Increment 增加使用计数，返回当前计数
	Increment(moduleName string) (int, error)
	// GetCount 获取当前计数
	GetCount(moduleName string) int
	// Reset 重置计数
	Reset(moduleName string)
}

QuotaTracker 配额追踪接口

type RunOptions 

type RunOptions struct {
	// contains filtered or unexported fields
}

RunOptions wraps OrtRunOptions for per-run settings and cancellation.

func NewRunOptions 

func NewRunOptions() (*RunOptions, error)

NewRunOptions creates a new OrtRunOptions instance. Must be closed when no longer needed.

func (*RunOptions) Destroy 

func (o *RunOptions) Destroy() error

Destroy releases the underlying OrtRunOptions.

func (*RunOptions) Terminate 

func (o *RunOptions) Terminate() error

Terminate sets the terminate flag so any ongoing Run using this RunOptions fails quickly.

func (*RunOptions) UnsetTerminate 

func (o *RunOptions) UnsetTerminate() error

UnsetTerminate clears the terminate flag so this RunOptions can be reused.

type Scalar 

type Scalar[T TensorData] struct {
	// contains filtered or unexported fields
}

Scalar is like a tensor but the underlying go slice is of length 1 and it has no dimension. It was introduced for use with the training API, but remains supported since it may be useful apart from the training API.

func NewEmptyScalar 

func NewEmptyScalar[T TensorData]() (*Scalar[T], error)

NewEmptyScalar creates a new scalar of type T.

func NewScalar 

func NewScalar[T TensorData](data T) (*Scalar[T], error)

NewScalar creates a new scalar of type T backed by a value of type T. Note that, differently from tensors, this is not a []T but just a value T.

func (*Scalar[T]) DataType 

func (t *Scalar[T]) DataType() C.ONNXTensorElementDataType

func (*Scalar[T]) Destroy 

func (s *Scalar[T]) Destroy() error

func (*Scalar[T]) GetData 

func (t *Scalar[T]) GetData() T

GetData returns the undelying data for the scalar. If you want to set the scalar's data, use Set.

func (*Scalar[_]) GetInternals 

func (t *Scalar[_]) GetInternals() *ValueInternalData

func (*Scalar[_]) GetONNXType 

func (t *Scalar[_]) GetONNXType() ONNXType

func (*Scalar[T]) GetShape 

func (s *Scalar[T]) GetShape() Shape

Always returns nil for Scalars.

func (*Scalar[T]) Set 

func (t *Scalar[T]) Set(value T)

Changes the underlying value of the scalar to the new value.

func (*Scalar[T]) ZeroContents 

func (s *Scalar[T]) ZeroContents()

type SecurityChecker 

type SecurityChecker interface {
	// Check 执行安全检查（反调试、虚拟机检测等）
	Check() error
}

SecurityChecker 安全检查接口（与 machineid/cert 兼容）

type Sequence 

type Sequence struct {
	// contains filtered or unexported fields
}

This wraps an ONNX_TYPE_SEQUENCE OrtValue. Satisfies the Value interface, though Tensor-related functions such as ZeroContents() may be no-ops.

func NewSequence 

func NewSequence(contents []Value) (*Sequence, error)

Creates a new ONNX sequence with the given contents. The returned Sequence must be Destroyed by the caller when no longer needed. Destroying the Sequence created by this function does _not_ destroy the Values it was created with, so the caller is still responsible for destroying them as well.

The contents of a sequence are subject to additional constraints. I can't find mention of some of these in the C API docs, but they are enforced by the onnxruntime API. Notably: all elements of the sequence must have the same type, and all elements must be either maps or tensors. Finally, the sequence must contain at least one element, and none of the elements may be nil. There may be other constraints that I am unaware of, as well.

func (*Sequence) DataType 

func (s *Sequence) DataType() C.ONNXTensorElementDataType

This function is meaningless for a Sequence and shouldn't be used. The return value is always TENSOR_ELEMENT_DATA_TYPE_UNDEFINED for now, but this may change in the future. This function is only present for compatibility with the Value interface and should not be relied on for sequences.

func (*Sequence) Destroy 

func (s *Sequence) Destroy() error

func (*Sequence) GetInternals 

func (s *Sequence) GetInternals() *ValueInternalData

func (*Sequence) GetONNXType 

func (s *Sequence) GetONNXType() ONNXType

Always returns ONNXTypeSequence

func (*Sequence) GetShape 

func (s *Sequence) GetShape() Shape

This returns a 1-dimensional Shape containing a single element: the number of elements the sequence. Typically, Sequence users should prefer calling len(s.GetValues()) over this function. This function only exists to maintain compatibility with the Value interface.

func (*Sequence) GetValues 

func (s *Sequence) GetValues() ([]Value, error)

Returns the list of values in the sequence. Each of these values should _not_ be Destroy()'ed by the caller, they will be automatically destroyed upon calling Destroy() on the sequence. If this sequence was created via NewSequence, these are not the same Values that the sequence was created with, though if they are tensors they should still refer to the same underlying data.

func (*Sequence) ZeroContents 

func (s *Sequence) ZeroContents()

This function does nothing for a Sequence, and is only present for compatibility with the Value interface.

type Session 

type Session[T TensorData] struct {
	// contains filtered or unexported fields
}

DEPRECATED: This type was written with a type parameter despite the fact that a type parameter is not necessary for any of its underlying implementation. It is preserved only for compatibility with older code, and new users should use AdvancedSession instead. Despite the name, AdvancedSession is equally simple to use and far more flexible.

func NewSession 

func NewSession[T TensorData](onnxFilePath string, inputNames,
	outputNames []string, inputs, outputs []*Tensor[T]) (*Session[T], error)

DEPRECATED: See the notes on Session[T]. Use NewAdvancedSession instead.

func NewSessionWithONNXData 

func NewSessionWithONNXData[T TensorData](onnxData []byte, inputNames,
	outputNames []string, inputs, outputs []*Tensor[T]) (*Session[T], error)

DEPRECATED: See the notes on Session[T]. Use NewAdvancedSessionWithONNXData instead.

func (*Session[_]) Destroy 

func (s *Session[_]) Destroy() error

func (*Session[T]) Run 

func (s *Session[T]) Run() error

type SessionOptions 

type SessionOptions struct {
	// contains filtered or unexported fields
}

Used to set options when creating an ONNXRuntime session. There is currently not a way to change options after the session is created, apart from destroying the session and creating a new one. This struct opaquely wraps a C OrtSessionOptions struct, which users must modify via function calls. (The OrtSessionOptions struct is opaque in the C API, too.)

Users must instantiate this struct using the NewSessionOptions function. Instances must be destroyed by calling the Destroy() method after the options are no longer needed (after NewAdvancedSession(...) has returned).

func NewSessionOptions 

func NewSessionOptions() (*SessionOptions, error)

Initializes and returns a SessionOptions struct, used when setting options in new AdvancedSession instances. The caller must call the Destroy() function on the returned struct when it's no longer needed.

func (*SessionOptions) AddSessionConfigEntry 

func (o *SessionOptions) AddSessionConfigEntry(key, value string) error

Sets a session configuration key to the given value. See the onnxruntime_session_options_config_keys.h file in the onnxruntime sources for documentation on valid keys and values. If the key was already set, this will overwrite its old setting with the given value.

func (*SessionOptions) AppendExecutionProvider 

func (o *SessionOptions) AppendExecutionProvider(providerName string,
	options map[string]string) error

Wraps the AppendExecutionProvider onnxruntime C API function. See the documentation for that function for a list of all names and supported options.

func (*SessionOptions) AppendExecutionProviderCUDA 

func (o *SessionOptions) AppendExecutionProviderCUDA(
	cudaOptions *CUDAProviderOptions) error

Takes a pointer to an initialized CUDAProviderOptions instance, and applies them to the session options. This is what you'll need to call if you want the session to use CUDA. Returns an error if your device (or onnxruntime library) does not support CUDA. The CUDAProviderOptions struct can be destroyed after this.

func (*SessionOptions) AppendExecutionProviderCoreML 

func (o *SessionOptions) AppendExecutionProviderCoreML(flags uint32) error

Enables the CoreML backend for the given session options on supported platforms. The meanings of the flag bits are currently defined in the coreml_provider_factory.h file which is provided in the include/ directory of the onnxruntime releases for Apple platforms. AppendExecutionProviderCoreML is now deprecated. Please use AppendExecutionProviderCoreMLV2 instead. See: https://onnxruntime.ai/docs/execution-providers/CoreML-ExecutionProvider.html

func (*SessionOptions) AppendExecutionProviderCoreMLV2 

func (o *SessionOptions) AppendExecutionProviderCoreMLV2(options map[string]string) error

AppendExecutionProviderCoreMLV2 is the new API for adding CoreML provider to ONNX Runtime. This is the recommended way to add CoreML provider as of ONNX Runtime 1.20.0.

For CoreML options, see: https://onnxruntime.ai/docs/execution-providers/CoreML-ExecutionProvider.html

func (*SessionOptions) AppendExecutionProviderDirectML 

func (o *SessionOptions) AppendExecutionProviderDirectML(deviceID int) error

Enables the DirectML backend for the given session options on supported platforms. See the notes on device_id in coreml_provider_factory.h in the onnxruntime source code, but a device ID of 0 should correspond to the default device, "which is typically the primary display GPU" according to the docs.

func (*SessionOptions) AppendExecutionProviderOpenVINO 

func (o *SessionOptions) AppendExecutionProviderOpenVINO(
	options map[string]string) error

Enables the OpenVINO backend for the given session options on supported platforms. See https://onnxruntime.ai/docs/execution-providers/OpenVINO-ExecutionProvider.html#summary-of-options for a list of supported keys and values that can be passed in the options map.

func (*SessionOptions) AppendExecutionProviderTensorRT 

func (o *SessionOptions) AppendExecutionProviderTensorRT(
	tensorRTOptions *TensorRTProviderOptions) error

Takes an initialized TensorRTProviderOptions instance, and applies them to the session options. You'll need to call this if you want the session to use TensorRT. Returns an error if your device (or onnxruntime library version) does not support TensorRT. The TensorRTProviderOptions can be destroyed after this.

func (*SessionOptions) Destroy 

func (o *SessionOptions) Destroy() error

func (*SessionOptions) GetSessionConfigEntry 

func (o *SessionOptions) GetSessionConfigEntry(key string) (string, error)

Returns the session config entry corresponding to the given key, or an error if one occurs. Returns an error if the key doesn't exist, so it may be cheaper to check for the key with HasSessionConfigEntry first. See also AddSessionConfigEntry.

func (*SessionOptions) HasSessionConfigEntry 

func (o *SessionOptions) HasSessionConfigEntry(key string) (bool, error)

Returns true, nil if the SessionOptions has a configuration entry with the given key. Returns false if the key isn't defined. Returns an error if onnxruntime indicates an error, though it isn't clear from the docs what may cause an error to occur here. See also GetSessionConfigEntry and AddSessionConfigEntry.

func (*SessionOptions) SetCpuMemArena 

func (o *SessionOptions) SetCpuMemArena(isEnabled bool) error

Enable/Disable the usage of the memory arena on CPU. Arena may pre-allocate memory for future usage.

func (*SessionOptions) SetExecutionMode 

func (o *SessionOptions) SetExecutionMode(newMode ExecutionMode) error

Sets the session's execution mode. The newMode must be ExecutionModeSequential or ExecutionModeParallel.

func (*SessionOptions) SetGraphOptimizationLevel 

func (o *SessionOptions) SetGraphOptimizationLevel(
	level GraphOptimizationLevel) error

Sets the optimization level to apply when loading a graph. Refer to the C API documentation for SetSessionGraphOptimizationLevel.

func (*SessionOptions) SetInterOpNumThreads 

func (o *SessionOptions) SetInterOpNumThreads(n int) error

Sets the number of threads used to parallelize execution across separate onnxruntime graph nodes. A value of 0 uses the default number of threads.

func (*SessionOptions) SetIntraOpNumThreads 

func (o *SessionOptions) SetIntraOpNumThreads(n int) error

Sets the number of threads used to parallelize execution within onnxruntime graph nodes. A value of 0 uses the default number of threads.

func (*SessionOptions) SetLogSeverityLevel 

func (o *SessionOptions) SetLogSeverityLevel(level LoggingLevel) error

Sets the sessions log severity level. Must be one of LoggingLevelVerbose, LoggingLevelInfo, LoggingLevelWarning, LoggingLevelError, or LoggingLevelFatal.

func (*SessionOptions) SetMemPattern 

func (o *SessionOptions) SetMemPattern(isEnabled bool) error

Enable/Disable the memory pattern optimization. If this is enabled memory is preallocated if all shapes are known.

type Shape 

type Shape []int64

The Shape type holds the shape of the tensors used by the network input and outputs.

func NewShape 

func NewShape(dimensions ...int64) Shape

Returns a Shape, with the given dimensions.

func (Shape) Clone 

func (s Shape) Clone() Shape

Makes and returns a deep copy of the Shape.

func (Shape) Equals 

func (s Shape) Equals(other Shape) bool

Returns true if both shapes match in every dimension.

func (Shape) FlattenedSize 

func (s Shape) FlattenedSize() int64

Returns the total number of elements in a tensor with the given shape. Note that this may be an invalid value due to overflow or negative dimensions. If a shape comes from an untrusted source, it may be a good practice to call Validate() prior to trusting the FlattenedSize.

func (Shape) String 

func (s Shape) String() string

func (Shape) Validate 

func (s Shape) Validate() error

Returns a non-nil error if the shape has bad or zero dimensions. May return a ZeroShapeLengthError, a ShapeOverflowError, or a BadShapeDimensionError. In the future, this may return other types of errors if it others become necessary.

type StringTensor 

type StringTensor struct {
	// contains filtered or unexported fields
}

This represents an onnxruntime tensor containing strings. This still satisfies the Value interface, but has several important differences with Tensor[T] instances for numerical values. Most notably, StringTensor.GetContents() returns a _copy_ of the tensor's contents, and modifying these strings will not modify the contents of the underlying tensor. Instead, users must use StringTensor.SetElement(...) or StringTensor.SetContents(...) to modify the contents of an existing string tensor.

func NewStringTensor 

func NewStringTensor(shape Shape) (*StringTensor, error)

Creates and returns a string tensor. The contents are _not_ initialized yet, and must be initialized using SetContents or, less efficiently, using SetElement to set each string individually. As with all Values, StringTensors must be freed using Destroy() when no longer needed.

func (*StringTensor) DataType 

func (t *StringTensor) DataType() C.ONNXTensorElementDataType

Always returns C.ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING

func (*StringTensor) Destroy 

func (t *StringTensor) Destroy() error

func (*StringTensor) GetContents 

func (t *StringTensor) GetContents() ([]string, error)

Returns all contents of the string tensor, in order by flattened index. If you need to get all contents in the tensor, this should be more efficient than using GetElement(...) to retrieve all strings individually. This copies the tensor's contents; modifying the returned slice will not modify the tensor.

func (*StringTensor) GetElement 

func (t *StringTensor) GetElement(index int64) (string, error)

Returns a single string from t, at the given flattened index.

func (*StringTensor) GetInternals 

func (t *StringTensor) GetInternals() *ValueInternalData

func (*StringTensor) GetONNXType 

func (t *StringTensor) GetONNXType() ONNXType

func (*StringTensor) GetShape 

func (t *StringTensor) GetShape() Shape

func (*StringTensor) SetContents 

func (t *StringTensor) SetContents(contents []string) error

This sets all of the strings in t to the contents from the flattened slice of strings. The length of the contents slice must match t.GetShape().FlattenedSize(), otherwise this will return an error.

func (*StringTensor) SetElement 

func (t *StringTensor) SetElement(index int64, s string) error

Sets the string at the flattened index in t to s.

func (*StringTensor) ZeroContents 

func (t *StringTensor) ZeroContents()

ZeroContents() is unsupported for string tensors. To clear all strings, use err := t.SetContents(make([]string, t.GetShape().FlattenedSize())) instead.

type Tensor 

type Tensor[T TensorData] struct {
	// contains filtered or unexported fields
}

Used to manage all input and output data for onnxruntime networks. A Tensor always has an associated type and refers to data contained in an underlying Go slice. New tensors should be created using the NewTensor or NewEmptyTensor functions, and must be destroyed using the Destroy function when no longer needed.

func NewEmptyTensor 

func NewEmptyTensor[T TensorData](s Shape) (*Tensor[T], error)

Creates a new empty tensor with the given shape. The shape provided to this function is copied, and is no longer needed after this function returns.

func NewTensor 

func NewTensor[T TensorData](s Shape, data []T) (*Tensor[T], error)

Creates a new tensor backed by an existing data slice. The shape provided to this function is copied, and is no longer needed after this function returns. If the data slice is longer than s.FlattenedSize(), then only the first portion of the data will be used.

func (*Tensor[T]) Clone 

func (t *Tensor[T]) Clone() (*Tensor[T], error)

Makes a deep copy of the tensor, including its ONNXRuntime value. The Tensor returned by this function must be destroyed when no longer needed. The returned tensor will also no longer refer to the same underlying data; use GetData() to obtain the new underlying slice.

func (*Tensor[T]) DataType 

func (t *Tensor[T]) DataType() C.ONNXTensorElementDataType

Returns the value from the ONNXTensorElementDataType C enum corresponding to the type of data held by this tensor.

NOTE: This function was added prior to the introduction of the Go TensorElementDataType int wrapping the C enum, so it still returns the CGo type.

func (*Tensor[_]) Destroy 

func (t *Tensor[_]) Destroy() error

Cleans up and frees the memory associated with this tensor.

func (*Tensor[T]) GetData 

func (t *Tensor[T]) GetData() []T

Returns the slice containing the tensor's underlying data. The contents of the slice can be read or written to get or set the tensor's contents.

func (*Tensor[_]) GetInternals 

func (t *Tensor[_]) GetInternals() *ValueInternalData

func (*Tensor[_]) GetONNXType 

func (t *Tensor[_]) GetONNXType() ONNXType

Always returns ONNXTypeTensor for any Tensor[T] even if the underlying tensor is invalid for some reason.

func (*Tensor[_]) GetShape 

func (t *Tensor[_]) GetShape() Shape

Returns the shape of the tensor. The returned shape is only a copy; modifying this does *not* change the shape of the underlying tensor. (Modifying the tensor's shape can only be accomplished by Destroying and recreating the tensor with the same data.)

func (*Tensor[T]) ZeroContents 

func (t *Tensor[T]) ZeroContents()

Sets every element in the tensor's underlying data slice to 0.

type TensorData 

type TensorData interface {
	FloatData | IntData | ~bool
}

This is used as a type constraint for the generic Tensor type.

type TensorElementDataType 

type TensorElementDataType int

Wraps the ONNXTEnsorElementDataType enum in C.

func (TensorElementDataType) String 

func (t TensorElementDataType) String() string

type TensorInternalData 

type TensorInternalData = ValueInternalData

As with the ArbitraryTensor type, this type alias only exists to facilitate renaming an old type without breaking existing code.

type TensorRTProviderOptions 

type TensorRTProviderOptions struct {
	// contains filtered or unexported fields
}

Like the CUDAProviderOptions struct, but used for configuring TensorRT options. Instances of this struct must be initialized using NewTensorRTProviderOptions() and cleaned up by calling their Destroy() function when they are no longer needed.

func NewTensorRTProviderOptions 

func NewTensorRTProviderOptions() (*TensorRTProviderOptions, error)

Initializes and returns a TensorRTProviderOptions struct, used when enabling the TensorRT backend. The caller must call the Destroy() function on the returned struct when it's no longer needed.

func (*TensorRTProviderOptions) Destroy 

func (o *TensorRTProviderOptions) Destroy() error

Must be called when the TensorRTProviderOptions are no longer needed, in order to free internal state. The struct is not needed as soon as you have passed it to the AppendExecutionProviderTensorRT function.

func (*TensorRTProviderOptions) Update 

func (o *TensorRTProviderOptions) Update(options map[string]string) error

Wraps the call to the UpdateTensorRTProviderOptions in the C API. Requires a map of string keys to values.

The onnxruntime headers refer users to https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html#cc for the list of available keys and values.

type TrainingInputOutputNames 

type TrainingInputOutputNames struct {
	TrainingInputNames  []string
	EvalInputNames      []string
	TrainingOutputNames []string
	EvalOutputNames     []string
}

Support for TrainingInputOutputNames has been removed from onnxruntime_go following the deprecation of the training API in onnxruntime 1.20.0.

func GetInputOutputNames 

func GetInputOutputNames(checkpointStatePath string, trainingModelPath string,
	evalModelPath string) (*TrainingInputOutputNames, error)

Always returns (nil, TrainingAPIRemovedError).

type TrainingSession 

type TrainingSession struct{}

Support for TrainingSessions has been removed from onnxruntime_go following the deprecation of the training API in onnxruntime 1.20.0.

func NewTrainingSession 

func NewTrainingSession(checkpointStatePath, trainingModelPath, evalModelPath,
	optimizerModelPath string, inputs, outputs []Value,
	options *SessionOptions) (*TrainingSession, error)

Always returns (nil, TrainingAPIRemovedError).

func NewTrainingSessionWithOnnxData 

func NewTrainingSessionWithOnnxData(checkpointData, trainingData, evalData,
	optimizerData []byte, inputs, outputs []Value,
	options *SessionOptions) (*TrainingSession, error)

Always returns (nil, TrainingAPIRemovedError).

func (*TrainingSession) Destroy 

func (s *TrainingSession) Destroy() error

Always returns TrainingAPIRemovedError.

func (*TrainingSession) ExportModel 

func (s *TrainingSession) ExportModel(path string, outputNames []string) error

Always returns TrainingAPIRemovedError.

func (*TrainingSession) LazyResetGrad 

func (s *TrainingSession) LazyResetGrad() error

Always returns TrainingAPIRemovedError.

func (*TrainingSession) OptimizerStep 

func (s *TrainingSession) OptimizerStep() error

Always returns TrainingAPIRemovedError.

func (*TrainingSession) SaveCheckpoint 

func (s *TrainingSession) SaveCheckpoint(path string,
	saveOptimizerState bool) error

Always returns TrainingAPIRemovedError.

func (*TrainingSession) TrainStep 

func (s *TrainingSession) TrainStep() error

Always returns TrainingAPIRemovedError.

type Value 

type Value interface {
	DataType() C.ONNXTensorElementDataType
	GetShape() Shape
	Destroy() error
	GetInternals() *ValueInternalData
	ZeroContents()
	GetONNXType() ONNXType
}

An interface for managing tensors or other onnxruntime values where we don't necessarily need to access the underlying data slice. All typed tensors will support this interface regardless of the underlying data type.

type ValueInternalData 

type ValueInternalData struct {
	// contains filtered or unexported fields
}

This wraps internal implementation details to avoid exposing them to users via the Value interface.

Source Files

  • authorized_session.go
  • crypto.go
  • legacy_code.go
  • onnxruntime_go.go
  • setup_env.go
  • setup_env_dynamic.go
  • tensor_type_constraints.go

Directories

Path Synopsis
授权模型推理示例 展示如何将 onnxruntime_go 与 machineid/cert 集成
 授权模型推理示例 展示如何将 onnxruntime_go 与 machineid/cert 集成

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.