You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Go] Update to the current C API and add plugin execution providers - #33282
Brings the Go binding up to the current C API and adds plugin execution provider support.
Version handling
The vendored headers are now exact copies of include/onnxruntime/core/session/ (API 32).
Init requests the API version that matches the loaded runtime and falls back to API 17. It does not probe from the top, because every rejected GetApi call prints to stderr.
Each new wrapper returns a typed error when the runtime is too old for it.
SetDeterministicCompute, IOBinding.SynchronizeInputs / SynchronizeOutputs, GetBuildInfo, error codes 12 to 15
10 to 17
Workaround for #33281: cloned options and EP selection policies drop the weightless buffer. The binding restores the buffer on every clone and rejects a selection policy combined with a buffer, so explicit device selection is the supported path.
Session creation: sessions created from the same options are serialized. During creation, mutations return an error and Close defers release, so legacy factories such as XNNPACK's never read freed options.
Test harness: the tests used to exit 0 without running anything when no runtime was set. They now fail.
The change is split into eight commits that each build and pass on their own:
Require a runtime in the test harness
Synchronize C headers and negotiate the runtime API version
Add plugin execution provider loading and device selection
Add inference controls and the weightless source model buffer
Add EPContext data reader callbacks
Test plugin execution providers on real plugins
Parse go env CC with the toolchain's quoting rules in tests
Keep source options valid while a session is created
The binding was written against API 27 and could not use anything newer.
This PR touches 23 files: three are exact header copies, one is a generated test model and two are test-only bridges (-tags=orttest). Header sync, version negotiation and plugin and callback lifetimes share code and tests, so I kept them in one PR. Release-archive CI, module tags and the README will be separate PRs.
Testing
Linux x64: go test -race passes against released ORT 1.27.0, 1.28.3, 1.29.1, 1.30.0 and 1.31.0, negotiating API 27 to 31 with nothing written to stderr.
The -tags=orttest suite passes with -race on all five, and with GOEXPERIMENT=cgocheck2 on 1.31.0.
Linux x64 and macOS arm64 (ORT 1.21.0, 1.22.0, 1.23.0, 1.31.1) and Linux arm64 (1.22.0, 1.31.1): go test -race passes, default and -tags=orttest. Runtimes with API 21 or 22 skip the features they lack.
With the WebGPU plugin EP 0.4.0 on an Intel Arc 130V/140V GPU, the session profile shows the model ran on WebGpuExecutionProvider.
gofmt, go vet and golangci-lint are clean.
Not yet covered: the example_plugin_ep and EPContext integration tests, which need a runtime built from the same source; Windows.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The parent test incorrectly fails on API versions 17–21 because the child skips the unsupported feature while the parent requires a PASS marker.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Skip unsupported API versions before parent/child split
go/onnxruntime/ep_test.go:475
With API 17–21 (including a build capped at API 21), the child takes the t.Skip branch below and exits successfully, but prints --- SKIP: TestEPShutdown, not the --- PASS marker required here. The parent therefore fails the suite for an intentionally unsupported feature. Move the API-version skip before the parent/child split so both processes handle it consistently.
The test binary exited zero without running any test when ORT_LIB_PATH
was unset and no fallback library existed, so a missing runtime could
look like a passing suite. TestMain now fails with a setup diagnostic
in that case, every child mode must initialize successfully, and the
profiling test no longer writes to a fixed Unix path. TestMain sets
ORT_RUNNING_UNIT_TESTS before initializing so the runtime's telemetry
does not leave files in the package directory.
Subprocess tests cover an unset runtime, an explicit missing path, an
invalid library and a successful setup.
Signed-off-by: Danny <dannyontheair@outlook.com>
The vendored headers had drifted to API 29 while the canonical headers
are at API 32, and unrelated C API changes had to hand-edit the copies.
The three headers are now exact copies of include/onnxruntime/core/session,
and the binding requests the API version that matches the loaded runtime:
it reads the runtime version string, asks for min(runtime minor, 32) and
falls back down to API 17. Requesting a version the runtime does not
have makes ONNX Runtime print to stderr, so the binding no longer probes
from the top.
Every wrapper now checks the negotiated version before reading a
function pointer and returns a typed error when the runtime is too old.
Adds the error codes 12 to 15 and moves the API 27 option getter checks
into the shim.
Signed-off-by: Danny <dannyontheair@outlook.com>
Execution providers now ship as plugin libraries (for example the CUDA
and WebGPU plugin EPs), but Go callers could only append providers by
name. This adds RegisterExecutionProviderLibrary and
UnregisterExecutionProviderLibrary, device discovery through
GetEpDevices with immutable Go snapshots of the device and hardware
metadata, explicit selection with SessionOptionsAppendExecutionProvider_V2,
and the native selection policies.
Unregistering a library fails while any session or session options
still reference its devices, because the native objects borrow from the
library. All plugin calls require API 22 and return a typed error on
older runtimes.
Signed-off-by: Danny <dannyontheair@outlook.com>
Adds SetDeterministicCompute, explicit I/O binding synchronization and
SessionOptionsSetWeightlessSourceModelBuffer (API 29), which lets a
weightless compiled model take its initializers from memory. The binding
copies the buffer into C memory and keeps it alive for every options
object and session that uses it.
Two native paths drop the buffer on the runtimes released so far:
CloneSessionOptions does not copy it, and automatic EP selection builds
fresh options without it. The binding restores the buffer on every
clone, and rejects combining a buffer with a selection policy, so
explicit device selection is the supported path. Session constructors
work on a private options snapshot so the public options lock is not
held while native code runs.
The orttest build tag adds native checks that read the buffer back from
a clone and from inside a test provider's CreateEp.
Signed-off-by: Danny <dannyontheair@outlook.com>
SessionOptionsSetEpContextDataReadFunc (API 31) lets an application
supply named external EPContext data from memory instead of the file
system. The Go reader receives the data name and returns the bytes,
bounded by a caller-set limit; the binding copies them into memory from
the allocator ONNX Runtime passes in and translates errors and panics
into an OrtStatus. The reader stays alive for every options clone and
session that can call it, and session creation does not hold the
options lock while the reader runs, so the reader may call options
getters or Clone.
Tests drive the callback from native code, including concurrent calls,
calls from a C thread, re-entry from both constructors and a bounded
subprocess timeout, and run under cgocheck2 as well as the race
detector.
Signed-off-by: Danny <dannyontheair@outlook.com>
Adds integration tests that load a real plugin library, select its
device explicitly and run a float32 Mul model, then check the session
profile to prove the nodes ran on the plugin rather than falling back
to CPU. The tests are opt-in through ORT_TEST_WEBGPU_EP_PATH and
ORT_TEST_PLUGIN_EP_PATH, skip when the variable is unset, and fail when
it is set but the plugin cannot be loaded or used.
The tests prove execution on the plugin EP, not which adapter the plugin
chose. Verified with the WebGPU plugin EP 0.4.0 and ONNX Runtime 1.31.0
on an Intel Arc 130V/140V GPU, with the Intel Vulkan driver loaded and no
software Vulkan driver. The example_plugin_ep test needs a runtime built
from the same source and belongs in source-build CI.
Signed-off-by: Danny <dannyontheair@outlook.com>
Two test fixtures split the output of `go env CC` with strings.Fields,
which breaks a compiler path that contains spaces or quoted flags. They
now use a test-only parser that follows cmd/internal/quoted.Split, and
the orttest bridge receives the parsed command from its test callers.
Signed-off-by: Danny <dannyontheair@outlook.com>
CloneSessionOptions shares provider factories, and legacy factories such
as XNNPACK's keep a pointer to the original options. Creating a session
from the snapshot after unlocking the source let a concurrent Close free
memory the factory still reads, and let setters race with it.
Sessions created from the same options are now serialized, as they were
before the snapshot. During creation, mutations return an error, getters
and Clone still work, and Close marks the options closed and releases
them when creation finishes.
Signed-off-by: Danny <dannyontheair@outlook.com>
Hi Tianlei Wu (@tianleiwu), this follows up #29615: the Go binding now matches the current C API and can load plugin EPs (#32535). Could you take a look, or point me to the right reviewer? CI hasn't run yet, so the workflows need approval. Happy to split it into smaller PRs if that helps. Thanks!
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Brings the Go binding up to the current C API and adds plugin execution provider support.
Version handling
include/onnxruntime/core/session/(API 32).Initrequests the API version that matches the loaded runtime and falls back to API 17. It does not probe from the top, because every rejectedGetApicall prints to stderr.New APIs
RegisterExecutionProviderLibrary,UnregisterExecutionProviderLibraryGetEPDevices,EPDevice,HardwareDeviceSessionOptions.AppendExecutionProviderV2,SessionOptions.SetEPSelectionPolicySessionOptions.SetWeightlessSourceModelBufferSessionOptions.SetEPContextDataReader,ClearEPContextDataReaderSetDeterministicCompute,IOBinding.SynchronizeInputs/SynchronizeOutputs,GetBuildInfo, error codes 12 to 15Workaround for #33281: cloned options and EP selection policies drop the weightless buffer. The binding restores the buffer on every clone and rejects a selection policy combined with a buffer, so explicit device selection is the supported path.
Session creation: sessions created from the same options are serialized. During creation, mutations return an error and
Closedefers release, so legacy factories such as XNNPACK's never read freed options.Test harness: the tests used to exit 0 without running anything when no runtime was set. They now fail.
The change is split into eight commits that each build and pass on their own:
Motivation and Context
This PR touches 23 files: three are exact header copies, one is a generated test model and two are test-only bridges (
-tags=orttest). Header sync, version negotiation and plugin and callback lifetimes share code and tests, so I kept them in one PR. Release-archive CI, module tags and the README will be separate PRs.Testing
go test -racepasses against released ORT 1.27.0, 1.28.3, 1.29.1, 1.30.0 and 1.31.0, negotiating API 27 to 31 with nothing written to stderr.-tags=orttestsuite passes with-raceon all five, and withGOEXPERIMENT=cgocheck2on 1.31.0.go test -racepasses, default and-tags=orttest. Runtimes with API 21 or 22 skip the features they lack.WebGpuExecutionProvider.gofmt,go vetandgolangci-lintare clean.Not yet covered: the
example_plugin_epand EPContext integration tests, which need a runtime built from the same source; Windows.