[Copilot]

Description

• Share idle device-stream collections across calling threads only when graph capture is disabled and stream-creating providers register device-selection callbacks. Keep capture and providers without callbacks, including MIGraphX, thread-affine.
• Select reused streams' owning devices during acquisition, before input allocation, as well as before notification creation. This also covers reuse of CUDA's retired per-thread contexts, which do not reselect the device.
• Cover shared pooling, graph-capture affinity, missing-callback affinity, same-thread device restoration, and real GPU-1 asynchronous reuse across caller exit and notification destruction. Verify deferred pinned memory remains allocated while copies are pending and is released after completion.

Local validation

• python3 build/pr33053-cuda/build_focused_session_state_tests.py — passed. The isolated focused runner uses the actual session_state_test.cc and original test main, linked with the original CMake production-library recipe.
• cd build/pr33053-cuda/Release && LD_LIBRARY_PATH="$PWD:/usr/local/cuda/lib64" timeout 120s ./onnxruntime_session_state_focused_test --gtest_filter='SessionStateTest.DeviceStreamPool*' --gtest_repeat=3 — passed all four tests in each of three iterations (12 passes, no skips), including real CUDA asynchronous copies on GPU 1.
• build/pr33053-cuda/tools-venv/bin/clang-format --dry-run --Werror onnxruntime/core/framework/session_state.cc onnxruntime/core/providers/cuda/cuda_stream_handle.cc onnxruntime/test/framework/session_state_test.cc — passed with repository-pinned clang-format 20.1.8.
• git diff --check — passed.

Validation used an isolated CUDA-minimal build with two build jobs and one nvcc worker. An existing unused-parameter warning in the CUDA-minimal per-thread-context constructor was downgraded only in that build's generated CUDA-provider flags. Full CUDA compute kernels/cuDNN, real HIP/MIGraphX hardware, and the full test suite were not tested. The focused build helper and runner are local validation artifacts, not committed repository tooling; the same four tests can be selected from the normal onnxruntime_test_all binary.

Motivation and Context

Since 1.26, thread-affine pooling retains roughly 9.7 MiB of GPU memory per calling thread per session—even when runs never overlap. Long-lived thread-pool hosts can exhaust VRAM without increasing inference concurrency. Reusing exclusively borrowed stream collections bounds retained resources by overlapping runs rather than the number of calling threads.

• Fixes [CUDA] GPU memory grows ~10 MiB per calling thread per session since 1.26 (per-thread device stream pool from #28002) #33037

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Copilot review overview

🟡 Changes recommended

Cross-thread MIGraphX stream reuse can allocate input buffers on the wrong GPU.

Review effort: Balanced
Findings: 1

Open (1)

• MIGraphX shared reuse can allocate on the wrong GPU · New

What changed in this PR

Reduces per-thread GPU memory retention by sharing idle device streams when graph capture is disabled, addressing #33037.

Changes:

• Preserves thread-affine pooling for graph capture.
• Selects stream devices before creating notifications.
• Adds tests for reuse, thread affinity, and overlapping acquisitions.

File	Description
onnxruntime/​test/​framework/​session_state_test.cc	Adds stream-pooling and device-selection tests.
onnxruntime/​core/​framework/​stream_execution_context.cc	Selects devices before notification creation.
onnxruntime/​core/​framework/​session_state.h	Adds pooling-policy state and comments.
onnxruntime/​core/​framework/​session_state.cc	Implements conditional sharing and pruning.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Tianlei Wu (tianleiwu)]

Re-reviewed the shared-pool gating, exclusive collection borrowing, device restoration before allocation/notification creation, and deferred-buffer lifetime tests. The prior MIGraphX/thread-affinity concern remains addressed, and I found no additional actionable issues in the current diff.

The PR records focused validation of the earlier implementation, but there are no reported checks for this merge head. I did not treat an existing build of another revision as current-head validation. Please obtain the current-head checks before merging. No approval submitted.

[Tianlei Wu (tianleiwu)]

Final maintenance verification at 86a03362d7: the complete, freshly paginated all-20 audit at2026-11-09T14:33:58Z verifies this PR's complete current-head CI green, OPEN/non-draft/MERGEABLE state and zero unresolved feedback. The previously pending WebGPU packaging gates have completed. Normal reviewer approval remains external; no merge or self-approval was performed.

The stream-pool main-merge repair and shared React Native lifetime/navigation fixes remain intact. The original AsyncWorker header's real pinned-Hermes/JSI ASan UAF was reproduced when resolve dispatch released the final worker before its post-dispatch completion hook. 89a1d6a0f60deedc4b4b27c0ba81fd744169fcd0 moves completion before dispatch and preserves once-only completion when dispatch throws without whole-thread ownership of JSI values. Its fixed header remains unchanged and passed320 synchronized resolve/reject/cancel/Env-invalidation/dispatch-exception scenarios in each of two independent ASan+LSan processes:640 real executions.

The retained 1727227563024ea357842aa2b072f35a2cd979ef navigation fix assigns the actual native Button a unique testID and uses by.id while preserving accessibility and the exact full14 assertion. Eight actual component/navigation cases and four scoped linters passed. Earlier actual-import App checking introduced no diagnostics; three pre-existing MNIST diagnostics were explicitly unchanged. This is not a claim of an unrelated clean full MNIST type-check.

The Windows CPUINFO repair changes only two regression-test files. It checks allocation cleanup after CPUIDInfo destruction and before later CRT/loader address reuse, repeats16 unload cycles and preserves process-lifetime result storage when a retained module delays callbacks. Five actual-source native-loader protocol fixtures passed, but the decisive proof is the actual current86 Windows debug42610467134: real job129137252263 reports the mandatory shared-lib CPUINFO detector Passed in3.97seconds and35/35 CTest cases passed. Protocol mocks and older-head outcomes are not substituted for that Windows runtime execution.

The actual current86 React Native42610467299 completed SUCCESS. Fresh Android artifact13671586695 and iOS artifact13672671135 each verify exactly three real passing MNIST, Basic Types and EPContext cases, no failures/errors/skips and the unchanged14/14 checks passed assertion in device logs/traces. Historical172 artifacts and superseded infrastructure retries are not used as current86 proof. No tests, tolerances, teardown checks or runtime assertions were bypassed.