Repository navigation
Fix release planning for cross-stream aliased buffers - #33881
Tianlei Wu (tianleiwu) with Copilot wants to merge 6 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Co-authored-by: tianleiwu <30328909+tianleiwu@users.noreply.github.com>
Follow the full reuse chain when assigning consumers to buffer release actions. Per-device stream planning can retain cross-stream alias links; following only one link drops late consumers and releases live host shape tensors prematurely. Preserve single-stream release and existing allocation ownership behavior, with five focused alias ownership regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/azp run Linux Android Emulator QNN CI Pipeline |
|
Azure Pipelines: No pipelines were found matching this branch/path. |
There was a problem hiding this comment.
🔵 Needs a closer look
Cross-stream host-buffer lifetime changes lack end-to-end CUDA validation, leaving the reported failure and arena-disabled case unverified by the planner-level, CPU-only checks.
No open issues 🎉
What changed in this PR
Fixes premature release of host buffers reused and aliased across execution streams, addressing the allocation-lifetime issue reported in #33807.
Changes:
- Follows the full buffer-reuse chain when assigning consumers to the owning allocation, with bounded cycle detection.
- Preserves existing allocation ownership and release policies.
- Adds a GPU-independent cross-stream regression and five single-stream ownership cases.
No actionable findings were identified.
| File | Description |
|---|---|
onnxruntime/test/framework/allocation_planner_test.cc |
Adds cross-stream alias-lifetime and single-stream ownership coverage. |
onnxruntime/core/framework/allocation_planner.cc |
Resolves reuse chains to the owning allocation before planning releases. |
Review effort: Balanced
|
Re: #33881 (review) — the current published head is The GPU-independent accounting repair in The simulator-preparation repair in The genuine current-head React Native workflow 42603603023 completed successfully. Fresh iOS artifact The fresh complete all-20 audit at 2026-11-09T13:20:03Z verified this exact currentd26 head open, non-draft, mergeable, with zero unresolved review threads and all current-head checks green. Current Android/iOS device proof and the retained native accounting proof remain valid. This PR's CI preparation is complete; normal human approval remains external. The full #33807 CUDA model remains unverified locally. |
Exercise the cross-stream host-buffer reuse and alias chain through InferenceSession with the CPU arena enabled and disabled. Verify the allocator type, root release accounting, and sixteen runtime outputs for each setting without requiring a separate GPU build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the CPU provider arena-capability policy when checking the allocator selected for each session option. ASan and 32-bit builds deliberately use a device allocator even when an arena is requested. Keep both runtime variants and all alias-lifetime and output checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
iOS device runs have stalled during cold simulator setup before app installation. Reset only the simulator selected by the Detox configuration, then wait for boot and app services without restarting the global CoreSimulator service. Keep app reinstallation, device assertions, and the existing Detox setup deadline unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Description
Fix premature release of host allocations whose buffers are reused and then aliased across execution streams. Deallocation planning now follows the full
kReusechain, so consumers of Squeeze/Reshape-style aliases retain the allocation that owns the buffer.Summary of Changes
InferenceSessionwithenable_cpu_mem_arenaboth disabled and enabled, verifying allocator type, root release accounting, and runtime outputs.Testing
onnxruntime_test_allwith streams enabled and at most eight build jobs../onnxruntime_test_all --gtest_filter='PlannerTest.*:AllocationPlannerTest.*:FreeBufferIndexTest.*:AllocationKinds/AliasedBufferReleasePlannerTest.*:CpuArena/AliasedCpuBufferRuntimeTest.*'660105743cec40554650e6995d639c31480bb196. The test-only portability push starts new exact-head CI, which is not yet claimed green.Motivation and Context
Per-device stream planning can leave an alias pointing to a value whose buffer was itself reused on another stream. Following only one reuse link omits later alias consumers from the root release action, allowing a live CPU shape buffer to be recycled before a CUDA Expand reads it.
Fixes #33807.
Checklist