Repository navigation
[XNNPACK] Support Conv/ConvTranspose with symbolic spatial dims - #34015
Open
Danny On The Air (dannyota) wants to merge 1 commit into
Open
Danny On The Air (dannyota) wants to merge 1 commit into
Danny On The Air (dannyota) wants to merge 1 commit into
Conversation
ConvBase::IsOnnxNodeSupported rejected any Conv/ConvTranspose whose C, H or W was unknown at partitioning time, so a graph with a symbolic width got no XNNPACK convolutions, although the kernel is created the same way for any width. Allow symbolic or unknown spatial dims when C is static. Kernels are still created once at prepack; the output shape and pads are computed per run from the actual input. Static shapes keep the cached path. ConvTranspose with SAME_* auto_pad or an output_shape attribute still falls back to CPU, since those derive padding from the input size. Reshaping an XNNPACK operator for a new spatial size reallocates its indirection buffer, but xnn_reallocate was not implemented. Implement it with realloc semantics on ORT's allocator and cap request sizes. XNNPACK's deconvolution reshape updates part of its state before it reallocates, so after a failed reshape a dynamic ConvTranspose returns an error instead of running on inconsistent state. Signed-off-by: Danny <dannyontheair@outlook.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The pinned XNNPACK sources are absent from this snapshot, so human review must confirm the reshape state-update ordering underlying Conv retry safety and ConvTranspose failure handling.
No open issues 🎉
What changed in this PR
Enables XNNPACK to handle Conv and ConvTranspose with symbolic spatial dimensions and static channels, addressing #32387’s streaming-inference use case while preserving the cached path for static shapes.
Changes:
- Computes dynamic output shapes at runtime and retains CPU fallback for unsupported ConvTranspose attributes.
- Implements allocator reallocation and prevents reuse of dynamic ConvTranspose operators after reshape or setup failures.
- Adds coverage for changing input sizes, provider placement, fallback behavior, allocation limits, and failure handling.
| File | Description |
|---|---|
onnxruntime/test/providers/xnnpack/xnnpack_basic_test.cc |
Adds dynamic-shape, allocator, fallback, and failure tests. |
onnxruntime/core/providers/xnnpack/xnnpack_init.h |
Declares the test-only allocation-failure hook. |
onnxruntime/core/providers/xnnpack/xnnpack_init.cc |
Implements aligned reallocation semantics and allocation-size limits. |
onnxruntime/core/providers/xnnpack/nn/conv.cc |
Uses runtime output-shape calculation. |
onnxruntime/core/providers/xnnpack/nn/conv_transpose.h |
Tracks whether the operator is unusable after failure. |
onnxruntime/core/providers/xnnpack/nn/conv_transpose.cc |
Calculates runtime shapes and guards against reuse after failure. |
onnxruntime/core/providers/xnnpack/nn/conv_base.h |
Declares shared shape calculation and dynamic-dimension state. |
onnxruntime/core/providers/xnnpack/nn/conv_base.cc |
Relaxes spatial-dimension checks and implements dynamic shape calculation. |
cmake/onnxruntime_providers_xnnpack.cmake |
Enables allocation-failure injection for unit-test builds. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ConvBase::IsOnnxNodeSupportedrejected any Conv/ConvTranspose whose C, H or W was unknown at partitioning time, so a graph with static channels and a symbolic width got no XNNPACK convolutions, although the kernel is created the same way for any width.SAME_*auto_pad or anoutput_shapeattribute still falls back to CPU, since those derive padding from the input size.xnn_reallocatewasORT_NOT_IMPLEMENTED. Implement it with realloc semantics on ORT's allocator (size header, alignment kept, nullptr/zero handled) and cap request sizes atSIZE_MAX / 2.reshape_subconv2d_path) updates part of its state before it reallocates. After a failed reshape, a dynamic ConvTranspose now returns an error instead of running on inconsistent state. Conv is not affected: it reallocates before updating cached dims.Testing (macOS arm64,
--use_xnnpack --build_shared_lib)onnxruntime_provider_test --gtest_filter='*Conv*:*Xnnpack*': 443 pass, 0 fail (CUDA-only tests skipped).TestConvDynamicSpatial,TestConvTransposeDynamicSpatialandTestDynamicSpatialRepeatedRunscompare with CPU across widths 17, 64, 17, 18 and assert per-node XNNPACK placement; fallback tests for unknown C and the excluded ConvTranspose attributes;AllocatorReallocate,AllocatorSizeLimit; failed-reshape tests with an injected allocation failure and an allocator exception.Motivation and Context
Fixes #32387.