Skip to content

[XNNPACK] Support Conv/ConvTranspose with symbolic spatial dims - #34015

Open
Danny On The Air (dannyota) wants to merge 1 commit into
microsoft:mainfrom
dannyota:xnnpack/conv-symbolic-spatial-dims
Open

Danny On The Air (dannyota) wants to merge 1 commit into
microsoft:mainfrom
dannyota:xnnpack/conv-symbolic-spatial-dims

Conversation

@dannyota

Copy link
Copy Markdown
Contributor

Description

ConvBase::IsOnnxNodeSupported rejected any Conv/ConvTranspose whose C, H or W was unknown at partitioning time, so a graph with static channels and a symbolic width got no XNNPACK convolutions, although the kernel is created the same way for any width.

  • Allow symbolic or unknown spatial dims when C is static. Kernels are still created once at prepack; the output shape and pads are computed per run from the actual input. Static shapes keep the cached path.
  • ConvTranspose with SAME_* auto_pad or an output_shape attribute still falls back to CPU, since those derive padding from the input size.
  • Reshaping an XNNPACK operator for a new spatial size reallocates its indirection buffer, but ORT's xnn_reallocate was ORT_NOT_IMPLEMENTED. Implement it with realloc semantics on ORT's allocator (size header, alignment kept, nullptr/zero handled) and cap request sizes at SIZE_MAX / 2.
  • XNNPACK's deconvolution reshape (reshape_subconv2d_path) updates part of its state before it reallocates. After a failed reshape, a dynamic ConvTranspose now returns an error instead of running on inconsistent state. Conv is not affected: it reallocates before updating cached dims.

Testing (macOS arm64, --use_xnnpack --build_shared_lib)

  • onnxruntime_provider_test --gtest_filter='*Conv*:*Xnnpack*': 443 pass, 0 fail (CUDA-only tests skipped).
  • New: TestConvDynamicSpatial, TestConvTransposeDynamicSpatial and TestDynamicSpatialRepeatedRuns compare with CPU across widths 17, 64, 17, 18 and assert per-node XNNPACK placement; fallback tests for unknown C and the excluded ConvTranspose attributes; AllocatorReallocate, AllocatorSizeLimit; failed-reshape tests with an injected allocation failure and an allocator exception.

Motivation and Context

Fixes #32387.

ConvBase::IsOnnxNodeSupported rejected any Conv/ConvTranspose whose C, H
or W was unknown at partitioning time, so a graph with a symbolic width
got no XNNPACK convolutions, although the kernel is created the same way
for any width.

Allow symbolic or unknown spatial dims when C is static. Kernels are
still created once at prepack; the output shape and pads are computed
per run from the actual input. Static shapes keep the cached path.
ConvTranspose with SAME_* auto_pad or an output_shape attribute still
falls back to CPU, since those derive padding from the input size.

Reshaping an XNNPACK operator for a new spatial size reallocates its
indirection buffer, but xnn_reallocate was not implemented. Implement it
with realloc semantics on ORT's allocator and cap request sizes.
XNNPACK's deconvolution reshape updates part of its state before it
reallocates, so after a failed reshape a dynamic ConvTranspose returns
an error instead of running on inconsistent state.

Signed-off-by: Danny <dannyontheair@outlook.com>
Copilot AI balanced review requested due to automatic review settings November 9, 2026 15:44
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The pinned XNNPACK sources are absent from this snapshot, so human review must confirm the reshape state-update ordering underlying Conv retry safety and ConvTranspose failure handling.

No open issues 🎉

What changed in this PR

Enables XNNPACK to handle Conv and ConvTranspose with symbolic spatial dimensions and static channels, addressing #32387’s streaming-inference use case while preserving the cached path for static shapes.

Changes:

  • Computes dynamic output shapes at runtime and retains CPU fallback for unsupported ConvTranspose attributes.
  • Implements allocator reallocation and prevents reuse of dynamic ConvTranspose operators after reshape or setup failures.
  • Adds coverage for changing input sizes, provider placement, fallback behavior, allocation limits, and failure handling.
File Description
onnxruntime/​test/​providers/​xnnpack/​xnnpack_basic_test.cc Adds dynamic-shape, allocator, fallback, and failure tests.
onnxruntime/​core/​providers/​xnnpack/​xnnpack_init.h Declares the test-only allocation-failure hook.
onnxruntime/​core/​providers/​xnnpack/​xnnpack_init.cc Implements aligned reallocation semantics and allocation-size limits.
onnxruntime/​core/​providers/​xnnpack/​nn/​conv.cc Uses runtime output-shape calculation.
onnxruntime/​core/​providers/​xnnpack/​nn/​conv_transpose.h Tracks whether the operator is unusable after failure.
onnxruntime/​core/​providers/​xnnpack/​nn/​conv_transpose.cc Calculates runtime shapes and guards against reuse after failure.
onnxruntime/​core/​providers/​xnnpack/​nn/​conv_base.h Declares shared shape calculation and dynamic-dimension state.
onnxruntime/​core/​providers/​xnnpack/​nn/​conv_base.cc Relaxes spatial-dimension checks and implements dynamic shape calculation.
cmake/​onnxruntime_providers_xnnpack.cmake Enables allocation-failure injection for unit-test builds.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] XNNPACK EP: allow Conv/ConvTranspose with symbolic spatial dims where kernel creation doesn't depend on them

2 participants