Skip to content

Improve session creation time further - #34009

Open
Christian Bourjau (cbourjau) wants to merge 2 commits into
microsoft:mainfrom
cbourjau:more-session-creation-speedups
Open

Christian Bourjau (cbourjau) wants to merge 2 commits into
microsoft:mainfrom
cbourjau:more-session-creation-speedups

Conversation

@cbourjau

Copy link
Copy Markdown
Contributor

Description

#33407 and #33939 removed several superlinear costs from session creation. With those in place, two more become visible. Both are fixed by replacing linear scans with hash-based lookups, which also shortens the code.

Number of graph inputs/outputs

SaveInputOutputNamesToNodeMapping checks every node input against the list of graph inputs, and every node output against the list of graph outputs, by linear scan and string comparison. Session creation is therefore O(nodes × graph inputs/outputs). The fix builds the name sets once per call. This applies at every graph optimization level.

n inputs/outputs Before After
4000 0.17 s 0.09 s
8000 0.57 s 0.17 s
16000 2.28 s 0.51 s

(n inputs → n Neg nodes → n outputs, ORT_DISABLE_ALL)

Number of distinct constants

ConstantSharing finds the first equal initializer by scanning all distinct values seen so far, so it is O(constants × distinct constants). The fix keys a hash map by data type, shape, and raw bytes. The new code is shorted at the expense of less readable tails on the key name, but those are never read anyhow. Which initializers are shared, and which one is kept, is unchanged.

n distinct scalars ConstantSharing cost before after
4000 0.13 s 0.005 s
8000 0.35 s 0.006 s
16000 1.36 s 0.020 s

(n nodes Mul(x, c_i) with distinct c_i, ORT_ENABLE_BASIC; cost = session creation with minus without ConstantSharing)

A new test, ConstantSharing_ShareOnlyEqualValuesAndShapes, checks that initializers are shared exactly when their shapes and values are equal.

Motivation and Context

Hitting these corner cases is admittedly rare. We use ONNX Runtime to JIT-compile NumPy/pandas code. This requires fast session creation and may generate graphs with many inputs/outputs and many distinct literal constants. I can add the benchmark scripts, but I didn't want to add clutter unnecessarily.

Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
Signed-off-by: Christian Bourjau <christian.bourjau@quantco.com>
Copilot AI balanced review requested due to automatic review settings November 9, 2026 12:35
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the hash-based lookup and constant-deduplication changes.

No open issues 🎉

What changed in this PR

Improves session creation scalability by replacing two remaining linear-scan bottlenecks with hash-based lookups, building on #33407 and #33939.

Changes:

  • Precomputes graph input, output, and implicit-input name sets for node mapping.
  • Deduplicates constants using keys containing data type, shape, and raw bytes.
  • Adds a regression test verifying that equal constants share initializers while differing values or shapes remain separate.

No actionable findings were identified.

File Description
onnxruntime/​test/​optimizer/​graph_transform_test.cc Tests constant sharing across equal and differing values and shapes.
onnxruntime/​core/​optimizer/​constant_sharing.cc Replaces constant-value scans with direct hash-map lookups.
onnxruntime/​core/​framework/​session_state_utils.cc Uses name sets to accelerate graph input/output membership checks.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants