Repository navigation
Add a C API to serialize an OrtGraph to an ONNX model - #34014
Open
Danny On The Air (dannyota) wants to merge 1 commit into
Open
Danny On The Air (dannyota) wants to merge 1 commit into
Danny On The Air (dannyota) wants to merge 1 commit into
Conversation
Plugin EPs receive an OrtGraph but had no public way to turn it into ONNX ModelProto bytes, for example to hand a partition to a compiler. Add OrtGraphSerializationOptions with an external-initializer policy, a size threshold, an initializer-location callback and a write chunk size, plus Graph_SerializeToOnnxModel (allocator-owned bytes) and Graph_SerializeToOnnxModelWithWriteFunc (streaming). Only the supplied graph view is serialized, in Graph_GetNodes order; captured outer-scope values of a control-flow subgraph become inputs or copied initializers. Training info is not serialized. Output is deterministic for unchanged initializers and deterministic callbacks. C++ wrappers included. Graph keeps the original proto of sparse initializers whose values or indices use external data, so PRESERVE can keep those references after densification. This also fixes a dangling reference: Graph:: AddConstantProtoAsInitializer stored the name of a local TensorProto in sparse_tensor_names_, a set of string references. Signed-off-by: Danny <dannyontheair@outlook.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Serialization can dereference unvalidated model-supplied memory addresses in unused local functions, risking crashes or disclosure of process memory.
7 open issues
What changed in this PR
Adds public C and C++ APIs to serialize execution-provider graph views into ONNX model bytes, addressing #32799 without requiring plugin providers to implement their own conversion.
Changes:
- Adds allocator-owned and streaming output with configurable external-initializer policies.
- Serializes graph views, nested subgraphs, captured values, and local functions deterministically.
- Preserves sparse external-data references, fixes a sparse-name lifetime issue, and adds serialization tests.
| File | Description |
|---|---|
onnxruntime/test/ep_graph/test_ep_graph.cc |
Adds policy, ownership, streaming, sparse, and round-trip tests. |
onnxruntime/core/session/ort_apis.h |
Declares internal serialization API entry points. |
onnxruntime/core/session/onnxruntime_c_api.cc |
Appends serialization functions to the API table. |
onnxruntime/core/session/graph_serialization.cc |
Implements graph conversion, initializer policies, and output handling. |
onnxruntime/core/graph/model.h |
Declares graph-independent model metadata export. |
onnxruntime/core/graph/model.cc |
Copies model fields without graph or training information. |
onnxruntime/core/graph/graph.cc |
Preserves sparse references and fixes sparse-name ownership. |
include/onnxruntime/core/session/onnxruntime_cxx_inline.h |
Implements C++ serialization wrappers. |
include/onnxruntime/core/session/onnxruntime_cxx_api.h |
Declares C++ options and owned output types. |
include/onnxruntime/core/session/onnxruntime_c_api.h |
Defines and documents public serialization APIs. |
include/onnxruntime/core/graph/graph.h |
Adds sparse-source storage and serialization accessors. |
Review effort: Balanced
Comment on lines
+263
to
+264
| for (auto& attr : *function.mutable_attribute_proto()) { | ||
| ORT_RETURN_IF_ERROR(ProcessAttribute(attr, view.GetGraph().ModelPath(), options)); |
| words_output->set_name("words"); | ||
| words_output->mutable_type()->mutable_tensor_type()->set_elem_type(ONNX_NAMESPACE::TensorProto_DataType_STRING); | ||
| words_output->mutable_type()->mutable_tensor_type()->mutable_shape()->add_dim()->set_dim_value(2); | ||
| auto* sparse = mixed.mutable_graph()->add_sparse_initializer(); |
| InlinedVector<std::string> names; | ||
| for (const auto& [name, tensor] : initializers) names.push_back(name); | ||
| std::sort(names.begin(), names.end()); | ||
| for (const auto& name : names) { |
Comment on lines
+109
to
+110
| ORT_RETURN_IF_ERROR(utils::DenseTensorToSparseTensorProto(*initializers.at(name), scope->ModelPath(), | ||
| *result.add_sparse_initializer())); |
Comment on lines
+190
to
+191
| auto replacement = utils::TensorToTensorProto(data, tensor.name()); | ||
| replacement.clear_raw_data(); |
Comment on lines
+214
to
+215
| if (attr.has_g()) ORT_RETURN_IF_ERROR(ProcessGraph(*attr.mutable_g(), path, options)); | ||
| for (auto& graph : *attr.mutable_graphs()) ORT_RETURN_IF_ERROR(ProcessGraph(graph, path, options)); |
| ORT_RETURN_IF(options.policy == ORT_GRAPH_EXTERNAL_INITIALIZER_REPACKAGE_VIA_CALLBACK && !options.location, | ||
| "REPACKAGE_VIA_CALLBACK requires an initializer location callback"); | ||
| const auto& view = ep->GetGraphViewer(); | ||
| model = view.GetGraph().GetModel().ToProtoWithoutGraph(); |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implements the API proposed in #32799: serialize an
OrtGraphto ONNXModelProtobytes.OrtGraphSerializationOptions(create/release) with setters for the external-initializer policy (PRESERVE, EMBED, REPACKAGE, FAIL), a minimum size threshold, anOrtGetInitializerLocationFunccallback and a maximum write chunk size.Graph_SerializeToOnnxModelreturns allocator-owned bytes;Graph_SerializeToOnnxModelWithWriteFuncstreams throughOrtWriteBufferFunc. Both reuse the existing callback types from the compile API.Graph_GetNodesorder, including nested subgraphs and local functions. Captured outer-scope values of a control-flow subgraph become graph inputs or copied initializers. Training info is not serialized.Ort::GraphSerializationOptionsand twoConstGraphmethods.Graphnow keeps the original proto of sparse initializers whose values or indices use external data, so PRESERVE keeps those references after densification. This also fixes a dangling reference on main:Graph::AddConstantProtoAsInitializerinserted the name of a localTensorProtointosparse_tensor_names_, a set ofstd::reference_wrapper<const std::string>.Testing (macOS arm64,
--build_shared_lib)onnxruntime_ep_graph_test --gtest_filter='EpGraphTest.*': 29 pass, including 11 newSerializeOnnx_*tests: streaming vs allocated bytes, chunking and first-error stop, failure outputs and allocator ownership, all initializer policies and the threshold, filtered views and nested/captured values, local functions, sparse Constants,Ifinlining, training info, determinism, and a REPACKAGE round trip that reloads the model and compares CPU outputs.onnxruntime_test_allGraph, sparse, inlining,Ifand Constant tests: 229 pass.Motivation and Context
Fixes #32799. Plugin EPs receive an
OrtGraphbut had no public way to get ONNX bytes for it, for example to hand a partition to a compiler.