[fix](nereids) Refresh signatures after type-changing rewrites - #67887
Open
morrySnow wants to merge 1 commit into
Open
[fix](nereids) Refresh signatures after type-changing rewrites#67887morrySnow wants to merge 1 commit into
morrySnow wants to merge 1 commit into
Conversation
morrySnow
requested review from
924060929,
englefly and
starocean999
as code owners
September 11, 2026 18:34
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
morrySnow
force-pushed
the
fix/refresh-derived-struct-signatures
branch
from
September 11, 2026 18:36
7824e24 to
893e26c
Compare
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
morrySnow
force-pushed
the
fix/refresh-derived-struct-signatures
branch
from
September 11, 2026 20:51
893e26c to
98b4312
Compare
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
morrySnow
changed the title
morrySnow
marked this pull request as draft
September 12, 2026 03:08
morrySnow
force-pushed
the
fix/refresh-derived-struct-signatures
branch
from
September 12, 2026 10:00
98b4312 to
281f985
Compare
morrySnow
changed the title
morrySnow
marked this pull request as ready for review
September 12, 2026 10:00
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 16988 ms |
Contributor
TPC-DS: Total hot run time: 83247 ms |
Contributor
ClickBench: Total hot run time: 15.08 s |
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Nereids rewrites intentionally preserve a bound function's resolved signature so overload selection, coercion, and value-dependent precision remain stable. A raw-child type gate confused pre-coercion literals with rewritten children that already carried the resolved argument types: recomputing lcm after TINYINT-to-SMALLINT promotion promoted the signature a second time to INT and caused analysis to reject valid SMALLINT children. At the same time, unconditional reuse can leave nested struct, map, and array metadata stale when an outer-join rewrite changes child nullability. Preserve the resolved binding for every eligible rewrite and add a targeted child-derived signature hook that refreshes only current-child complex metadata without searching overloads or rerunning generic signature computation. The hook keeps argument and return metadata self-consistent for struct/map constructors, map entry conversions, and to_json, while retaining object identity when metadata is unchanged. Return a backend status for incompatible struct metadata rather than reaching a fatal cast.
### Release note
Rewritten function signatures keep their resolved overload and promotion decisions while nested complex-type metadata follows current child nullability, preventing both repeated type promotion and struct-column mismatches.
### Check List (For Author)
- Test:
- Unit Test: DerivedFunctionSignatureTest, MapConstructionFunctionsTest, StructLiteralTest, and NullableAliasTest (35 tests).
- Regression test: derived_struct_signature (1 suite, 0 failures/fatal scripts).
- Regression test: fold_constant_numeric_arithmatic (1 suite, 0 failures/fatal scripts), including lcm(2, 4).
- Build/checkstyle: DISABLE_BUILD_UI=ON ./build.sh --fe (success, 0 Checkstyle violations).
- Backend validation: affected production and test translation units compiled with the real build flags under clang 16; the fresh GCC 11 unit-test tree was blocked by an existing C++20 toolchain incompatibility.
- Behavior changed: Yes. Resolved overload, coercion, and precision decisions remain frozen across equivalent rewrites; only explicitly child-derived complex signature metadata is refreshed from current children.
- Does this need documentation: No.
morrySnow
force-pushed
the
fix/refresh-derived-struct-signatures
branch
from
September 12, 2026 11:40
281f985 to
1196eb9
Compare
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 16858 ms |
Contributor
TPC-DS: Total hot run time: 82107 ms |
Contributor
ClickBench: Total hot run time: 14.77 s |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Nereids rewrites preserve resolved function signatures so overload selection, coercion, and value-dependent precision remain stable. However, signatures for complex constructors also contain argument and return metadata derived from their current children. Reusing that metadata after outer-join nullability changes can make FE describe a nested struct differently from the column produced by BE.
An attempted raw-child type gate exposed the opposite failure: rewritten children may already have the resolved coercion type, so treating that as a new overload input can apply type promotion twice.
Root cause
The framework mixed two independent responsibilities:
Comparing current children with the origin function's raw children is not a safe boundary. For example, tiny integer literals resolve
lcm(2, 4)to a SMALLINT signature. A rewrite supplies the already-coerced SMALLINT children; rejecting reuse and computing the signature again promotes them to INT, after which analysis rejects the SMALLINT arguments.Reproduction
The CI witness is
SELECT lcm(2, 4): the first resolution chooses SMALLINT, while a repeated signature computation chose INT and failed argument validation.The complex-type regression constructs a map whose value is a datetime struct, passes it through
map_entriesandmap_from_entries, and inspects the microsecond-bearing value. Before the fix, FE retained stale nested field nullability and BE rejected the declared/actual struct column mismatch. Outer-join tests cover nullable extension forstruct,named_struct, nested structures, andto_json.Fix
ChildDerivedSignaturehook that refreshes only argument and return metadata owned by the current children, without searching overloads or rerunning generic promotion/precision logic.struct,named_struct,map,map_entries,map_from_entries, and complex-inputto_json, keeping nested signatures self-consistent across the full propagation chain.Tests
expected SMALLINT, gotINT) and passes after it.DerivedFunctionSignatureTest,MapConstructionFunctionsTest,StructLiteralTest, andNullableAliasTest: 35 tests, 0 failures/errors.derived_struct_signatureregression: 1 suite, 0 failures/fatal scripts.fold_constant_numeric_arithmaticregression: 1 suite, 0 failures/fatal scripts, includinglcm(2, 4).