Constant canonicalisation uses similar techniques to SparseArray - #9889
robert3005 wants to merge 2 commits into
Conversation
…ization `ConstantArray::append_to_builder` already shares one copy of a constant run's values the way `SparseArray` shares its fill value, but `constant_canonicalize` still materialized per row. Bring it in line: - A constant fixed-size list now builds one row's elements and tiles that single copy over the run, as `fixed_size_list_fill_tile` does for a sparse fill. Elements that are all the same scalar - which a null list's placeholders always are - stay one `ConstantArray` covering the whole run, so the common cases stop being `list_size * len` scalar appends. - A constant string or binary run adopts the scalar's own buffer instead of copying its bytes, and only when the value is too long to inline. A value of exactly `MAX_INLINED_SIZE` bytes lives in its view, so the buffer pushed for it was never referenced; `uncompressed_size_in_bytes` carried the same off-by-one and is corrected to match. - A constant list's offsets and sizes take the narrowest width that can describe the list, via `match_smallest_list_offset_type!`, rather than always costing eight bytes a row once decoded. Signed-off-by: "Robert" <robert@spiraldb.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KwhzKebXYiuUjPxBmDdTF9
`execute_sparse_decimal` pushed the fill value through `DecimalBuilder` a value at a time. Build the buffer directly instead, the way `constant_canonicalize` already does for a constant decimal run. Also tightens the comments added in the previous commit. Signed-off-by: "Robert" <robert@spiraldb.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KwhzKebXYiuUjPxBmDdTF9
Merging this PR will regress 1 benchmark
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | dict_canonicalize_gt_u8_neon[1000000] |
487.6 µs | 560.7 µs | -13.03% |
| ⚡ | WallTime | scalar_subtract_neon |
13.4 µs | 11.9 µs | +12.59% |
| ⚡ | Simulation | compact_sliced[(1024, 90)] |
1.5 µs | 1.3 µs | +12.53% |
| ⚡ | Simulation | compact_sliced[(2048, 90)] |
1.5 µs | 1.4 µs | +11.99% |
| ⚡ | WallTime | mul_u32_nonnull_avx512 |
6.3 µs | 5.6 µs | +11.05% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/inspiring-maxwell-he0qon (cc4d6f2) with develop (3e2b65c)
Footnotes
-
218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
1 participant
I have spent quite a bit of time looking at SparseArray canonicalisation and noticed that ConstantArray would benefit from similar improvements