Skip to content

Cut ResultSet.next() CPU per row on Arrow results - #1811

Merged
vuanhphung merged 2 commits into
mainfrom
perf/narrowcf
Nov 17, 2026
Merged

vuanhphung merged 2 commits into
mainfrom
perf/narrowcf

Conversation

@vuanhphung

@vuanhphung vuanhphung commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Description

The nightly Extended Types CloudFetch tests only drain rows, and for narrow columns (80M BOOLEAN, 50M FLOAT, 30M DOUBLE rows) most of the client's time goes to ResultSet.next(). On a live Thrift BOOLEAN drain the reading thread used 4.5 ns of CPU per row (about 365 ms of a ~500 ms drain; most of the rest is waiting for the first chunk), while in-process the download threads needed about 45 ms of CPU to download and decode all 80M rows. Each next() called the chunk iterator's hasNextRow() three times and advanced three per-row counters, and DatabricksResultSet.next() read the telemetry collector field on every row.

When the chunk iterator enters a record batch, it now records the last row it may read within the chunk's row count, so moving within a batch is one comparison and one increment. Batch changes and the row-count limit still go through the previous logic. ArrowStreamResult.next() tries the current iterator first and asks the chunk provider only when the chunk is exhausted, and DatabricksResultSet.next() checks its iteration-telemetry flags before the collector. Rows, row positions (including the existing empty-chunk behavior) and telemetry are unchanged. The iterator is shared by CloudFetch and inline Arrow results on both backends.

Testing

A new test compares the iterator step by step with the previous algorithm over edge-case and random batch layouts. New ArrowStreamResultTest cases pin rows, values and getCurrentRow() across chunks and after close, and mutants of the fix fail them. The full jdbc-core suite passes except ArrowBufferAllocatorTest.testCreateDatabricksBufferAllocator (outside the NIO profile) and ChunkDownloadTaskTest.interruptedDownload_isCancelledWithoutRetryOrFailureReport (order-dependent; it passes on its own), which fail the same way on main. Live ThriftCloudFetchTests and LargeQueryExecutionTests pass. Value checksums match main in-process (both chunk providers, with max rows, early close and cancel) and live on Thrift and SEA (full drain, setMaxRows, early close, cancel).

Medians, first iteration of each JVM dropped. In-process runs replay captured chunks through the SEA path from a local mock with no network limit (5 fresh JVMs × 9 iterations per side). Live runs use the test warehouse with the nightly's URL options, one connection per JVM, interleaved (BOOLEAN rows: 6 rounds × 5 iterations; reader CPU: 5 rounds × 4).

main This PR
next() CPU per row, in-process (BOOLEAN / FLOAT / DOUBLE) 4.5 / 4.7 / 4.7 ns 2.4 / 2.5 / 2.5 ns
BOOLEAN drain, in-process 390 ms 224 ms
BOOLEAN drain, live Thrift / SEA 497 / 458 ms 330 / 325 ms
BOOLEAN execute + drain, live Thrift / SEA 1272 / 1192 ms 1036 / 1082 ms
Reader CPU per drain, live Thrift (BOOLEAN / FLOAT / DOUBLE) 365 / 231 / 140 ms 196 / 133 / 74 ms

Adversarial review: an independent re-run on a separate harness reproduced the gains (live BOOLEAN drain Thrift 485 → 293 ms, SEA 455 → 335 ms; reader CPU about halved on every type) and found no correctness issues; its two low findings (a test comment, live checksum ordering) are addressed.

Telemetry Errors

  • Not applicable — this PR does not add or change a telemetry-visible error.
  • Applicable — the error uses DatabricksDriverErrorCode where appropriate, and any
    new code is uniquely numbered and tested.
  • Applicable — its driver/server/user classification is linked, or maintainer help is
    requested because the author cannot access the classification.

Additional Notes to the Reviewer

FLOAT and DOUBLE drains are bound by downloading and decoding chunks rather than by the reader: their reader CPU halves, but their drain time changes by only 1–6% in-process (also with simulated storage at 50 ms to first byte and 150 MB/s per stream) and stays within run-to-run noise live. Execute time varies with the server, so totals are noisier than drains; across three live runs the BOOLEAN drain dropped 21–34% on Thrift and 29–42% on SEA. Getter-heavy reads (getObject on every row) are about 5% faster. The branch merges cleanly with the open #1786, #1791, #1794, #1795 and #1799.

This pull request and its description were written by Isaac.

vuanhphung and others added 2 commits November 16, 2026 07:24
The chunk iterator now records, when it enters a record batch, the last
row it may read within the chunk's row count, so moving within a batch is
one comparison and one increment. ArrowStreamResult.next() tries the
current iterator first instead of calling hasNextRow() three times per
row, and DatabricksResultSet.next() checks its iteration-telemetry flags
before the collector field.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…vior

The test pins that an empty chunk in the middle makes next() return false, as main does. Say that this is an existing bug kept unchanged, not intended behavior.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@vuanhphung
vuanhphung marked this pull request as ready for review November 16, 2026 20:09

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

@aakash-saravanan-db aakash-saravanan-db left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Computing the readable limit once per record batch puts the optimization at the right boundary and keeps the per-row path simple. The limit still respects the chunk row count, while chunk transitions and cleanup stay with the existing providers. The step-by-step equivalence tests, explicit row/value assertions, and performance measurements distinguishing reader CPU from end-to-end drain time give good confidence in the change.

Non-blocking follow-up: could we track the existing empty intermediate chunk behavior called out in ArrowStreamResultTest in a separate issue? Returning false from next() while later chunks still contain rows makes a normal while (rs.next()) consumer stop early. A follow-up should skip empty chunks and verify that the full loop returns all subsequent rows with correct row positions. Keeping that fix separate from this optimization makes sense.

@vuanhphung
vuanhphung added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 7a95254 Nov 17, 2026
27 checks passed
@vuanhphung
vuanhphung deleted the perf/narrowcf branch November 17, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants