Skip to content

fix(thrift): honor per-statement rowLimit - #623

Open
vuanhphung wants to merge 2 commits into
mainfrom
vuanhphung/thrift-rowlimit
Open

vuanhphung wants to merge 2 commits into
mainfrom
vuanhphung/thrift-rowlimit

Conversation

@vuanhphung

@vuanhphung vuanhphung commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Thrift accepted executeStatement(sql, { rowLimit }) but dropped the server-side cap: rowLimit: 7 over RANGE(20) returned all 20 rows. Forward explicitly set values as the existing Thrift resultRowLimit i64 field, keep unset requests uncapped, and warn only for unsupported statementConf. Coerce with Number like the kernel: numeric strings become numbers; null, NaN, and values truncated to zero produce an empty result. Document that upgrade effect, the supported Thrift range, invalid-value failures, and older servers ignoring the field. Fetch sizing and kernel behavior are unchanged.

Verified lint, Prettier, type-check, build, and 1,808 unit tests, including numeric-string/null coercion regressions. The inherited CODEX_CI environment exposes an existing agent-attribution test failure on both main and this branch; clearing only that variable makes the full suite pass. Live repository e2e: 11 passing, no skips. A fresh npm-tarball consumer type-checks and fetches the exact seven ordered rows; all four Thrift tripwires XPASS again on the final head, and eight live coercion checks pass across Thrift/SEA.

Coverage source: databricks/databricks-driver-test#2161. At coverage head 152dbaf5, three main runs reproduce all four Thrift bugs; three fixed runs produce all 12 intended XPASS results after wire, exact-row, inline/CloudFetch, and exhaustion assertions. SEA passes all 12 cases. Wire cap / returned rows: 7/7, 100003/100003, 11/11, 10/7. Final-head driver-test passthrough CI run 44282933469 passes Thrift, SEA, and SEA-Reyden.

Fresh-process execute+fetchAll medians, main → fix, three runs per cell with alternating build order on the shared warehouse: Thrift 100k rows 501 → 472 ms, CloudFetch 2392 → 2494 ms; SEA 402 → 405 ms, 2076 → 1775 ms. The coercion follow-up leaves this uncapped benchmark path unchanged; shared-warehouse variation is not an isolated throughput conclusion.

Security Scan's picomatch@2.3.2 finding (GHSA-fc5q-39xc-vjpf) also fails pristine main; the dependency lockfile is unchanged by this PR.

Cross-driver state

Driver State Notes
JDBC Supported Thrift and SEA: setMaxRows (default 0, unlimited); DatabricksThriftServiceClient.java:313, DatabricksSdkClient.java:961 @ d5bdaa3b.
Python Supported Thrift, SEA, kernel: cursor(row_limit=None); thrift_backend.py:1162, sea/models/requests.py:48, kernel/client.py:668 @ b8259e4b.
Go Not supported WithMaxRows is page sizing, not a statement cap; connector.go:429, internal/backend/backend.go:120 @ d9008a9e.
Node.js Thrift: Not supported → Supported; SEA: Supported rowLimit defaults unset/unlimited; explicit 0 returns no rows on both, unlike JDBC's setMaxRows(0).
Kernel Supported; unchanged StatementSpec.row_limit: Option<i64> (default None); src/statement/spec.rs:217, napi/src/connection.rs:217 @ pinned dac32e2b.

Adversarial review: round 1 converged with no high findings; all four remaining documentation/coercion findings are fixed.

This pull request and its description were written by Isaac.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@vuanhphung
vuanhphung marked this pull request as ready for review November 18, 2026 10:06

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Terra), checked by a judge model (Claude Sonnet 5.5).

Verdict: 1 Medium

Looks good — 1 low concern. Thrift now forwards rowLimit as resultRowLimit, and its Number coercion matches the kernel path (KernelSessionBackend.ts:311). Unset requests stay uncapped, and only statementConf still triggers the warning.

This overview is Claude Opus 5.5's initial take; GPT-6.1 Terra added findings it doesn't mention — see the findings below.

Other findings

  • 🟡 Medium — The new tests inspect a stubbed request but the repository's e2e suite does not exercise rowLimit, so it cannot catch regressions in actual capped results or fetch exhaustion. Add a warehouse e2e test for capped and uncapped results, including zero and a multi-fetch case. (Found by GPT-6.1 Terra · the judge couldn't confirm it)

@vuanhphung

Copy link
Copy Markdown
Collaborator Author

Re: the Medium e2e-coverage finding: live warehouse coverage is in the shared conformance suite, not just the mocked unit tests: RESULTFETCH-020..023 at the tested coverage head. All four XPASS on final head f0fa993 after checking wire caps, exact ordered rows, inline/CloudFetch delivery, payload integrity, and fetch exhaustion (including cap 100003 over 200000). Main reproduces all four bugs. Eight additional live checks on this head confirm numeric-string and zero/null/NaN/fraction behavior across Thrift and SEA.

I’m retaining the assignment’s shared-conformance coverage rather than duplicating it in this repository; the expected-failure wrappers will be flipped separately after merge. No uncovered runtime regression is demonstrated by this finding.


This comment was generated with GitHub MCP.

This branch was successfully deployed

1 active deployment
azure-prod — f0fa993f Deployed Nov 18, 2026 by vuanhphung via e2e-test (26.12.1, 26_12_1) #2113
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant