Skip to content

[6/7] Add native batch integration coverage - #1633

Merged
sreekanth-db merged 3 commits into
mainfrom
sreekanth-db/stack/native-batch-integration-tests
Nov 16, 2026
Merged

sreekanth-db merged 3 commits into
mainfrom
sreekanth-db/stack/native-batch-integration-tests

Conversation

@sreekanth-db

@sreekanth-db sreekanth-db commented Aug 11, 2026 •

Copy link
Copy Markdown
Collaborator

🥞 Stacked PR

Use this link to review incremental changes.


Description

  • Add fake-service integration coverage for native PreparedStatement batching.
  • Record real SEA warehouse and Thrift V10 all-purpose responses.
  • Verify SEA sends one ordered parameter_sets request and both transports return ordered JDBC update counts.
  • Add debug logging for native SEA batch execution.

Testing

  • FAKE_SERVICE_TYPE=SQL_EXEC FAKE_SERVICE_TEST_MODE=REPLAY mvn test -pl jdbc-core -Dtest=NativePreparedStatementBatchIntegrationTests -Ddependency-check.skip=true
  • FAKE_SERVICE_TYPE=THRIFT_SERVER FAKE_SERVICE_TEST_MODE=REPLAY mvn test -pl jdbc-core -Dtest=NativePreparedStatementBatchIntegrationTests -Ddependency-check.skip=true
  • mvn test -pl jdbc-core -Dtest=PreparedStatementBatchExecutorTest,DatabricksSdkClientTest,DatabricksThriftServiceClientTest -Ddependency-check.skip=true (119 tests)
  • isaac review --uncommitted (0 findings)

Additional Notes to the Reviewer

Only real recorded native-success responses are included. Compatibility fallback remains covered by unit tests because the available backends support native batching and do not emit the legacy fallback signal.

NO_CHANGELOG=true

@github-actions

Copy link
Copy Markdown

JDBC integration tests triggered (replay, thrift + sea). View workflow runs.

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Medium

Test-only PR (one benign debug log + WireMock replay fixtures + a new native-batch integration test). Cleanup logic correctly mirrors the sibling THRIFT+REPLAY skip pattern. One medium concern: the SEA request-body assertion uses equalToJson(..., true, true) and omits parameter values, so it does not actually verify the ordering it claims to.

Scope: reviewed as a stacked increment (8b0bfbb..head); the parent's changes are reviewed on the parent PR.

@github-actions

Copy link
Copy Markdown

Integration test approval reset.

New commits were pushed to this PR. Label(s) integration-test were removed for security.

A maintainer must re-review and re-add a label to preview tests again. (The real gate runs in the merge queue.)

Latest commit: 07968b8

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — test-only PR (fake-service replay coverage for native PreparedStatement batching) plus one benign debug log line. The cleanUp() condition correctly mirrors the sibling close-unless-THRIFT+REPLAY convention, and the SEA verify() uses ignoreArrayOrder=false to properly assert ordered parameter_sets. One low-severity robustness note on the Thrift routing workaround in setUp.

Scope: reviewed as a stacked increment (8b0bfbb..head); the parent's changes are reviewed on the parent PR.

@github-actions

Copy link
Copy Markdown

JDBC integration tests triggered (replay, thrift + sea). View workflow runs.

@sreekanth-db sreekanth-db changed the title Add native batch integration coverage [6/7] Add native batch integration coverage Aug 11, 2026
@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-integration-tests branch from 07968b8 to 2d0f4da Compare August 24, 2026 23:48
@github-actions

Copy link
Copy Markdown

Integration test approval reset.

New commits were pushed to this PR. Label(s) integration-test were removed for security.

A maintainer must re-review and re-add a label to preview tests again. (The real gate runs in the merge queue.)

Latest commit: 2d0f4da

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — test-only coverage plus a safe debug-log line; 1 low note. The recorded SEA/Thrift stubs and the ordered parameter_sets verify align with the stated intent, and cleanup mirrors the existing ExecutionIntegrationTests idiom. Only concern is the unguarded/unrestored system-property rewrite in setUp (F1).

Scope: reviewed as a stacked increment (07f3135..head); the parent's changes are reviewed on the parent PR.

@sreekanth-db sreekanth-db added the skip-coverage Skip the coverage fan-out for this PR (no tracking issue opened in databricks-driver-test) label Sep 15, 2026
@vikrantpuppala
vikrantpuppala removed their request for review October 23, 2026 09:06

@shiminlin-db shiminlin-db left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one small comment!

@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-integration-tests branch from 2d0f4da to 56d4b57 Compare November 5, 2026 10:03
@sreekanth-db
sreekanth-db changed the base branch from main to sreekanth-db/stack/native-batch-sea November 5, 2026 10:03

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Nit

Looks good, with one low concern: the Thrift setup rewrites localhost to 127.0.0.1 with no explanation, and no other fake-service test does this. On the Thrift side, the binaryEqualTo request matching already confirms the exact batch payload. Skipping the connection close in Thrift REPLAY mode follows the existing pattern in the other fake-service tests.

Other findings

  • ⚪ Nit — The Thrift branch rewrites the WireMock route property from localhost to 127.0.0.1, and there is no comment explaining why. No other *IntegrationTests class does this (I grepped src/test/java/com/databricks/jdbc/integration). That suggests either an environment-specific problem (for example, IPv6 resolution of localhost on the recording machine) or a real issue in the native-batch Thrift path that the rewrite hides.

If the rewrite is needed, please add a comment with the reason. Better still, fix it in FakeServiceExtension so all Thrift tests get the same behaviour. If it isn't needed, remove it.

Also, System.getProperty(routeProperty) is dereferenced without a null check. If the extension hasn't set the property yet, or the Thrift target property is missing, setUp throws a bare NPE instead of a clear test-setup failure.

@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-sea branch from 89eddf8 to f2b870e Compare November 5, 2026 12:38
@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-integration-tests branch from 56d4b57 to 9fb5f65 Compare November 5, 2026 12:38

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good overall, with one low-severity concern about an unexplained, test-local change to the fake-service routing property. The SEA check (exactly one ordered parameter_sets POST, matched with ignoreArrayOrder=false) and the exact-binary Thrift replay mappings do verify native batching on both transports. Skipping close() for Thrift in REPLAY mode follows the existing pattern in the other fake-service tests.

gibkim pushed a commit to gibkim/databricks-jdbc that referenced this pull request Nov 5, 2026
## 🥞 Stacked PR
Use this
[link](https://github.com/databricks/databricks-jdbc/pull/1623/files) to
review incremental changes.
-
[**stack/native-batch-routing**](databricks#1623)
[[Files
changed](https://github.com/databricks/databricks-jdbc/pull/1623/files)]
← _this PR_
-
[stack/native-batch-thrift](databricks#1625)
[[Files
changed](https://github.com/databricks/databricks-jdbc/pull/1625/files/597237e946403890862ba875b54121b489cfec12..b88cd288f12efdd3126506dc37a443dfea7c1842)]
-
[stack/native-batch-sea](databricks#1632)
[[Files
changed](https://github.com/databricks/databricks-jdbc/pull/1632/files/b88cd288f12efdd3126506dc37a443dfea7c1842..07f31353e7ff8c94a644c74a4ace3d970dd63bac)]
-
[stack/native-batch-integration-tests](databricks#1633)
[[Files
changed](https://github.com/databricks/databricks-jdbc/pull/1633/files/07f31353e7ff8c94a644c74a4ace3d970dd63bac..2d0f4da302b183f15de9804af1c12346b5531bf4)]
-
[stack/native-batch-activation](databricks#1634)
[[Files
changed](https://github.com/databricks/databricks-jdbc/pull/1634/files/2d0f4da302b183f15de9804af1c12346b5531bf4..f4e3ff24e5d54451eec9603e429fce7549f291b8)]

---------
## Description

Add the left-side capture and routing boundary for native
prepared-statement batching.

- Snapshot ordered parameter sets when `addBatch()` is called.
- Preserve one-based JDBC indexes in the shared model; transport
adapters will convert wire ordinals.
- Use the same parameter-set model for existing legacy execution.
- Add an injectable `NativeBatchExecutor` capability boundary.
- Route native-enabled parameterized INSERTs to that boundary when
supported.

Production still supplies an unsupported native executor, so customer
execution remains on the existing legacy path.

## Testing

- Focused batch capture, routing, and PreparedStatement tests: 69
passed.
- Live serverless warehouse validation passed for individual,
parameterized rewrite, and interpolated rewrite flows.
- Isaac review completed with zero final findings.

## Additional Notes to the Reviewer

This PR does not add a Thrift/SEA native transport implementation,
result extraction, fallback, telemetry, or customer-visible behavior
changes.

NO_CHANGELOG=true

---------

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>
@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-integration-tests branch from 9fb5f65 to 69036d7 Compare November 16, 2026 10:10
@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-sea branch from f2b870e to 9690269 Compare November 16, 2026 10:10

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-sea branch from 9690269 to a1c2023 Compare November 16, 2026 11:25
@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-integration-tests branch from 69036d7 to b200ff9 Compare November 16, 2026 11:25

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Nit

Looks good: the test-only coverage is consistent with how the other fake-service tests are set up. The SEA side checks the ordered parameter_sets body, and the Thrift side is checked through binaryEqualTo replay of the recorded requests. Skipping close() for Thrift REPLAY follows the pattern already used in the existing fake-service tests.

Other findings

  • ⚪ Nit — setUp rewrites the Thrift fake-service route property from localhost to 127.0.0.1, and no other fake-service test does this. Please add a one-line comment explaining why this test needs it. Without the comment, the next person to re-record may drop it or copy it without knowing what it's for. Also note that System.getProperty(routeProperty) throws an NPE if the extension hasn't set the property.

Base automatically changed from sreekanth-db/stack/native-batch-sea to main November 16, 2026 12:14
Record real SEA and Thrift responses to guard native parameter batching end to end.

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>
Validate exact parameter values and array order so replay coverage detects reordered native batches.

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>
Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>
@sreekanth-db
sreekanth-db force-pushed the sreekanth-db/stack/native-batch-integration-tests branch from b200ff9 to 7437989 Compare November 16, 2026 12:19

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

@sreekanth-db
sreekanth-db added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 860ba47 Nov 16, 2026
39 of 41 checks passed
@sreekanth-db
sreekanth-db deleted the sreekanth-db/stack/native-batch-integration-tests branch November 16, 2026 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-coverage Skip the coverage fan-out for this PR (no tracking issue opened in databricks-driver-test)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants