Skip to content

feat(bigquery): add Storage Read API slow-path fallback for row-based query() - #14409

Draft
jinseopkim0 wants to merge 2 commits into
feat-bigquery-arrow-query-rowbasedfrom
feat-bigquery-arrow-query-slowpath-fallback
Draft

jinseopkim0 wants to merge 2 commits into
feat-bigquery-arrow-query-rowbasedfrom
feat-bigquery-arrow-query-slowpath-fallback

Conversation

@jinseopkim0

Copy link
Copy Markdown
Contributor

This PR implements slow-path execution fallback for row-based queries requesting Arrow results format (QueryResultsFormat.ARROW). When queries cannot be evaluated via the fast query path (such as queries writing to destination tables or exceeding fast-path limits), BigQuery job execution is triggered and table results are streamed via the BigQuery Storage Read API in Arrow format.

Follow-up PR stacked on top of #14405.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements support for executing slow-path query jobs in the Arrow results format by introducing a fallback mechanism (queryFallbackArrow) that streams result rows via the BigQuery Storage Read API. Feedback on these changes suggests optimizing performance by retrieving the estimated row count directly from the ReadSession instead of making an extra synchronous API call to fetch the destination table. Additionally, the reviewer recommends simplifying the page creation logic by passing null instead of instantiating dummy TableDataPageFetcher objects when there are no pages or streams to fetch.

@jinseopkim0
jinseopkim0 added this pull request to stack #14406 September 17, 2026 02:05
@jinseopkim0
jinseopkim0 force-pushed the feat-bigquery-arrow-query-slowpath-fallback branch from 95df637 to 4c70cf6 Compare September 17, 2026 02:19
@jinseopkim0

Copy link
Copy Markdown
Contributor Author

@gemini-code-assist review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements support for the Arrow query results format on the slow query execution path. It replaces the previous UnsupportedOperationException with a fallback mechanism (queryFallbackArrow) that executes the query job, awaits its completion, and streams the result rows in Arrow format using the BigQuery Storage Read API. Additionally, ArrowQueryPageFetcher has been updated to support a custom stream name, and a comprehensive unit test has been added to verify the fallback behavior. There are no review comments to address, and no further feedback is provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant