feat(bigquery): add Storage Read API slow-path fallback for row-based query() - #14409
jinseopkim0 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request implements support for executing slow-path query jobs in the Arrow results format by introducing a fallback mechanism (queryFallbackArrow) that streams result rows via the BigQuery Storage Read API. Feedback on these changes suggests optimizing performance by retrieving the estimated row count directly from the ReadSession instead of making an extra synchronous API call to fetch the destination table. Additionally, the reviewer recommends simplifying the page creation logic by passing null instead of instantiating dummy TableDataPageFetcher objects when there are no pages or streams to fetch.
…atedRowCount in slow-path fallback
95df637 to
4c70cf6
Compare
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request implements support for the Arrow query results format on the slow query execution path. It replaces the previous UnsupportedOperationException with a fallback mechanism (queryFallbackArrow) that executes the query job, awaits its completion, and streams the result rows in Arrow format using the BigQuery Storage Read API. Additionally, ArrowQueryPageFetcher has been updated to support a custom stream name, and a comprehensive unit test has been added to verify the fallback behavior. There are no review comments to address, and no further feedback is provided.
This PR implements slow-path execution fallback for row-based queries requesting Arrow results format (
QueryResultsFormat.ARROW). When queries cannot be evaluated via the fast query path (such as queries writing to destination tables or exceeding fast-path limits), BigQuery job execution is triggered and table results are streamed via the BigQuery Storage Read API in Arrow format.Follow-up PR stacked on top of #14405.