Skip to content

feat: support configurable NQ for concurrent Milvus searches - #874

Merged
jamesgao-jpg merged 1 commit into
zilliztech:mainfrom
yanbinyang:codex/nq-batch-search
Sep 22, 2026
Merged

jamesgao-jpg merged 1 commit into
zilliztech:mainfrom
yanbinyang:codex/nq-batch-search

Conversation

@yanbinyang

@yanbinyang yanbinyang commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #873. Concurrent searches currently send one query vector per request and do not expose a per-request query batch size. VectorDBBench therefore cannot directly measure Milvus and Zilliz Cloud batch-search workloads with NQ greater than 1.

I added nq to the common case configuration and CLI (--nq, or nq in YAML), with the existing behavior preserved by a default of 1. Milvus advertises batch-search support through a client capability and implements one SDK search call for each fixed-size query batch; Zilliz Cloud inherits this implementation. The concurrent runner wraps through the query dataset to keep batches full, counts successful query vectors in qps and conc_qps_list, records latency per successful batch request, and excludes failed or incomplete batches from successful query-vector counts. Serial latency and recall remain single-query measurements, while unsupported backends and workloads reject nq > 1 before data loading.

A CLI unit test verifies that the configured NQ reaches the generated task configuration. Live Milvus validation used a custom runner's run_by_dur() path with NQ=64 on Cohere and BioASQ at 1M/10M and topK=10/100; all eight combinations reported no failed batches. This was not an end-to-end run through the standard CLI, and Zilliz Cloud was not live-tested.

@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yanbinyang
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@yanbinyang
yanbinyang force-pushed the codex/nq-batch-search branch from 8a19043 to 5625392 Compare September 15, 2026 12:07
@yanbinyang
yanbinyang marked this pull request as draft September 16, 2026 02:39
@yanbinyang
yanbinyang force-pushed the codex/nq-batch-search branch from 5625392 to b5bf5bc Compare September 16, 2026 03:00
@yanbinyang
yanbinyang marked this pull request as ready for review September 16, 2026 03:00
@yanbinyang

Copy link
Copy Markdown
Contributor Author

/assign @XuanYang-cn

Hi, when you have time, could you please take a look? The workflow is awaiting maintainer approval. Thanks!

@yanbinyang
yanbinyang force-pushed the codex/nq-batch-search branch from b5bf5bc to bbf1e38 Compare September 21, 2026 05:57
Signed-off-by: YangYanbin <warlock.yyb@alibaba-inc.com>
@yanbinyang
yanbinyang force-pushed the codex/nq-batch-search branch from bbf1e38 to 40df7e3 Compare September 21, 2026 07:31
@yanbinyang

Copy link
Copy Markdown
Contributor Author

Hi @jamesgao-jpg, I have rebased this PR on main and added regression coverage in 40df7e3. All CI checks pass. When convenient, could you please take a look? Thanks!

@jamesgao-jpg

Copy link
Copy Markdown
Collaborator

/lgtm

@jamesgao-jpg
jamesgao-jpg merged commit 1760db1 into zilliztech:main Sep 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Support configurable NQ for concurrent vector search

3 participants