Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion knowledge/repos/vllm-omni/ci/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ sources: [.buildkite/cuda/pipeline.yml, docs/contributing/ci/test_system_overvie
- 处理 `@hardware_test`/`hardware_marks`、SKU 与 `cards_N` 自动 marker 或按卡数切分时,先看
[测试分级与 markers](guides/test-tiers.md),再按 OMNI-CI-1a 审核 lane 的真实收集与执行。
- Qwen3-TTS Base/CustomVoice 的 ready/merge oracle 与 source dependency 见
[TTS CI 规则](rules-tts.md)。
[TTS CI 规则](rules-tts.md)(含 task↔checkpoint 配对与 speaker-embedding miss-EOS retry)。


## MiniCPM-o 4.5 full-payload regression coverage

Expand Down
19 changes: 17 additions & 2 deletions knowledge/repos/vllm-omni/ci/rules-tts.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "vLLM-Omni TTS CI 规则"
created: 2026-09-05
updated: 2026-09-05
updated: 2026-09-08
type: rule
tags: [vllm-omni, ci]
sources: ["PR #6861", .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, .buildkite/amd/test-amd-merge.yml, tests/e2e/online_serving/test_qwen3_tts_base.py]
sources: ["PR #6861", .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, .buildkite/amd/test-amd-merge.yml, tests/e2e/online_serving/test_qwen3_tts_base.py, "PR #7054", "PR #6924"]
confidence: high
---

Expand All @@ -19,3 +19,18 @@ confidence: high
- 禁止:用 Ready dummy-weight 绿灯证明 Base real-weight/EOS 行为,或把 pending merge job 写成已证明通过。
- 验收:Ready `core_model` collect-only 对 Base 为空;CUDA/AMD merge marker matrix 仍含 Base offline+online;
source-file mutation 分别验证上述 CUDA scope。^[PR #6861] ^[issue #6855]

## OMNI-CI-3g — Qwen3-TTS perf/CI case 必须把 task 与 checkpoint 变体成对绑定

- 触发:修改 Qwen3-TTS benchmark registry、DFX perf JSON、nightly/merge TTS workload,或新增 `voice_clone` / `default_voice` / `voice_design` case。
- 强制:每个 task 必须绑定与之匹配的 checkpoint 变体并在 server params 中显式启动该变体:`Base` 只服务 `voice_clone`,`CustomVoice` 只服务 `default_voice`,`VoiceDesign` 只服务 `voice_design`。切换 task 时必须重启到匹配 checkpoint;仅改 benchmark request body 或 `--model` 字段而不换 server,不构成有效 coverage。
- 强制:registry/README/CI JSON 三处必须同步声明支持矩阵,防止 validation 已收紧后仍把旧 task 发到错误 checkpoint,令 suite 以 HTTP 400 失败。
- 禁止:沿用“相邻变体应该也能跑”的历史假设;把 one checkpoint covers all 三种 task 写进 perf smoke;或只改文档不改实际 perf case。
- 验收:至少一条实际 perf/CI 配置分别覆盖 Base、CustomVoice、VoiceDesign 的目标 task,并证明错误配对会在 serving validation 提前失败;通过结果只绑定所配对的那个 checkpoint-task workload。^[PR #7054]

## OMNI-CI-3f — Qwen3-TTS Base speaker-embedding 测试不得钉死 `max_new_tokens`

- 触发:修改 `test_qwen3_tts_speaker_embedding`、Base dummy/x-vector embedding E2E,或依赖 serving miss-EOS retry 的 TTS CI。
- 强制:非流式请求省略 `max_new_tokens`,以便 serving 在 miss-EOS 时丢弃不完整生成并按 [Q3TTS-4b](../models/qwen3-tts/rules.md) 重试一次。raw PCM streaming 无法在 serving 内重试时,测试层可对 codec-EOS HTTP 500 或 `RemoteProtocolError` 整请求最多再试一次;无关 500 不得重试。
- 禁止:为“加快测试”固定 `max_new_tokens` 从而关掉 serving retry;把测试层成功重试写成 miss-EOS 已不是 failure;或对任意 500 盲目重试。
- 验收:CPU 单测覆盖 EOS-miss 重试、无关 500 不重试、`extra_retries=0` 上抛;GPU real-weight 路径在省略 max 的非流式与带一次 client retry 的 streaming 上通过。^[PR #6924]
15 changes: 13 additions & 2 deletions knowledge/repos/vllm-omni/ci/rules.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "vLLM-Omni CI 规则"
created: 2026-08-23
updated: 2026-09-05
updated: 2026-09-08
type: rule
tags: [vllm-omni, ci]
sources: ["PR #3422", "PR #5074", "PR #5255", "PR #5310", "PR #5402", "PR #5524", "PR #5543", "PR #5670", "PR #5713", "PR #5780", "PR #5823", "PR #5836", "PR #5957", "PR #5976", docker/Dockerfile.ci, docker/Dockerfile.xpu, .buildkite/intel/scripts/run-xpu-test.sh, .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, "PR #5845", "PR #5872", "PR #6008", "PR #6048", "PR #6056", "PR #6096", "PR #6102", "PR #6202", "PR #6208", "PR #6273", "PR #6293", "PR #6311", "PR #6339", "PR #6343", "PR #6468", "PR #6523", "PR #6613", "PR #6555", "PR #6650", .buildkite/common/scripts/run_cov_split.sh, pyproject.toml, tests/helpers/tests/test_mark.py, tools/pre_commit/check_test_marks.py, .buildkite/common/scripts/upload_pipeline.py, .buildkite/cuda/test-nightly.yml, .buildkite/cuda/test-weekly.yml, .buildkite/npu/test-npu-nightly.yml, .pre-commit-config.yaml, tests/helpers/clean.py, tests/helpers/client.py, tests/helpers/mark.py, tests/helpers/runtime.py, tests/helpers/stage_config.py, tests/buildkite/test_upload_pipeline.py, tests/dfx/perf/scripts/run_benchmark.py, tests/dfx/perf/tests/test_minicpmo_4_5.json, tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json, tests/dfx/perf/tests/test_qwen_image_vllm_omni.json, tests/dfx/stability/, tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py, tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_flux_kontext_expansion.py, tests/e2e/online_serving/test_minicpmo_4_5.py, tests/e2e/online_serving/test_minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_minicpmo_4_5_expansion.py, tests/e2e/online_serving/test_qwen_image_expansion.py, tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py, tests/model_tests/diffusion/diff_model_builders.py, tests/model_tests/diffusion/model_settings.py, tests/model_tests/diffusion/test_alignment.py, tools/nightly/run_nightly_jobs.sh, tools/pre_commit/check_tts_adapter.py, tests/tools/test_check_tts_adapter.py, .buildkite/amd/scripts/bootstrap-amd-omni.sh, .buildkite/amd/test-amd-merge.yml, .buildkite/amd/test-amd-ready.yml, tests/diffusion/distributed/test_tensor_parallel.py, tests/diffusion/offloader/test_diffusion_layerwise_offload.py, "PR #6704", tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json, tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json, "PR #6743", "PR #6696", vllm_omni/benchmarks/metrics/metrics.py, vllm_omni/benchmarks/patch/patch.py, tests/benchmarks/metrics/test_metrics.py, tests/benchmarks/patch/test_patch.py, "PR #6674", docker/Dockerfile.npu, docker/Dockerfile.npu.a3, docker/Dockerfile.npu.ci, docker/Dockerfile.npu.ci.a3, "PR #6818", "PR #6830", "PR #6884", tests/diffusion/conftest.py, tests/diffusion/attention/test_flash_attn.py, tests/buildkite/test_amd_pipeline.py, tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py, "PR #6947"]
sources: ["PR #3422", "PR #5074", "PR #5255", "PR #5310", "PR #5402", "PR #5524", "PR #5543", "PR #5670", "PR #5713", "PR #5780", "PR #5823", "PR #5836", "PR #5957", "PR #5976", docker/Dockerfile.ci, docker/Dockerfile.xpu, .buildkite/intel/scripts/run-xpu-test.sh, .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, "PR #5845", "PR #5872", "PR #6008", "PR #6048", "PR #6056", "PR #6096", "PR #6102", "PR #6202", "PR #6208", "PR #6273", "PR #6293", "PR #6311", "PR #6339", "PR #6343", "PR #6468", "PR #6523", "PR #6613", "PR #6555", "PR #6650", .buildkite/common/scripts/run_cov_split.sh, pyproject.toml, tests/helpers/tests/test_mark.py, tools/pre_commit/check_test_marks.py, .buildkite/common/scripts/upload_pipeline.py, .buildkite/cuda/test-nightly.yml, .buildkite/cuda/test-weekly.yml, .buildkite/npu/test-npu-nightly.yml, .pre-commit-config.yaml, tests/helpers/clean.py, tests/helpers/client.py, tests/helpers/mark.py, tests/helpers/runtime.py, tests/helpers/stage_config.py, tests/buildkite/test_upload_pipeline.py, tests/dfx/perf/scripts/run_benchmark.py, tests/dfx/perf/tests/test_minicpmo_4_5.json, tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json, tests/dfx/perf/tests/test_qwen_image_vllm_omni.json, tests/dfx/stability/, tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py, tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_flux_kontext_expansion.py, tests/e2e/online_serving/test_minicpmo_4_5.py, tests/e2e/online_serving/test_minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_minicpmo_4_5_expansion.py, tests/e2e/online_serving/test_qwen_image_expansion.py, tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py, tests/model_tests/diffusion/diff_model_builders.py, tests/model_tests/diffusion/model_settings.py, tests/model_tests/diffusion/test_alignment.py, tools/nightly/run_nightly_jobs.sh, tools/pre_commit/check_tts_adapter.py, tests/tools/test_check_tts_adapter.py, .buildkite/amd/scripts/bootstrap-amd-omni.sh, .buildkite/amd/test-amd-merge.yml, .buildkite/amd/test-amd-ready.yml, tests/diffusion/distributed/test_tensor_parallel.py, tests/diffusion/offloader/test_diffusion_layerwise_offload.py, "PR #6704", tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json, tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json, "PR #6743", "PR #6696", vllm_omni/benchmarks/metrics/metrics.py, vllm_omni/benchmarks/patch/patch.py, tests/benchmarks/metrics/test_metrics.py, tests/benchmarks/patch/test_patch.py, "PR #6674", docker/Dockerfile.npu, docker/Dockerfile.npu.a3, docker/Dockerfile.npu.ci, docker/Dockerfile.npu.ci.a3, "PR #6818", "PR #6830", "PR #6884", tests/diffusion/conftest.py, tests/diffusion/attention/test_flash_attn.py, tests/buildkite/test_amd_pipeline.py, tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py, "PR #6947", "PR #7118"]
confidence: high
---

Expand All @@ -28,6 +28,9 @@ confidence: high
| diffusion tiny builder、model settings、alignment exclusion、重模型 OOM | `OMNI-CI-2d` | `tests/model_tests/diffusion/{diff_model_builders,model_settings,test_alignment}.py` → common offline tests |
| perf baseline、hardware label、DFX result artifact、assert-baseline | `OMNI-CI-3a` | `tests/dfx/conftest.py`、`tests/dfx/perf/scripts/run_benchmark.py`、`run_diffusion_benchmark.py`、`tests/dfx/perf/tests/**` |
| ASR transcript、文本相似度或词汇归一化 | `OMNI-CI-3d` | `tests/helpers/media.py::{preprocess_text, cosine_similarity_text}` → `tests/helpers/tests/test_media.py` |
| Qwen3-TTS perf/CI task↔checkpoint 配对、Base speaker-embedding miss-EOS retry | [OMNI-CI-3g](rules-tts.md)、[OMNI-CI-3f](rules-tts.md) | `tests/dfx/perf/tests/**` TTS JSON → serving validation;`test_qwen3_tts_speaker_embedding` |
| scheduled CUDA mirror secret env、`HF_TOKEN` alias、GPU preset expansion | `OMNI-CI-1f` | `upload_pipeline.py` GPU preset expansion → pod env / step commands |

| AMD/ROCm CI timeout、quarantine、diagnostic hang、memory assertion、Qwen3-TTS argv 或 Qwen3-Omni sleep/abort control plane | [OMNI-CI-2f](rules-amd.md#omni-ci-2f-amd-ci-stabilization-必须保留有界执行与测量信号) | `.buildkite/amd/**` → `tests/buildkite/test_amd_pipeline.py` / `ci/qwen3_omni_moe_colocate_async.yaml` → target test |

## OMNI-CI-1a — 硬件 lane 必须真实收集并执行目标路径
Expand Down Expand Up @@ -260,3 +263,11 @@ PR 报告的 collection=1 和 local two-card B300 184.97 s 是该精确 workload
- 验收:`"eight kilohertz"` 与 `"8 kHz"` 归一化后一致,且
`cosine_similarity_text(...)` 约等于 `1.0`;焦点 helper 回归测试在 CPU 上通过。
^[PR #6947]

## OMNI-CI-1f — scheduled CUDA mirror jobs 的 secret env 必须避开 Buildkite 同名覆盖

- 触发:修改 CUDA mirror hardware preset、scheduled build env、Hugging Face token 注入,或 `upload_pipeline.py` 对 GPU preset 的 command expansion。
- 强制:Kubernetes secret 不得直接以 `HF_TOKEN` 注入 CUDA pod env;必须使用 collision-proof alias,并在 job shell 开头恢复导出到 `HF_TOKEN`,这样 scheduled build 级 env merge 也不会把 secret-backed token 覆盖成空值或低权限值。该恢复只适用于实际消费 HF_TOKEN 的 CUDA mirror preset;NPU lane 保持原合同。
- 强制:preset expansion tests 必须同时断言 pod env 中存在 alias、不再存在裸 `HF_TOKEN`,并验证 L4/H100/B200 全部 CUDA mirror preset 的 commands 都 prepend 同一恢复语句;非 CUDA preset 不得无故注入该 wrapper。
- 禁止:依赖 Buildkite/Kubernetes 的 env merge 顺序碰巧正确;只修一个 GPU family;或把本地 shell/export 成功外推为 secret 已进入 pod env。
- 验收:render 后的每个 CUDA mirror step 都保留 secret alias 和 runtime export,scheduled build 的同名 `HF_TOKEN` 不再导致 gated-model jobs 401;NPU steps 的 env 与 commands 保持不变。^[PR #7118]
3 changes: 2 additions & 1 deletion knowledge/repos/vllm-omni/components/configuration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ sources: ["claude-workflow-starter-private@296ea45", vllm_omni/config/]
| 遇到什么 | 查看哪里 |
|---|---|
| 理解配置从 deploy、CLI、默认 factory 到 structured/legacy config 的稳定边界 | [配置构造架构](architecture.md) |
| 根据 PR 描述直达 strict schema、pipeline sampling constraints、deploy/topology、composable strategy 或显存配置的规则组与第一批源码 | [配置开发门禁与代码地图](rules.md) |
| 根据 PR 描述直达 strict schema、pipeline sampling constraints、deploy/topology、composable strategy、global engine args 的 owner 校验或显存配置的规则组与第一批源码 | [配置开发门禁与代码地图](rules.md) |

| 对象存储 URI、HF cache snapshot 或空 `config.json` 的模型名称回退与 pipeline 路由 | [模型引用路由规则](rules-model-reference-routing.md) |
| stage full-payload transport capability、topology-owned projection 或 deploy/CLI override rejection | [stage transport capability](rules-stage-transport.md) |
| diffusion Ulysses SymmMem transport flag 的 deploy/CLI/default-stage projection | [diffusion parallel transport rules](rules-diffusion-parallel-transport.md) |
Expand Down
Loading
Loading