From 5ec9111aa6a0f41f229568bdcf2be1534aa9a1f9 Mon Sep 17 00:00:00 2001 From: tzhouam Date: Thu, 10 Sep 2026 15:25:20 +0800 Subject: [PATCH 1/2] knowledge: land the curated 2026-09-08 intake batch The bot distilled rows 50-69 (20 merged vllm-omni PRs) into 19 rules on 2026-09-08/09 but could never publish them: three hourly push failures (publish token not yet deployed) parked the rows and the branch was left orphaned in the host work clone (see omni-reviewbot#53). This lands that content, curated by hand the same way as the 09-09 batch: - IDs that already headed sections elsewhere renamed so every ID heads exactly one section tree-wide: OMNI-CI-3e -> OMNI-CI-3g, OMNI-CI-1e -> OMNI-CI-1f, DIFF-1o -> DIFF-1ah, DIFF-2ab -> DIFF-2af. - Every cited PR added to its page's `sources:` frontmatter. - Routing: Direct/review-group table rows for all 19 rules on the owner entry pages, and the nearest `_index.md` rows extended. - rules-system-runtime.md appends DIFF-4aa/4ab after the 09-09 batch's DIFF-4ac (stacked on knowledge/intake-2026-09-09). - MMH3-2j reconciled with the new MMH3-2p: its exact-artifact clauses (single v1.0 filename, alpha 128, fixed 4-step/shift-6 schedule, Ref2VA rejected outright) now defer to the artifact-derived TurboSpec that MMH3-2p defines, while its lifecycle, packing and DLO requirements stay in force. Validators: check_knowledge_tree 0 errors, check_wiki_lint 0 errors, test_knowledge_source green. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_018MAFmwP3u5qAMDgHfM6LFi --- knowledge/repos/vllm-omni/ci/_index.md | 3 +- knowledge/repos/vllm-omni/ci/rules-tts.md | 19 ++++++++++-- knowledge/repos/vllm-omni/ci/rules.md | 15 +++++++-- .../components/configuration/_index.md | 3 +- .../components/configuration/rules.md | 14 +++++++-- .../vllm-omni/components/diffusion/_index.md | 9 ++++-- .../components/diffusion/rules-attention.md | 12 +++++-- .../diffusion/rules-checkpoint-loading.md | 11 +++++-- .../diffusion/rules-output-lifecycle.md | 11 +++++-- .../diffusion/rules-system-runtime.md | 16 +++++++++- .../vllm-omni/components/diffusion/rules.md | 9 ++++-- .../components/distributed/_index.md | 3 +- .../vllm-omni/components/distributed/rules.md | 11 +++++-- .../components/model-executor/_index.md | 3 +- .../model-executor/rules-runtime-hot-paths.md | 11 +++++-- .../components/model-executor/rules.md | 3 +- .../vllm-omni/models/boogu-image/_index.md | 3 +- .../vllm-omni/models/boogu-image/rules.md | 21 +++++++++++-- .../vllm-omni/models/hunyuan-image3/_index.md | 3 +- .../vllm-omni/models/hunyuan-image3/rules.md | 13 ++++++-- .../vllm-omni/models/minicpm-o-4-5/_index.md | 3 +- .../models/minicpm-o-4-5/rules-duplex.md | 11 +++++-- .../vllm-omni/models/minicpm-o-4-5/rules.md | 3 +- .../vllm-omni/models/minimax-h3/_index.md | 6 ++-- .../models/minimax-h3/rules-cache-task.md | 31 +++++++++++++++---- .../models/minimax-h3/rules-media.md | 11 +++++-- .../vllm-omni/models/minimax-h3/rules.md | 5 +-- .../vllm-omni/models/qwen-image/_index.md | 3 +- .../vllm-omni/models/qwen-image/rules.md | 14 +++++++-- 29 files changed, 227 insertions(+), 53 deletions(-) diff --git a/knowledge/repos/vllm-omni/ci/_index.md b/knowledge/repos/vllm-omni/ci/_index.md index 85fbc6b33..5726ad069 100644 --- a/knowledge/repos/vllm-omni/ci/_index.md +++ b/knowledge/repos/vllm-omni/ci/_index.md @@ -15,7 +15,8 @@ sources: [.buildkite/cuda/pipeline.yml, docs/contributing/ci/test_system_overvie - 处理 `@hardware_test`/`hardware_marks`、SKU 与 `cards_N` 自动 marker 或按卡数切分时，先看 [测试分级与 markers](guides/test-tiers.md)，再按 OMNI-CI-1a 审核 lane 的真实收集与执行。 - Qwen3-TTS Base/CustomVoice 的 ready/merge oracle 与 source dependency 见 - [TTS CI 规则](rules-tts.md)。 + [TTS CI 规则](rules-tts.md)（含 task↔checkpoint 配对与 speaker-embedding miss-EOS retry）。 + ## MiniCPM-o 4.5 full-payload regression coverage diff --git a/knowledge/repos/vllm-omni/ci/rules-tts.md b/knowledge/repos/vllm-omni/ci/rules-tts.md index dc6a20652..07e1d51cf 100644 --- a/knowledge/repos/vllm-omni/ci/rules-tts.md +++ b/knowledge/repos/vllm-omni/ci/rules-tts.md @@ -1,10 +1,10 @@ --- title: "vLLM-Omni TTS CI 规则" created: 2026-09-05 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, ci] -sources: ["PR #6861", .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, .buildkite/amd/test-amd-merge.yml, tests/e2e/online_serving/test_qwen3_tts_base.py] +sources: ["PR #6861", .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, .buildkite/amd/test-amd-merge.yml, tests/e2e/online_serving/test_qwen3_tts_base.py, "PR #7054", "PR #6924"] confidence: high --- @@ -19,3 +19,18 @@ confidence: high - 禁止：用 Ready dummy-weight 绿灯证明 Base real-weight/EOS 行为，或把 pending merge job 写成已证明通过。 - 验收：Ready `core_model` collect-only 对 Base 为空；CUDA/AMD merge marker matrix 仍含 Base offline+online； source-file mutation 分别验证上述 CUDA scope。^[PR #6861] ^[issue #6855] + +## OMNI-CI-3g — Qwen3-TTS perf/CI case 必须把 task 与 checkpoint 变体成对绑定 + +- 触发：修改 Qwen3-TTS benchmark registry、DFX perf JSON、nightly/merge TTS workload，或新增 `voice_clone` / `default_voice` / `voice_design` case。 +- 强制：每个 task 必须绑定与之匹配的 checkpoint 变体并在 server params 中显式启动该变体：`Base` 只服务 `voice_clone`，`CustomVoice` 只服务 `default_voice`，`VoiceDesign` 只服务 `voice_design`。切换 task 时必须重启到匹配 checkpoint；仅改 benchmark request body 或 `--model` 字段而不换 server，不构成有效 coverage。 +- 强制：registry/README/CI JSON 三处必须同步声明支持矩阵，防止 validation 已收紧后仍把旧 task 发到错误 checkpoint，令 suite 以 HTTP 400 失败。 +- 禁止：沿用“相邻变体应该也能跑”的历史假设；把 one checkpoint covers all 三种 task 写进 perf smoke；或只改文档不改实际 perf case。 +- 验收：至少一条实际 perf/CI 配置分别覆盖 Base、CustomVoice、VoiceDesign 的目标 task，并证明错误配对会在 serving validation 提前失败；通过结果只绑定所配对的那个 checkpoint-task workload。^[PR #7054] + +## OMNI-CI-3f — Qwen3-TTS Base speaker-embedding 测试不得钉死 `max_new_tokens` + +- 触发：修改 `test_qwen3_tts_speaker_embedding`、Base dummy/x-vector embedding E2E，或依赖 serving miss-EOS retry 的 TTS CI。 +- 强制：非流式请求省略 `max_new_tokens`，以便 serving 在 miss-EOS 时丢弃不完整生成并按 [Q3TTS-4b](../models/qwen3-tts/rules.md) 重试一次。raw PCM streaming 无法在 serving 内重试时，测试层可对 codec-EOS HTTP 500 或 `RemoteProtocolError` 整请求最多再试一次；无关 500 不得重试。 +- 禁止：为“加快测试”固定 `max_new_tokens` 从而关掉 serving retry；把测试层成功重试写成 miss-EOS 已不是 failure；或对任意 500 盲目重试。 +- 验收：CPU 单测覆盖 EOS-miss 重试、无关 500 不重试、`extra_retries=0` 上抛；GPU real-weight 路径在省略 max 的非流式与带一次 client retry 的 streaming 上通过。^[PR #6924] diff --git a/knowledge/repos/vllm-omni/ci/rules.md b/knowledge/repos/vllm-omni/ci/rules.md index c43010682..123ece092 100644 --- a/knowledge/repos/vllm-omni/ci/rules.md +++ b/knowledge/repos/vllm-omni/ci/rules.md @@ -1,10 +1,10 @@ --- title: "vLLM-Omni CI 规则" created: 2026-08-23 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, ci] -sources: ["PR #3422", "PR #5074", "PR #5255", "PR #5310", "PR #5402", "PR #5524", "PR #5543", "PR #5670", "PR #5713", "PR #5780", "PR #5823", "PR #5836", "PR #5957", "PR #5976", docker/Dockerfile.ci, docker/Dockerfile.xpu, .buildkite/intel/scripts/run-xpu-test.sh, .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, "PR #5845", "PR #5872", "PR #6008", "PR #6048", "PR #6056", "PR #6096", "PR #6102", "PR #6202", "PR #6208", "PR #6273", "PR #6293", "PR #6311", "PR #6339", "PR #6343", "PR #6468", "PR #6523", "PR #6613", "PR #6555", "PR #6650", .buildkite/common/scripts/run_cov_split.sh, pyproject.toml, tests/helpers/tests/test_mark.py, tools/pre_commit/check_test_marks.py, .buildkite/common/scripts/upload_pipeline.py, .buildkite/cuda/test-nightly.yml, .buildkite/cuda/test-weekly.yml, .buildkite/npu/test-npu-nightly.yml, .pre-commit-config.yaml, tests/helpers/clean.py, tests/helpers/client.py, tests/helpers/mark.py, tests/helpers/runtime.py, tests/helpers/stage_config.py, tests/buildkite/test_upload_pipeline.py, tests/dfx/perf/scripts/run_benchmark.py, tests/dfx/perf/tests/test_minicpmo_4_5.json, tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json, tests/dfx/perf/tests/test_qwen_image_vllm_omni.json, tests/dfx/stability/, tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py, tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_flux_kontext_expansion.py, tests/e2e/online_serving/test_minicpmo_4_5.py, tests/e2e/online_serving/test_minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_minicpmo_4_5_expansion.py, tests/e2e/online_serving/test_qwen_image_expansion.py, tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py, tests/model_tests/diffusion/diff_model_builders.py, tests/model_tests/diffusion/model_settings.py, tests/model_tests/diffusion/test_alignment.py, tools/nightly/run_nightly_jobs.sh, tools/pre_commit/check_tts_adapter.py, tests/tools/test_check_tts_adapter.py, .buildkite/amd/scripts/bootstrap-amd-omni.sh, .buildkite/amd/test-amd-merge.yml, .buildkite/amd/test-amd-ready.yml, tests/diffusion/distributed/test_tensor_parallel.py, tests/diffusion/offloader/test_diffusion_layerwise_offload.py, "PR #6704", tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json, tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json, "PR #6743", "PR #6696", vllm_omni/benchmarks/metrics/metrics.py, vllm_omni/benchmarks/patch/patch.py, tests/benchmarks/metrics/test_metrics.py, tests/benchmarks/patch/test_patch.py, "PR #6674", docker/Dockerfile.npu, docker/Dockerfile.npu.a3, docker/Dockerfile.npu.ci, docker/Dockerfile.npu.ci.a3, "PR #6818", "PR #6830", "PR #6884", tests/diffusion/conftest.py, tests/diffusion/attention/test_flash_attn.py, tests/buildkite/test_amd_pipeline.py, tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py, "PR #6947"] +sources: ["PR #3422", "PR #5074", "PR #5255", "PR #5310", "PR #5402", "PR #5524", "PR #5543", "PR #5670", "PR #5713", "PR #5780", "PR #5823", "PR #5836", "PR #5957", "PR #5976", docker/Dockerfile.ci, docker/Dockerfile.xpu, .buildkite/intel/scripts/run-xpu-test.sh, .buildkite/cuda/test-merge.yml, .buildkite/cuda/test-ready.yml, "PR #5845", "PR #5872", "PR #6008", "PR #6048", "PR #6056", "PR #6096", "PR #6102", "PR #6202", "PR #6208", "PR #6273", "PR #6293", "PR #6311", "PR #6339", "PR #6343", "PR #6468", "PR #6523", "PR #6613", "PR #6555", "PR #6650", .buildkite/common/scripts/run_cov_split.sh, pyproject.toml, tests/helpers/tests/test_mark.py, tools/pre_commit/check_test_marks.py, .buildkite/common/scripts/upload_pipeline.py, .buildkite/cuda/test-nightly.yml, .buildkite/cuda/test-weekly.yml, .buildkite/npu/test-npu-nightly.yml, .pre-commit-config.yaml, tests/helpers/clean.py, tests/helpers/client.py, tests/helpers/mark.py, tests/helpers/runtime.py, tests/helpers/stage_config.py, tests/buildkite/test_upload_pipeline.py, tests/dfx/perf/scripts/run_benchmark.py, tests/dfx/perf/tests/test_minicpmo_4_5.json, tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json, tests/dfx/perf/tests/test_qwen_image_vllm_omni.json, tests/dfx/stability/, tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py, tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_flux_kontext_expansion.py, tests/e2e/online_serving/test_minicpmo_4_5.py, tests/e2e/online_serving/test_minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_minicpmo_4_5_expansion.py, tests/e2e/online_serving/test_qwen_image_expansion.py, tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py, tests/model_tests/diffusion/diff_model_builders.py, tests/model_tests/diffusion/model_settings.py, tests/model_tests/diffusion/test_alignment.py, tools/nightly/run_nightly_jobs.sh, tools/pre_commit/check_tts_adapter.py, tests/tools/test_check_tts_adapter.py, .buildkite/amd/scripts/bootstrap-amd-omni.sh, .buildkite/amd/test-amd-merge.yml, .buildkite/amd/test-amd-ready.yml, tests/diffusion/distributed/test_tensor_parallel.py, tests/diffusion/offloader/test_diffusion_layerwise_offload.py, "PR #6704", tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json, tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json, "PR #6743", "PR #6696", vllm_omni/benchmarks/metrics/metrics.py, vllm_omni/benchmarks/patch/patch.py, tests/benchmarks/metrics/test_metrics.py, tests/benchmarks/patch/test_patch.py, "PR #6674", docker/Dockerfile.npu, docker/Dockerfile.npu.a3, docker/Dockerfile.npu.ci, docker/Dockerfile.npu.ci.a3, "PR #6818", "PR #6830", "PR #6884", tests/diffusion/conftest.py, tests/diffusion/attention/test_flash_attn.py, tests/buildkite/test_amd_pipeline.py, tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py, "PR #6947", "PR #7118"] confidence: high --- @@ -28,6 +28,9 @@ confidence: high | diffusion tiny builder、model settings、alignment exclusion、重模型 OOM | `OMNI-CI-2d` | `tests/model_tests/diffusion/{diff_model_builders,model_settings,test_alignment}.py` → common offline tests | | perf baseline、hardware label、DFX result artifact、assert-baseline | `OMNI-CI-3a` | `tests/dfx/conftest.py`、`tests/dfx/perf/scripts/run_benchmark.py`、`run_diffusion_benchmark.py`、`tests/dfx/perf/tests/**` | | ASR transcript、文本相似度或词汇归一化 | `OMNI-CI-3d` | `tests/helpers/media.py::{preprocess_text, cosine_similarity_text}` → `tests/helpers/tests/test_media.py` | +| Qwen3-TTS perf/CI task↔checkpoint 配对、Base speaker-embedding miss-EOS retry | [OMNI-CI-3g](rules-tts.md)、[OMNI-CI-3f](rules-tts.md) | `tests/dfx/perf/tests/**` TTS JSON → serving validation；`test_qwen3_tts_speaker_embedding` | +| scheduled CUDA mirror secret env、`HF_TOKEN` alias、GPU preset expansion | `OMNI-CI-1f` | `upload_pipeline.py` GPU preset expansion → pod env / step commands | + | AMD/ROCm CI timeout、quarantine、diagnostic hang、memory assertion、Qwen3-TTS argv 或 Qwen3-Omni sleep/abort control plane | [OMNI-CI-2f](rules-amd.md#omni-ci-2f-amd-ci-stabilization-必须保留有界执行与测量信号) | `.buildkite/amd/**` → `tests/buildkite/test_amd_pipeline.py` / `ci/qwen3_omni_moe_colocate_async.yaml` → target test | ## OMNI-CI-1a — 硬件 lane 必须真实收集并执行目标路径 @@ -260,3 +263,11 @@ PR 报告的 collection=1 和 local two-card B300 184.97 s 是该精确 workload - 验收：`"eight kilohertz"` 与 `"8 kHz"` 归一化后一致，且 `cosine_similarity_text(...)` 约等于 `1.0`；焦点 helper 回归测试在 CPU 上通过。 ^[PR #6947] + +## OMNI-CI-1f — scheduled CUDA mirror jobs 的 secret env 必须避开 Buildkite 同名覆盖 + +- 触发：修改 CUDA mirror hardware preset、scheduled build env、Hugging Face token 注入，或 `upload_pipeline.py` 对 GPU preset 的 command expansion。 +- 强制：Kubernetes secret 不得直接以 `HF_TOKEN` 注入 CUDA pod env；必须使用 collision-proof alias，并在 job shell 开头恢复导出到 `HF_TOKEN`，这样 scheduled build 级 env merge 也不会把 secret-backed token 覆盖成空值或低权限值。该恢复只适用于实际消费 HF_TOKEN 的 CUDA mirror preset；NPU lane 保持原合同。 +- 强制：preset expansion tests 必须同时断言 pod env 中存在 alias、不再存在裸 `HF_TOKEN`，并验证 L4/H100/B200 全部 CUDA mirror preset 的 commands 都 prepend 同一恢复语句；非 CUDA preset 不得无故注入该 wrapper。 +- 禁止：依赖 Buildkite/Kubernetes 的 env merge 顺序碰巧正确；只修一个 GPU family；或把本地 shell/export 成功外推为 secret 已进入 pod env。 +- 验收：render 后的每个 CUDA mirror step 都保留 secret alias 和 runtime export，scheduled build 的同名 `HF_TOKEN` 不再导致 gated-model jobs 401；NPU steps 的 env 与 commands 保持不变。^[PR #7118] diff --git a/knowledge/repos/vllm-omni/components/configuration/_index.md b/knowledge/repos/vllm-omni/components/configuration/_index.md index 31b9f81c4..fb9b7eeef 100644 --- a/knowledge/repos/vllm-omni/components/configuration/_index.md +++ b/knowledge/repos/vllm-omni/components/configuration/_index.md @@ -35,7 +35,8 @@ sources: ["claude-workflow-starter-private@296ea45", vllm_omni/config/] | 遇到什么 | 查看哪里 | |---|---| | 理解配置从 deploy、CLI、默认 factory 到 structured/legacy config 的稳定边界 | [配置构造架构](architecture.md) | -| 根据 PR 描述直达 strict schema、pipeline sampling constraints、deploy/topology、composable strategy 或显存配置的规则组与第一批源码 | [配置开发门禁与代码地图](rules.md) | +| 根据 PR 描述直达 strict schema、pipeline sampling constraints、deploy/topology、composable strategy、global engine args 的 owner 校验或显存配置的规则组与第一批源码 | [配置开发门禁与代码地图](rules.md) | + | 对象存储 URI、HF cache snapshot 或空 `config.json` 的模型名称回退与 pipeline 路由 | [模型引用路由规则](rules-model-reference-routing.md) | | stage full-payload transport capability、topology-owned projection 或 deploy/CLI override rejection | [stage transport capability](rules-stage-transport.md) | | diffusion Ulysses SymmMem transport flag 的 deploy/CLI/default-stage projection | [diffusion parallel transport rules](rules-diffusion-parallel-transport.md) | diff --git a/knowledge/repos/vllm-omni/components/configuration/rules.md b/knowledge/repos/vllm-omni/components/configuration/rules.md index 18fdaa04e..c7aac4265 100644 --- a/knowledge/repos/vllm-omni/components/configuration/rules.md +++ b/knowledge/repos/vllm-omni/components/configuration/rules.md @@ -1,10 +1,10 @@ --- title: "vLLM-Omni 配置开发门禁" created: 2026-07-16 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, components, config] -sources: ["claude-workflow-starter-private@296ea45", "PR #4281", "PR #5031", "PR #5073", "PR #5671", "PR #5678", "zuiho-kai/claude-workflow-starter@c217fc6", vllm_omni/config/model.py, vllm_omni/config/stage_config.py, vllm_omni/config/config_factory.py, vllm_omni/config/omni_config.py, vllm_omni/config/composable_parallel/, vllm_omni/deploy/qwen3_omni_moe.yaml, vllm_omni/engine/stage_init_utils.py, tests/config/test_config_factory.py, tests/engine/test_arg_utils.py, tests/engine/test_stage_engine_args.py, "PR #4795", "PR #5842", "PR #6082", "PR #6156", "PR #5741", "PR #6068", "PR #4765", "PR #5666", "PR #4222", "PR #5604", "PR #6293", "PR #6094", "vllm_omni/diffusion/data.py", "PR #6050", "PR #6322", "vllm_omni/config/pipeline_registry.py", "vllm_omni/diffusion/models/pi0_pipeline_config.py", "vllm_omni/diffusion/models/pi0/pipeline_pi0.py", "PR #5048", "PR #6458", "PR #6102", "PR #6308", "PR #6182", "PR #4820", "PR #6619", "PR #6680", "PR #6422", vllm_omni/deploy/higgs_multimodal_qwen3.yaml] +sources: ["claude-workflow-starter-private@296ea45", "PR #4281", "PR #5031", "PR #5073", "PR #5671", "PR #5678", "zuiho-kai/claude-workflow-starter@c217fc6", vllm_omni/config/model.py, vllm_omni/config/stage_config.py, vllm_omni/config/config_factory.py, vllm_omni/config/omni_config.py, vllm_omni/config/composable_parallel/, vllm_omni/deploy/qwen3_omni_moe.yaml, vllm_omni/engine/stage_init_utils.py, tests/config/test_config_factory.py, tests/engine/test_arg_utils.py, tests/engine/test_stage_engine_args.py, "PR #4795", "PR #5842", "PR #6082", "PR #6156", "PR #5741", "PR #6068", "PR #4765", "PR #5666", "PR #4222", "PR #5604", "PR #6293", "PR #6094", "vllm_omni/diffusion/data.py", "PR #6050", "PR #6322", "vllm_omni/config/pipeline_registry.py", "vllm_omni/diffusion/models/pi0_pipeline_config.py", "vllm_omni/diffusion/models/pi0/pipeline_pi0.py", "PR #5048", "PR #6458", "PR #6102", "PR #6308", "PR #6182", "PR #4820", "PR #6619", "PR #6680", "PR #6422", vllm_omni/deploy/higgs_multimodal_qwen3.yaml, "PR #5929"] --- # vLLM-Omni 配置开发门禁 @@ -28,6 +28,8 @@ sources: ["claude-workflow-starter-private@296ea45", "PR #4281", "PR #5031", "PR | pipeline `sampling_constraints`、`StageConfig.to_omegaconf()`、runtime stage config | `stage-config-propagation`：`VOMNI-CFG-1o` | `merge_pipeline_deploy` → `StageConfig.to_omegaconf()` → `engine.stage_configs` | | stage transport、`requires_full_payload_input`、topology projection 或 override rejection | `stage-transport`：`VOMNI-CFG-1p` | `stage_config.py::{StagePipelineConfig,_build_engine_args}` → `omni_config.py::_build_model_config` → `engine/arg_utils.py::OmniEngineArgs` | | SymmMem Ulysses transport | [VOMNI-CFG-1q](rules-diffusion-parallel-transport.md) + `DIFF-4x` | deploy → parallel config → Ulysses | +| global stage engine args 的 owner 校验、`diffusion_offload_config` 跨进程 mapping、compile/offload gate | `strict-normalization`：`VOMNI-CFG-1s` | `stage_config.py` 规范化 → ownership 校验 → 按 stage `execution_type` 投影 → offload parser | + | HF cache snapshot path、空 `config.json`、name-based pipeline fallback | [`model-reference-routing`](rules-model-reference-routing.md#conf-7a-模型引用解析必须物化对象存储配置并只从受控名称组件匹配)：`CONF-7a` | `config_factory.py::{_name_match_candidate,StageConfigFactory._try_infer_model_type}` → `pipeline_registry.py::OMNI_PIPELINES` → `tests/config/test_config_factory.py::TestNameMatchCandidateSnapshotPaths` | ## 配置归一化与新老路径一致性 @@ -220,3 +222,11 @@ sources: ["claude-workflow-starter-private@296ea45", "PR #4281", "PR #5031", "PR - 验收：覆盖有效非默认值的 CLI、deploy/structured 和 direct attention-config 路径，断言最终 `backend_kwargs`；覆盖错误 backend、非正值，以及 Top-K 超过 runtime block count 的回退。 `WanDMDPipeline` 一类 checkpoint alias 还必须由 `model_index.json` 自动发现。^[PR #4820] + +## VOMNI-CFG-1s — global stage engine args 必须先验 owner，再按执行类型投影 + +- 触发：新增或修改全局 stage CLI 字段、stage-scoped alias、`diffusion_offload_config`，或调整 structured/legacy 的 stage runtime overrides。 +- 强制：先对完整规范化后的 stage CLI 映射做 ownership 校验，再按每个 stage 的 `execution_type` 投影到该 stage；显式 stage 覆盖优先，全局字段只有在目标 stage 实际拥有该字段时才可透传。任何 pipeline 中没有 owner 的显式全局 engine arg 都必须在构造前 fail closed，不能静默丢给无关 stage 或靠默认值吞掉。 +- 强制：`diffusion_offload_config` 这类跨进程序列化的 public mapping 必须保留 raw dict 进入 dataclass/config serialization，同时在 stage 构造前调用其专属 parser 做一次结构校验；兼容 alias 只能 materialize 成同一份内部策略，不能让 compile/offload 约束只检查旧布尔字段。 +- 禁止：先按 execution type 过滤再做 unknown/owner 校验；只在 direct path 校验而让 deploy/structured 漏检；把 raw mapping 先转成内部对象后再跨进程传输；或让 `full` compile compatibility 继续只识别 legacy offload flags。 +- 验收：覆盖 global 与 stage-specific override precedence、unowned global field rejection、`diffusion_offload_config` 的 direct/deploy/structured reachability，以及 compile-mode 对 module/layer/distributed-layer 三种 offload strategy 的统一 incompatibility gate。^[PR #5929] diff --git a/knowledge/repos/vllm-omni/components/diffusion/_index.md b/knowledge/repos/vllm-omni/components/diffusion/_index.md index febae7c0b..afc90cb7a 100644 --- a/knowledge/repos/vllm-omni/components/diffusion/_index.md +++ b/knowledge/repos/vllm-omni/components/diffusion/_index.md @@ -41,9 +41,11 @@ sources: ["PR #5543", vllm_omni/platforms/cuda/platform.py, vllm_omni/diffusion/ | PEFT 与 distilled LoRA、startup fusion、delta/key/alpha、Qwen/Wan transformer mapping、PEFT suspend/resume | [LoRA rules](rules-lora.md) | -| Blackwell FA4、显式/auto CUDA backend、per-role mask/SP pad；local FlashAttention deterministic opt-in、role-aware masked cross-attention、NPU packed mask-free/laser fallback、TRTLLM packed-padding 与 config propagation | [attention rules](rules-attention.md) | +| Blackwell FA4、显式/auto CUDA backend、per-role mask/SP pad；local FlashAttention deterministic opt-in、role-aware masked cross-attention、NPU packed mask-free/laser fallback、TRTLLM packed-padding 与 config propagation、equal-rank SP auto-pad fast path | [attention rules](rules-attention.md) | + | SP auto-padding、`mask_sp_padding`、dense/varlen 与 advanced UAA 边界 | [SP padding rules](rules-sp-padding.md) | -| video/audio mux、DLO DP wave、RPC reply ownership/result release、result queue、async pump、SHM ownership、async-output wait 与 shutdown | [output/runtime rules](rules-output-lifecycle.md) | +| video/audio mux、DLO DP wave、RPC reply ownership/result release、result queue、async pump、SHM ownership、async-output wait 与 shutdown、mid-stream interaction handler | [output/runtime rules](rules-output-lifecycle.md) | + | distilled continuous sigma schedule、boundary/step 语义与 modality shift | [sigma schedule rules](sigma-schedules.md) | | HunyuanImage3 Distil CFG、MeanFlow special-token / scheduler-r 合同 | [HunyuanImage3 HY3-8d](../../models/hunyuan-image3/rules.md) | | diffusion step 与 request/continuous batching；DIFF-1af BAGEL image step wave、packed state、geometry/CFG admission 与 fallback cleanup | [step and batching](step-and-batching.md) | @@ -54,6 +56,7 @@ sources: ["PR #5543", vllm_omni/platforms/cuda/platform.py, vllm_omni/diffusion/ | Cache-DiT、TeaCache 和 prefix cache | [cache acceleration](cache-acceleration.md) | | TP/PP/SP/CFG/VAE/HSDP 等并行策略 | [parallelism](parallelism.md) | | 实验性 world-model session 生命周期、LRU 与内存统计边界 | [session state](session-state.md) | -| checkpoint remap、HSDP/FSDP、final-layout HWR、registered HWR mmap DLO transport、component quantization 与在线量化加载 | [checkpoint 与加载合同](rules-checkpoint-loading.md) | +| checkpoint remap、HSDP/FSDP、final-layout HWR、registered HWR mmap DLO transport、component quantization、在线量化加载与 TorchAO `.bin` index fallback | [checkpoint 与加载合同](rules-checkpoint-loading.md) | + - [Diffusion 平台 kernel 与设备合同](rules-platform-kernels.md)：`DIFF-1t`, `DIFF-1u`, `DIFF-1v`, `DIFF-1w`。 diff --git a/knowledge/repos/vllm-omni/components/diffusion/rules-attention.md b/knowledge/repos/vllm-omni/components/diffusion/rules-attention.md index 226be8314..92aa64964 100644 --- a/knowledge/repos/vllm-omni/components/diffusion/rules-attention.md +++ b/knowledge/repos/vllm-omni/components/diffusion/rules-attention.md @@ -1,10 +1,10 @@ --- title: "Diffusion attention 规则" created: 2026-09-02 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, components, diffusion] -sources: ["PR #5543", "PR #5866", "PR #5887", "PR #5891", "PR #5897", "PR #5997", "PR #6000", "PR #6037", "PR #6518", "PR #6563", "PR #6724", "PR #6909", docs/design/feature/skip_softmax.md, docs/user_guide/diffusion/attention_backends.md, docs/user_guide/diffusion/attention_backends/trtllm.md, docs/user_guide/diffusion/attention_backends/fastvideo_vsa.md, docs/user_guide/diffusion/attention_backends/rainfusion.md, vllm_omni/config/omni_config.py, vllm_omni/config/stage_config.py, vllm_omni/diffusion/attention/backends/abstract.py, vllm_omni/diffusion/attention/backends/fastvideo_vsa.py, vllm_omni/diffusion/attention/backends/flash_attn.py, vllm_omni/diffusion/attention/backends/rainfusion_attn.py, vllm_omni/diffusion/attention/parallel/ulysses.py, vllm_omni/diffusion/diffusion_kv/paged_attention_adapter.py, vllm_omni/diffusion/models/minimax_h3/denoise_loop.py, vllm_omni/diffusion/models/minimax_h3/packed_sequence.py, vllm_omni/diffusion/data.py, vllm_omni/engine/arg_utils.py, vllm_omni/engine/async_omni_engine.py, vllm_omni/entrypoints/cli/serve.py, vllm_omni/platforms/cuda/platform.py, vllm_omni/platforms/npu/platform.py, tests/config/test_omni_config.py, tests/diffusion/attention/test_fastvideo_vsa.py, tests/diffusion/attention/test_flash_attn.py, tests/diffusion/attention/test_attention_config.py, tests/diffusion/attention/test_piecewise_attn.py, tests/diffusion/attention/test_rainfusion_plan.py, tests/diffusion/attention/test_ulysses_uaa.py, tests/diffusion/diffusion_kv/test_paged_attention_adapter.py, tests/diffusion/models/minimax_h3/test_minimax_h3_packing.py, tests/diffusion/cache/test_teacache_extractors.py, "PR #5500", "vllm_omni/diffusion/models/ltx2/ltx2_transformer.py", "PR #6070", "vllm_omni/diffusion/attention/backends/cudnn_attn.py", "PR #5614", "PR #5194", "vllm_omni/diffusion/models/hidream_o1_image/hidream_o1_image_transformer.py", "vllm_omni/diffusion/models/hidream_o1_image/pipeline_hidream_o1_image.py", "PR #6181", "vllm_omni/diffusion/cache/teacache/extractors.py", "vllm_omni/diffusion/models/longcat_image/pipeline_longcat_image.py", "vllm_omni/diffusion/models/longcat_image/pipeline_longcat_image_edit.py"] +sources: ["PR #5543", "PR #5866", "PR #5887", "PR #5891", "PR #5897", "PR #5997", "PR #6000", "PR #6037", "PR #6518", "PR #6563", "PR #6724", "PR #6909", docs/design/feature/skip_softmax.md, docs/user_guide/diffusion/attention_backends.md, docs/user_guide/diffusion/attention_backends/trtllm.md, docs/user_guide/diffusion/attention_backends/fastvideo_vsa.md, docs/user_guide/diffusion/attention_backends/rainfusion.md, vllm_omni/config/omni_config.py, vllm_omni/config/stage_config.py, vllm_omni/diffusion/attention/backends/abstract.py, vllm_omni/diffusion/attention/backends/fastvideo_vsa.py, vllm_omni/diffusion/attention/backends/flash_attn.py, vllm_omni/diffusion/attention/backends/rainfusion_attn.py, vllm_omni/diffusion/attention/parallel/ulysses.py, vllm_omni/diffusion/diffusion_kv/paged_attention_adapter.py, vllm_omni/diffusion/models/minimax_h3/denoise_loop.py, vllm_omni/diffusion/models/minimax_h3/packed_sequence.py, vllm_omni/diffusion/data.py, vllm_omni/engine/arg_utils.py, vllm_omni/engine/async_omni_engine.py, vllm_omni/entrypoints/cli/serve.py, vllm_omni/platforms/cuda/platform.py, vllm_omni/platforms/npu/platform.py, tests/config/test_omni_config.py, tests/diffusion/attention/test_fastvideo_vsa.py, tests/diffusion/attention/test_flash_attn.py, tests/diffusion/attention/test_attention_config.py, tests/diffusion/attention/test_piecewise_attn.py, tests/diffusion/attention/test_rainfusion_plan.py, tests/diffusion/attention/test_ulysses_uaa.py, tests/diffusion/diffusion_kv/test_paged_attention_adapter.py, tests/diffusion/models/minimax_h3/test_minimax_h3_packing.py, tests/diffusion/cache/test_teacache_extractors.py, "PR #5500", "vllm_omni/diffusion/models/ltx2/ltx2_transformer.py", "PR #6070", "vllm_omni/diffusion/attention/backends/cudnn_attn.py", "PR #5614", "PR #5194", "vllm_omni/diffusion/models/hidream_o1_image/hidream_o1_image_transformer.py", "vllm_omni/diffusion/models/hidream_o1_image/pipeline_hidream_o1_image.py", "PR #6181", "vllm_omni/diffusion/cache/teacache/extractors.py", "vllm_omni/diffusion/models/longcat_image/pipeline_longcat_image.py", "vllm_omni/diffusion/models/longcat_image/pipeline_longcat_image_edit.py", "PR #5717"] confidence: high --- @@ -202,3 +202,11 @@ confidence: high - 强制：以被 hook 的实际 `forward` 签名作为参数合同；extractor 必须兼容真实调用的可选字段，CFG 正负分支必须传递全部 required conditioning kwargs，尤其是 `guidance`，并保持对应 tensor 的语义与形状一致。 - 禁止：只让正分支携带 extractor 所需参数；假设 negative branch 可以省略 `guidance`；用宽泛 `**kwargs` 或不启用 CFG 的测试掩盖 hook 与 model `forward` 的签名漂移。 - 验收：TeaCache 启用且 `guidance_scale > 1` 时分别覆盖 LongCat T2I 与 Edit 的正/负分支，断言 extractor 两次均收到 `guidance` 且不抛 `TypeError`；再以固定输入、seed 和 checkpoint 对照无缓存输出。^[PR #6181] + +## DIFF-1ah — equal-rank SP auto-pad 只能在已发布合同下跳过长度 gather + +- 触发：修改 Ulysses/UAA sequence-parallel attention、forward context、auto-pad sharding metadata，或尝试移除每次 attention 的 `_all_gather_int`。 +- 强制：只有 framework-managed SP boundary 明确发布“各 rank local seq_len 相等”的 contract 时，attention 才可跳过长度 all-gather，并直接复用 `query.shape[1]` 作为每个 rank 的 local length；该 fast path 必须保留 `SymInt`，不能 `int(...)` materialize，否则会引入 host sync、graph break 和 shape specialization。混合 auto-pad/manual shard 时必须按 boundary stack 精确 push/pop，任何一层不保证等长都回到 slow path。 +- 强制：advanced UAA metadata 继续携带真实 `seq_lens` / `local_seq_len` 语义，但 fast path 可用 symbolic equal-length list 构造它们；hybrid Ulysses+Ring 仍保留自身等长检查合同。 +- 禁止：把“当前 shape 看起来相等”当成可省 collective 的证明；用单个 counter 代替 boundary stack；在 gather path 之外删掉 slow-path coverage；或把去掉 `int(...)` 的 compile 好处写成任意 SP mode 的通用结论。 +- 验收：覆盖 equal-pad fast path、manual/mixed boundary slow path、boundary pop 只影响自身层级，以及 UAA metadata 仍与各 path 一致；mutation 到 `any(...)` 或错误清空 stack 时测试必须失败。^[PR #5717] diff --git a/knowledge/repos/vllm-omni/components/diffusion/rules-checkpoint-loading.md b/knowledge/repos/vllm-omni/components/diffusion/rules-checkpoint-loading.md index 788b1e140..58c72d40f 100644 --- a/knowledge/repos/vllm-omni/components/diffusion/rules-checkpoint-loading.md +++ b/knowledge/repos/vllm-omni/components/diffusion/rules-checkpoint-loading.md @@ -1,10 +1,10 @@ --- title: "Checkpoint、加载与量化合同" created: 2026-09-04 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, components, diffusion] -sources: ["PR #5087", "PR #5088", "PR #5136", "PR #5544", "PR #5677", "PR #5737", "PR #5764", "PR #5802", "PR #5836", "PR #5839", "PR #5848", "PR #5872", "PR #5910", "PR #6070", "PR #6162", "PR #6234", "PR #6279", "PR #6445", "PR #6486", "PR #6591", "PR #6651", "PR #6573", vllm_omni/diffusion/model_loader/, vllm_omni/diffusion/model_loader/diffusers_loader.py, vllm_omni/quantization/, vllm_omni/quantization/component_config.py, vllm_omni/quantization/factory.py, vllm_omni/quantization/svdquant_config.py, vllm_omni/diffusion/distributed/hsdp.py, vllm_omni/diffusion/distributed/parallel_state.py, vllm_omni/diffusion/offloader/, tests/diffusion/model_loader/test_diffusers_loader.py, tests/diffusion/offloader/test_distributed_layerwise_backend.py, tests/diffusion/quantization/test_svdquant_config.py, tests/diffusion/quantization/test_svdquant_linear.py, tests/diffusion/quantization/test_svdquant_tp_loading.py, tests/diffusion/quantization/test_wan_autoround_mxfp4.py, "PR #5531", "PR #6722", "PR #4061"] +sources: ["PR #5087", "PR #5088", "PR #5136", "PR #5544", "PR #5677", "PR #5737", "PR #5764", "PR #5802", "PR #5836", "PR #5839", "PR #5848", "PR #5872", "PR #5910", "PR #6070", "PR #6162", "PR #6234", "PR #6279", "PR #6445", "PR #6486", "PR #6591", "PR #6651", "PR #6573", vllm_omni/diffusion/model_loader/, vllm_omni/diffusion/model_loader/diffusers_loader.py, vllm_omni/quantization/, vllm_omni/quantization/component_config.py, vllm_omni/quantization/factory.py, vllm_omni/quantization/svdquant_config.py, vllm_omni/diffusion/distributed/hsdp.py, vllm_omni/diffusion/distributed/parallel_state.py, vllm_omni/diffusion/offloader/, tests/diffusion/model_loader/test_diffusers_loader.py, tests/diffusion/offloader/test_distributed_layerwise_backend.py, tests/diffusion/quantization/test_svdquant_config.py, tests/diffusion/quantization/test_svdquant_linear.py, tests/diffusion/quantization/test_svdquant_tp_loading.py, tests/diffusion/quantization/test_wan_autoround_mxfp4.py, "PR #5531", "PR #6722", "PR #4061", "PR #6925"] confidence: high --- @@ -236,3 +236,10 @@ confidence: high 上比较真实 kernel 输出。当前合入测试未执行真实 kernel 或 released checkpoint E2E，故仅证明 loader contract；PR review 所要求的 frozen-head BF16-vs-SVDQuant quality/performance comparison 没有随本提交提供。^[PR #6162] + +## DIFF-2af — Diffusers loader 在无 safetensors index 时才解析 `.bin` TorchAO 清单 + +- 触发：修改 `diffusers_loader` 的 index discovery、`pt_weights_iterator`、TorchAO serialized checkpoint 路径，或 `allow_patterns_overrides` 与 load-format 选择。 +- 强制：无显式 override 时先解析 safetensors index（`diffusion_pytorch_model.safetensors.index.json` / `model.safetensors.index.json`），仅当二者都缺失才解析 `diffusion_pytorch_model.bin.index.json`。命中序列化 TorchAO（`quant_config.get_name()=="torchao"` 且 `is_checkpoint_torchao_serialized`）且非 safetensors 时，用自然序排序的 `pt_weights_iterator` 加载；safetensors 路径保持原 multithread/safetensors 合同。 +- 禁止：让 `.bin` index 抢在 safetensors 之前；对序列化 TorchAO `.bin` 开 multithread；或把该路径当成任意 PyTorch bin checkpoint 的通用 loader。 +- 验收：覆盖 safetensors 优先、bin index fallback、自然序 shard、TorchAO iterator 选择与非 TorchAO 负向对照。^[PR #6925] diff --git a/knowledge/repos/vllm-omni/components/diffusion/rules-output-lifecycle.md b/knowledge/repos/vllm-omni/components/diffusion/rules-output-lifecycle.md index 9fbc53f9d..12223046b 100644 --- a/knowledge/repos/vllm-omni/components/diffusion/rules-output-lifecycle.md +++ b/knowledge/repos/vllm-omni/components/diffusion/rules-output-lifecycle.md @@ -1,10 +1,10 @@ --- title: "Diffusion output 与 multiprocess runtime 规则" created: 2026-09-02 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, components, diffusion] -sources: ["PR #5550", "PR #5864", "PR #5885", "PR #5978", "PR #6750", vllm_omni/diffusion/diffusion_engine.py, vllm_omni/diffusion/executor/multiproc_executor.py, vllm_omni/diffusion/inline_stage_diffusion_client.py, vllm_omni/diffusion/io_support.py, vllm_omni/diffusion/model_metadata.py, vllm_omni/diffusion/output_formatter.py, vllm_omni/diffusion/ipc.py, vllm_omni/diffusion/sched/request_scheduler.py, vllm_omni/diffusion/stage_diffusion_proc.py, vllm_omni/diffusion/utils/media_utils.py, vllm_omni/diffusion/worker/diffusion_worker.py, tests/diffusion/test_async_output_timeout.py, tests/diffusion/test_async_output_worker.py, tests/diffusion/test_diffusion_engine.py, tests/diffusion/test_diffusion_engine_cleanup.py, tests/diffusion/test_diffusion_ipc.py, tests/diffusion/test_inline_stage_diffusion_client.py, tests/diffusion/test_ipc_async.py, tests/diffusion/test_multiproc_engine_concurrency.py, tests/diffusion/test_result_pump.py, tests/diffusion/test_stage_diffusion_proc.py, tests/entrypoints/openai_api/test_video_server.py, "PR #6023", "PR #5983", "PR #4222", "PR #6094", "PR #6255", "PR #6288", "PR #6308", "PR #6499", "PR #6749", "PR #6847", "PR #6953", vllm_omni/diffusion/data.py, vllm_omni/diffusion/registry.py, vllm_omni/diffusion/models/diffusers_adapter/pipeline_utils.py, tests/diffusion/test_diffusion_output_formatter.py, tests/diffusion/test_diffusion_plugin_hooks.py, tests/entrypoints/openai_api/test_video_pipeline_capability.py] +sources: ["PR #5550", "PR #5864", "PR #5885", "PR #5978", "PR #6750", vllm_omni/diffusion/diffusion_engine.py, vllm_omni/diffusion/executor/multiproc_executor.py, vllm_omni/diffusion/inline_stage_diffusion_client.py, vllm_omni/diffusion/io_support.py, vllm_omni/diffusion/model_metadata.py, vllm_omni/diffusion/output_formatter.py, vllm_omni/diffusion/ipc.py, vllm_omni/diffusion/sched/request_scheduler.py, vllm_omni/diffusion/stage_diffusion_proc.py, vllm_omni/diffusion/utils/media_utils.py, vllm_omni/diffusion/worker/diffusion_worker.py, tests/diffusion/test_async_output_timeout.py, tests/diffusion/test_async_output_worker.py, tests/diffusion/test_diffusion_engine.py, tests/diffusion/test_diffusion_engine_cleanup.py, tests/diffusion/test_diffusion_ipc.py, tests/diffusion/test_inline_stage_diffusion_client.py, tests/diffusion/test_ipc_async.py, tests/diffusion/test_multiproc_engine_concurrency.py, tests/diffusion/test_result_pump.py, tests/diffusion/test_stage_diffusion_proc.py, tests/entrypoints/openai_api/test_video_server.py, "PR #6023", "PR #5983", "PR #4222", "PR #6094", "PR #6255", "PR #6288", "PR #6308", "PR #6499", "PR #6749", "PR #6847", "PR #6953", vllm_omni/diffusion/data.py, vllm_omni/diffusion/registry.py, vllm_omni/diffusion/models/diffusers_adapter/pipeline_utils.py, tests/diffusion/test_diffusion_output_formatter.py, tests/diffusion/test_diffusion_plugin_hooks.py, tests/entrypoints/openai_api/test_video_pipeline_capability.py, "PR #6294"] confidence: high --- @@ -169,3 +169,10 @@ confidence: high - 强制：单 GPU 的 `uni` 路径必须在 engine 进程内构造并驱动 `WorkerWrapperBase`，仍初始化一个 rank 的 distributed environment，但不得创建 worker subprocess、`MessageQueue` 或 IPC 输出段；`collective_rpc` 直接调用 worker，保留 reply-rank 的返回形状，recoverable request error 留在请求级，sticky accelerator fault 必须 latch executor failure、只触发一次 failure callback，并让 `check_health()` 抛出 `EngineDeadError`。 - 禁止：单 GPU 默认继续走 `mp`，多 GPU 选择 `uni`，把 inline `timeout` 描述成可强制的 RPC deadline，或把所有 worker exception 都升级为 engine death；设备上下文已中毒后不得继续报告健康，也不得让部分构造失败或重复 shutdown 遗留 worker、callback、缓存或 device allocation。 - 验收：覆盖 omitted/显式 backend/多 GPU 的选择矩阵、直接 RPC 与返回形状、recoverable error、CUDA/NPU sticky fault、callback once、health failure、部分构造 cleanup 和幂等 shutdown；真实单 GPU smoke 还须验证一 rank group、`InlineStageDiffusionClient`、真实 `DiffusionOutput` 流程、worker shutdown，以及固定 seed 下与 pinned `mp` 的输出一致性，mock 不能替代这些证据。 ^[PR #6308] + +## DIFF-1ag — mid-stream interaction 必须经 modality handler 在 chunk 边界应用 + +- 触发：修改 `InteractionCoordinator`、`InteractionMixin`、`SupportsInteractionApply`、prompt/camera 等 interaction handler、chunk-boundary apply，或试图恢复 `PromptUpdateMixin`/`prompt_update.py`。 +- 强制：runner 与 pipeline 共有同一 `InteractionCoordinator`；仅对已注册 modality 构造 handler；`enqueue` 按 modality 路由；`apply_at_chunk_boundary` 以稳定顺序推进（prompt 优先，再及其他 modality）；per-request session 落在 `StepRequestState.interaction_sessions`，chunk ACK 元数据写入 `interaction_chunk_metadata`。未注册 modality 必须 fail closed。当前公开行为保持 prompt-only，直到另有 PR 注册并验收非 prompt handler。 +- 禁止：在 denoise 中途旁路 coordinator 直接改 `prompt_embeds`；在非 chunk 边界静默应用 interaction；未注册就接受 camera/`multi_modal_data`；或把内部 handler 重构当成用户 API 已扩展。 +- 验收：覆盖 prompt enqueue→chunk-boundary apply→ACK event id、未注册 modality 拒绝，以及无 session 的 modality 跳过；不得用本框架落地证明 camera/MM 路径已可用。^[PR #6294] diff --git a/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md b/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md index 2fb28e1c4..6bec90c78 100644 --- a/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md +++ b/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md @@ -4,7 +4,7 @@ created: 2026-09-03 updated: 2026-09-09 type: rule tags: [vllm-omni, components, diffusion] -sources: ["PR #5255", "PR #5344", "PR #5543", "PR #5838", "PR #6094", "PR #6102", "PR #6385", "PR #6340", "PR #6714", "PR #6814", "PR #6563", "PR #5716", "PR #6786", vllm_omni/diffusion/attention/, vllm_omni/diffusion/attention/parallel/ulysses.py, vllm_omni/diffusion/attention/parallel/ring_kernels.py, vllm_omni/diffusion/diffusion_kv/, vllm_omni/diffusion/distributed/cfg_parallel.py, vllm_omni/diffusion/distributed/parallel_state.py, vllm_omni/diffusion/worker/diffusion_model_runner.py, vllm_omni/platforms/interface.py, vllm_omni/platforms/npu/platform.py, tests/diffusion/diffusion_kv/, tests/diffusion/distributed/test_cfg_parallel.py, tests/diffusion/attention/test_piecewise_attn.py, tests/diffusion/attention/test_ulysses_uaa.py, "PR #5491", "PR #5194", "vllm_omni/diffusion/data.py", "vllm_omni/diffusion/utils/hf_utils.py", "PR #7041"] +sources: ["PR #5255", "PR #5344", "PR #5543", "PR #5838", "PR #6094", "PR #6102", "PR #6385", "PR #6340", "PR #6714", "PR #6814", "PR #6563", "PR #5716", "PR #6786", vllm_omni/diffusion/attention/, vllm_omni/diffusion/attention/parallel/ulysses.py, vllm_omni/diffusion/attention/parallel/ring_kernels.py, vllm_omni/diffusion/diffusion_kv/, vllm_omni/diffusion/distributed/cfg_parallel.py, vllm_omni/diffusion/distributed/parallel_state.py, vllm_omni/diffusion/worker/diffusion_model_runner.py, vllm_omni/platforms/interface.py, vllm_omni/platforms/npu/platform.py, tests/diffusion/diffusion_kv/, tests/diffusion/distributed/test_cfg_parallel.py, tests/diffusion/attention/test_piecewise_attn.py, tests/diffusion/attention/test_ulysses_uaa.py, "PR #5491", "PR #5194", "vllm_omni/diffusion/data.py", "vllm_omni/diffusion/utils/hf_utils.py", "PR #7041", "PR #6463", "PR #6844"] confidence: high --- @@ -212,3 +212,17 @@ confidence: high - 强制：字段声明为 `str` 且 canonical “无 cache” sentinel 为 `"none"` 时，key 缺失才允许走 env/`"none"` 默认；调用方传入显式 `None`（如 CLI `default=None`）必须在 kwargs 边界规范成 `"none"`，且不得回落到 `DIFFUSION_CACHE_BACKEND` 查找。下游校验可把残留 `None` 当 `"none"` 作 defense-in-depth，但不能代替边界规范化。 - 禁止：只在 key 缺失时默认、让显式 `None` 泄漏成 typed `str` 字段上的 `None`；把 `"none"` 与 env 覆盖混为一谈；把某一 pipeline 的 `None` 特判当成唯一修复点。 - 验收：`from_kwargs`/`create_default_diffusion` 对 `cache_backend=None` 断言最终为 `"none"`；至少一个严格 `("none", …)` 校验路径覆盖 `None` 与 `"none"` 均可通过，未知 backend 仍拒绝。^[PR #7041] + +## DIFF-4aa — AR-Diffusion paged K/V 必须分存储分配，禁止共享 `(2, …)` 视图 + +- 触发：修改 AR-Diffusion `allocate_kv_pool_with_views`、paged-write custom op 的 `mutates_args`、compiled graph 下的 key/value pool，或把 K/V 重新合并为单一 allocation。 +- 强制：每层 K 与 V 各自 `torch.empty` 独立 storage，布局仍为 `(num_blocks, block_size, num_kv_heads, head_dim)`；`kv_pools[layer][0]/[1]` 与 flat slot view 继续寻址，但不共享 `untyped_storage`。paged-write 若同时 mutate K 与 V，共享 storage 会让 Inductor reinplace 只重写第一个参数，并使 `auto_functionalized_v2` 克隆整份 V pool。 +- 禁止：恢复 `empty(2, num_blocks, …)` 再 `kv[0]`/`kv[1]` 切片；或把“eager 能跑”当成 compiled 路径无整池拷贝的证明。 +- 验收：断言各层 K/V `data_ptr` 不同且写 V 不扰动 K；编译生成代码不得出现 pool-sized clone；有界硬件证明 compiled 路径可运行且无 per-step 整池拷贝。^[PR #6463] + +## DIFF-4ab — AR-Diffusion stepwise 必须以一次 request 驱动整段 rollout + +- 触发：修改 `SupportsStepExecution`、`ARDiffusionModelRunner` 的 stepwise 绑定、AR block/`post_decode` chunk 提交、`step_execution`+`streaming_output` deploy，或把外部 tick 循环重新当作 session 边界。 +- 强制：一次 `generate()` 驱动整段 rollout：`prepare_encode` 只跑一次，随后多轮 `denoise_step`/`step_scheduler`，由 `post_decode` 产出每个 AR chunk。request-mode 与 stepwise 必须共用同一份 block/DMD math，禁止维护两套可漂移实现。runner 仅为实现该合同的 pipeline 打开 `step_execution`；绑定 runner-owned KV 时 `session_id == request_id`；错误路径 fail-closed 释放 session，完成或 scheduler abort 时退役。持有 AR-Diffusion paged KV 的 stage 必须保持 `max_num_seqs=1`。 +- 禁止：用多次 `generate()`/tick 冒充同一 session；在 denoise step 内提交本应属于 `post_decode` 的 clean-x0 KV commit；或把逐步 camera/prompt mid-request interaction 写成已由本合同覆盖。 +- 验收：覆盖 stepwise 与 request-mode 的共享 math 边界、session bind/release、abort/completion 退役，以及 `max_num_seqs=1` 拓扑；request-scoped camera script 等模型字段另由模型 owner 验收，不得外推为通用 mid-request interaction。^[PR #6844] diff --git a/knowledge/repos/vllm-omni/components/diffusion/rules.md b/knowledge/repos/vllm-omni/components/diffusion/rules.md index d51d7eb09..0ceecf093 100644 --- a/knowledge/repos/vllm-omni/components/diffusion/rules.md +++ b/knowledge/repos/vllm-omni/components/diffusion/rules.md @@ -20,11 +20,14 @@ confidence: high | PR 描述信号 | 规则入口 | |---|---| | graph/eager、compile、solver、RNG、dtype/device | `DIFF-1a`–`DIFF-1j` | -| checkpoint remap、HWR、HSDP、offload、quantization | `DIFF-2a`–`DIFF-2z`；[checkpoint/loading](rules-checkpoint-loading.md) | +| checkpoint remap、HWR、HSDP、offload、quantization、TorchAO `.bin` index fallback | `DIFF-2a`–`DIFF-2z`、`DIFF-2af`；[checkpoint/loading](rules-checkpoint-loading.md) | + | component、multi-DiT、LoRA、request cache lifecycle | [component lifecycle](rules-component-lifecycle.md)、[LoRA](rules-lora.md) | | quality evidence、distributed VAE | `DIFF-3a`、`DIFF-3b` | -| paged KV、attention backend/parallel、platform、admission | `DIFF-4a`–`DIFF-4x`；[system runtime](rules-system-runtime.md) | -| worker reply、shutdown、final output/formatter | `DIFF-1s`、`DIFF-1aa`、`DIFF-4z`；[output/runtime](rules-output-lifecycle.md) | +| paged KV、attention backend/parallel、platform、admission、AR-Diffusion K/V pool 与 stepwise rollout | `DIFF-4a`–`DIFF-4x`、`DIFF-4aa`–`DIFF-4ac`；[system runtime](rules-system-runtime.md)；equal-rank SP auto-pad `DIFF-1ah` [attention](rules-attention.md) | + +| worker reply、shutdown、final output/formatter、mid-stream interaction | `DIFF-1s`、`DIFF-1aa`、`DIFF-1ag`、`DIFF-4z`；[output/runtime](rules-output-lifecycle.md) | + | service/forward/step metrics | `DIFF-5a`、`DIFF-5b`；[metrics](rules-metrics.md) | | worker title/rank observability | `DIFF-4w`；[worker observability](rules-worker-observability.md) | diff --git a/knowledge/repos/vllm-omni/components/distributed/_index.md b/knowledge/repos/vllm-omni/components/distributed/_index.md index feeefe973..0c98510e6 100644 --- a/knowledge/repos/vllm-omni/components/distributed/_index.md +++ b/knowledge/repos/vllm-omni/components/distributed/_index.md @@ -53,4 +53,5 @@ sources: [vllm_omni/distributed/omni_connectors/, vllm_omni/distributed/omni_coo | 已修过的 connector/端口产品坑 | [connector pitfalls](connector-pitfalls.md) | | 选择和配置 connector backend | [connector backends](connector-backends.md) | | 跨 stage `async_chunk` 流式语义 | [async chunk](async-chunk.md) | -| TP KV receive consensus、chunk boundary 与 active-window 合同 | [distributed rules](rules.md) | +| TP KV receive consensus、chunk boundary、active-window 合同与 Mooncake TCP write completion | [distributed rules](rules.md) | + diff --git a/knowledge/repos/vllm-omni/components/distributed/rules.md b/knowledge/repos/vllm-omni/components/distributed/rules.md index 756cb4c57..d693a2d07 100644 --- a/knowledge/repos/vllm-omni/components/distributed/rules.md +++ b/knowledge/repos/vllm-omni/components/distributed/rules.md @@ -1,10 +1,10 @@ --- title: "Distributed 传输规则" created: 2026-08-05 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, components, distributed] -sources: ["PR #5744", "PR #5976", "PR #6001", "PR #6089", "PR #6834", vllm_omni/diffusion/distributed/parallel_state.py, tests/diffusion/distributed/test_expert_parallel_layout.py, vllm_omni/distributed/omni_connectors/adapter.py, vllm_omni/distributed/omni_connectors/kv_transfer_manager.py, vllm_omni/distributed/omni_connectors/transfer_adapter/chunk_transfer_adapter.py, vllm_omni/distributed/omni_connectors/transfer_adapter/base.py, vllm_omni/worker/omni_connector_model_runner_mixin.py, tests/distributed/omni_connectors/test_kv_recv_tp_consensus.py, tests/distributed/omni_connectors/test_chunk_transfer_adapter.py, tests/worker/test_omni_connector_mixin.py, "PR #5146", "PR #6021", "PR #6033", "PR #6360", "PR #6406", "PR #6626", "PR #6529"] +sources: ["PR #5744", "PR #5976", "PR #6001", "PR #6089", "PR #6834", vllm_omni/diffusion/distributed/parallel_state.py, tests/diffusion/distributed/test_expert_parallel_layout.py, vllm_omni/distributed/omni_connectors/adapter.py, vllm_omni/distributed/omni_connectors/kv_transfer_manager.py, vllm_omni/distributed/omni_connectors/transfer_adapter/chunk_transfer_adapter.py, vllm_omni/distributed/omni_connectors/transfer_adapter/base.py, vllm_omni/worker/omni_connector_model_runner_mixin.py, tests/distributed/omni_connectors/test_kv_recv_tp_consensus.py, tests/distributed/omni_connectors/test_chunk_transfer_adapter.py, tests/worker/test_omni_connector_mixin.py, "PR #5146", "PR #6021", "PR #6033", "PR #6360", "PR #6406", "PR #6626", "PR #6529", "PR #7136"] confidence: high --- @@ -135,3 +135,10 @@ confidence: high - 强制：当前 `OmniRequestOutput` 按其 dataclass 字段重建，不能落入基类 `RequestOutput` 的动态属性编码路径；decoder 只构造已知字段，并在旧 wire format 含嵌套 `request_output` 时完成一次向扁平对象的内容合并。基类 `RequestOutput` 的字段集合必须跟随 pinned upstream 合同。 - 禁止：让 subclass 走基类序列化而丢失 diffusion/stage 字段或形成递归；不得把未知 key 直接传给 `RequestOutput`/dataclass 构造器；upstream 已移除的 `multi_modal_placeholders` 不得仅为旧动态属性重新作为当前字段恢复。 - 验收：msgpack round-trip 分别覆盖 pipeline 文本输出和 diffusion 图像输出，保持 request metadata、prompt、token、outputs、finished、stage 字段及图像内容；旧嵌套 wire payload 能解码为扁平对象，编码不递归且解码后的字段可被直接消费。^[PR #5146] + +## DIST-1k — Mooncake TCP write completion 必须表示目的端已应用 + +- 触发：升级 `mooncake-transfer-engine`、修改 TCP `batch_transfer_sync_write`/completion 语义，或让零拷贝 consumer 在 write completion 后立即读取 destination buffer。 +- 强制：CUDA Mooncake 依赖必须钉在包含 acknowledged TCP writes 的版本（`0.3.12` 起）；`batch_transfer_sync_write` 返回成功后，destination 内存必须已更新到可读状态，不能只表示本地 send 入队。回归必须在 receiver 被挂起时证明 completion 不会提前返回，恢复后再断言目的端字节一致。 +- 禁止：降级到无 ACK 的 TCP 实现却不更新测试合同；把 RDMA 路径的 completion 语义外推到 TCP；或在 completion 前启动依赖 destination 内容的零拷贝消费。 +- 验收：强制 TCP 的挂起-恢复测试覆盖「停止期间不完成、恢复后成功且字节匹配」；依赖声明与该测试同步变更。^[PR #7136] diff --git a/knowledge/repos/vllm-omni/components/model-executor/_index.md b/knowledge/repos/vllm-omni/components/model-executor/_index.md index db1b12492..b06c63245 100644 --- a/knowledge/repos/vllm-omni/components/model-executor/_index.md +++ b/knowledge/repos/vllm-omni/components/model-executor/_index.md @@ -40,5 +40,6 @@ sources: [] | loader 的 dtype/config 获取、fused shard 与多模块 checkpoint 载入 | [loader 合同](rules-loader-contract.md) | | shared image example envelope 与 model_extras 参数声明 | [image task envelope 合同](rules-image-task-envelope.md) | | Omni 输出类型与字段/复制合同 | [输出类型合同](rules-output-contract.md) | -| 采样循环不变量、热路径缓存、AR 音频侧路 | [运行时热路径合同](rules-runtime-hot-paths.md) | +| 采样循环不变量、热路径缓存、AR 音频侧路、preprocess phase/one-token prefill | [运行时热路径合同](rules-runtime-hot-paths.md) | + | NPU runner、ROCm 分页注意力、NPU 模型补丁 | [平台后端合同](rules-platform-backends.md) | diff --git a/knowledge/repos/vllm-omni/components/model-executor/rules-runtime-hot-paths.md b/knowledge/repos/vllm-omni/components/model-executor/rules-runtime-hot-paths.md index 986fc4cf2..5b07098bb 100644 --- a/knowledge/repos/vllm-omni/components/model-executor/rules-runtime-hot-paths.md +++ b/knowledge/repos/vllm-omni/components/model-executor/rules-runtime-hot-paths.md @@ -1,10 +1,10 @@ --- title: "运行时热路径合同" created: 2026-09-04 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, components, model-executor] -sources: ["PR #4765", "PR #5068", "PR #5174", "PR #5666", vllm_omni/worker/, "PR #5452", "vllm_omni/worker/sparse_audio.py", "vllm_omni/worker/sampling_utils.py", "PR #5048", "PR #6424", "PR #6454", vllm_omni/data_entry_keys.py, "vllm_omni/model_executor/models/cosyvoice3/cosyvoice3.py", "vllm_omni/model_executor/models/cosyvoice3/code2wav_core/hifigan.py", "vllm_omni/model_executor/stage_input_processors/cosyvoice3.py", "PR #6458", "PR #6317"] +sources: ["PR #4765", "PR #5068", "PR #5174", "PR #5666", vllm_omni/worker/, "PR #5452", "vllm_omni/worker/sparse_audio.py", "vllm_omni/worker/sampling_utils.py", "PR #5048", "PR #6424", "PR #6454", vllm_omni/data_entry_keys.py, "vllm_omni/model_executor/models/cosyvoice3/cosyvoice3.py", "vllm_omni/model_executor/models/cosyvoice3/code2wav_core/hifigan.py", "vllm_omni/model_executor/stage_input_processors/cosyvoice3.py", "PR #6458", "PR #6317", "PR #7136"] confidence: high --- @@ -94,3 +94,10 @@ confidence: high 不得被拿来与旧 `torch.multinomial` 比较。当前实现把 `+inf` 也排除，并在整行无 finite candidate 时从 `sample()` 抛错、可能终止整个 EngineCore；这两项是 review 接受的 follow-up 风险，不能描述成 per-request 隔离或通用正无穷采样语义。^[PR #6424] + +## EXEC-4f — 单 token prefill 尾块必须保留 preprocess，不得当 decode/MTP + +- 触发：修改 runner `_preprocess` 的 `_omni_prompt_len` / `_omni_num_computed_tokens` / `_omni_is_prefill`、phase 路由、normal/batched MTP eligibility，或自定义 AR talker 的 chunked-prefill 行为。 +- 强制：上述字段为 Python `int`/`bool`，按真实 prompt 进度逐行生产。长度为 1 的 **prefill** 行（含 chunked prefill 的最后一 token）必须保留 preprocess embedding，且不得进入 runner-managed MTP；只有单 token **decode** 行才可走 MTP。out-of-tree 模型虽见下划线前缀，仍按公开 contract 消费。 +- 禁止：用当前 span 长度把 one-token prefill 当成 decode；只在模型 helper 手喂 metadata 却不测 runner producer；或把该合同当成已为所有模型开启 chunked prefill。 +- 验收：直接调用生产 `_preprocess`，在 mixed batch 中对照应/不应进入 MTP 的行，覆盖单 token prompt/tail、cached progress、normal/batched hooks；真实权重 E2E 至少锁一条 one-token prefill tail。^[PR #7136] diff --git a/knowledge/repos/vllm-omni/components/model-executor/rules.md b/knowledge/repos/vllm-omni/components/model-executor/rules.md index 7051e9def..092a4303b 100644 --- a/knowledge/repos/vllm-omni/components/model-executor/rules.md +++ b/knowledge/repos/vllm-omni/components/model-executor/rules.md @@ -22,7 +22,8 @@ sources: [vllm_omni/worker/gpu_model_runner.py, vllm_omni/worker/gpu_ar_model_ru | async output、MTP graph、output types | `EXEC-4c`、`EXEC-5a`–`5b`、`EXEC-7a`–`7b` | | NPU/ROCm/MUSA/XPU runner/backend | `EXEC-10a`–`EXEC-13h`；[platform backends](rules-platform-backends.md) | | image/video task envelope、`model_extras` | `EXEC-6a`–`EXEC-6d`；[image task envelope](rules-image-task-envelope.md) | -| sampling/codec hot path | `EXEC-8a`、`EXEC-9a`、`EXEC-11a`–`11h`；[runtime hot paths](rules-runtime-hot-paths.md) | +| preprocess phase/one-token prefill、sampling/codec hot path | `EXEC-4f`、`EXEC-8a`、`EXEC-9a`、`EXEC-11a`–`11h`；[runtime hot paths](rules-runtime-hot-paths.md) | + ## 完整代码路由 diff --git a/knowledge/repos/vllm-omni/models/boogu-image/_index.md b/knowledge/repos/vllm-omni/models/boogu-image/_index.md index 57a455965..8565cfe29 100644 --- a/knowledge/repos/vllm-omni/models/boogu-image/_index.md +++ b/knowledge/repos/vllm-omni/models/boogu-image/_index.md @@ -51,4 +51,5 @@ confidence: high | 遇到什么 | 查看哪里 | |---|---| -| revision/component loader、Edit-Turbo checkpoint、real RoPE/platform selection、Edit CFG branch/rank/VAE decode 合同，或 request-level T2I batching、TI2I batch=1 gate、request-major mask/generator/output mapping | [Boogu rules](rules.md) | +| revision/component loader、Edit-Turbo checkpoint、real RoPE/platform selection、Edit CFG branch/rank/VAE decode 合同，或 request-level T2I batching、TI2I batch=1 gate、request-major mask/generator/output mapping、Turbo DMD schedule 或 TorchAO FP8 scope | [Boogu rules](rules.md) | + diff --git a/knowledge/repos/vllm-omni/models/boogu-image/rules.md b/knowledge/repos/vllm-omni/models/boogu-image/rules.md index 1046549fc..965c94d54 100644 --- a/knowledge/repos/vllm-omni/models/boogu-image/rules.md +++ b/knowledge/repos/vllm-omni/models/boogu-image/rules.md @@ -1,10 +1,10 @@ --- title: "Boogu-Image 规则" created: 2026-09-05 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, models, diffusion] -sources: ["PR #6701", "PR #6571", "PR #6786", "PR #6968", docs/models/supported_models.md, recipes/Boogu/Boogu-Image.md, vllm_omni/config/omni_config.py, vllm_omni/diffusion/models/boogu_image/pipeline_boogu_image.py, vllm_omni/diffusion/models/boogu_image/boogu_image_transformer.py, vllm_omni/entrypoints/utils.py, tests/diffusion/models/boogu_image/test_pipeline_boogu_image.py, tests/diffusion/models/boogu_image/test_boogu_image_transformer.py, tests/entrypoints/test_utils.py, tests/e2e/online_serving/test_boogu_image_edit.py] +sources: ["PR #6701", "PR #6571", "PR #6786", "PR #6968", docs/models/supported_models.md, recipes/Boogu/Boogu-Image.md, vllm_omni/config/omni_config.py, vllm_omni/diffusion/models/boogu_image/pipeline_boogu_image.py, vllm_omni/diffusion/models/boogu_image/boogu_image_transformer.py, vllm_omni/entrypoints/utils.py, tests/diffusion/models/boogu_image/test_pipeline_boogu_image.py, tests/diffusion/models/boogu_image/test_boogu_image_transformer.py, tests/entrypoints/test_utils.py, tests/e2e/online_serving/test_boogu_image_edit.py, "PR #6699", "PR #6925"] confidence: high --- @@ -15,6 +15,8 @@ confidence: high | PR 描述信号 | 规则 | |---|---| | revision、RoPE、Edit CFG/rank/VAE、request batching | `BOOGU-1a`–`BOOGU-1d` | +| Turbo DMD few-step schedule/renoise、TorchAO FP8 DiT-only quantization | `BOOGU-1e`、`BOOGU-1f` | + ## BOOGU-1a — global revision 必须填入每个未显式设 revision 的 stage @@ -66,3 +68,18 @@ confidence: high batched TI2I raise、B=2/N=2 distinct-mask 行序、request-major generators 和每请求 output slice； CFG B=2 isolation 至少分别改变 partner 的 positive prompt、negative prompt 和 seed。CPU/mock tests 不证明 GPU CFG、TI2I batch safety、吞吐或 QPS/perf claim。^[PR #6968] + +## BOOGU-1e — Turbo DMD student 路径必须与 Base/Edit scheduler 路径严格分流 + +- 触发：新增或修改 `BooguImageTurboPipeline`、Turbo few-step sampling、explicit `timesteps`/sigmas、renoise loop，或 request batching。 +- 强制：Turbo 必须走独立的 DMD student contract，而不是复用 Base/Edit 的 regular scheduler+CFG path。其 sigma schedule 只能来自两种来源：显式 1D finite sequence（接受 `[0,1]` 或 `0..1000` 训练 timestep 归一化后形式），或由 `conditioning_sigma..1.0` 线性生成的 `num_inference_steps` 个 ascending sigma points；任一非法 shape、空序列、越界值或非正 step count 都必须 fail closed。 +- 强制：每个 Turbo step 执行 upstream `x + (1-sigma) * velocity` 预测，并在非末步以同一 request RNG renoise 到下一 sigma；解码输出与 Base/Edit 共用 VAE/postprocess，但 Turbo request batching 必须保持 `batch=1`，因为其额外 schedule/request args 不在现有 compatibility key 中。 +- 禁止：把 Turbo 混进 regular scheduler、接受 batched Turbo requests、从 mask/CFG state 猜测 sigmas，或把 few-step path 的通过外推到 Edit/Base quality parity。 +- 验收：覆盖 explicit sigma normalization、default four-step schedule、predict/renoise step math、batch>1 rejection，以及 Turbo/Base 两条路径互不串用；真实质量结论仍需独立的 upstream-reference 与硬件 E2E 证据。^[PR #6699] + +## BOOGU-1f — 预量化 TorchAO FP8 只量化 DiT，且不走 online 量化 + +- 触发：修改 Boogu-Image 的 TorchAO/`torchao_float8_weight_only` 配置、transformer `quant_config`/layer prefix、pipeline 组件量化 scope，或官方 Base-fp8/Edit-fp8 checkpoint 加载。 +- 强制：只支持加载已序列化的 TorchAO float8_weight_only checkpoint，不在运行时把 BF16/FP16 量化成 FP8。`quant_config` 与 fully-qualified prefix 必须传到可量化 `ReplicatedLinear`/`ColumnParallelLinear` 等层；量化范围隔离在 DiT transformer，不得波及 MLLM、VAE、scheduler。 +- 禁止：宣称 online FP8；对序列化 `.bin`/`.pt` TorchAO 权重启用 multithread load；或因 shorthand 配置而量化非 DiT 组件。 +- 验收：覆盖 serialized TorchAO config、shorthand、prefix 传播，以及 DiT-only scope；真实权重 smoke 另绑 checkpoint 与硬件。^[PR #6925] diff --git a/knowledge/repos/vllm-omni/models/hunyuan-image3/_index.md b/knowledge/repos/vllm-omni/models/hunyuan-image3/_index.md index 6e3c6ab8d..0747cc236 100644 --- a/knowledge/repos/vllm-omni/models/hunyuan-image3/_index.md +++ b/knowledge/repos/vllm-omni/models/hunyuan-image3/_index.md @@ -24,7 +24,8 @@ sources: ["PR #5541", "PR #6563", "PR #4048", vllm_omni/diffusion/models/hunyuan | 遇到什么 | 查看哪里 | |---|---| -| 修改公开入口、prompt、AR→DiT、`model_extras`、shared task examples、条件图、size 或 seed | [开发快速入口和 rules](rules.md#direct-开发快速入口) | +| 修改公开入口、prompt、AR→DiT、`model_extras`、shared task examples、条件图、size、seed 或 MoE expert mapping | [开发快速入口和 rules](rules.md#direct-开发快速入口) | + | Scheduler-managed paged KV、prepared layout、CFG logical prefix、Hunyuan Q/K/V spans 或空 hash 边界 | [shared paged KV control plane](../../components/diffusion/paged-kv-control-plane.md) | | 理解模型和 vLLM-Omni 代码地图 | [architecture](architecture.md) | | HF 接入常见偏差 | [HF alignment pitfalls](hf-alignment-pitfalls.md) | diff --git a/knowledge/repos/vllm-omni/models/hunyuan-image3/rules.md b/knowledge/repos/vllm-omni/models/hunyuan-image3/rules.md index 30a414137..9e1f4a20b 100644 --- a/knowledge/repos/vllm-omni/models/hunyuan-image3/rules.md +++ b/knowledge/repos/vllm-omni/models/hunyuan-image3/rules.md @@ -1,10 +1,10 @@ --- title: "HunyuanImage3 开发规则" created: 2026-07-13 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, models, hunyuan-image3] -sources: [incidents/painterly/_index.md, hf-alignment-pitfalls.md, vllm_omni/diffusion/models/hunyuan_image3/prompt_utils.py, vllm_omni/model_extras/hunyuan_image3.py, vllm_omni/model_extras/registry.py, "PR #6094", "vllm_omni/diffusion/models/hunyuan_image3/hunyuan_image3_transformer.py", "PR #6306", "PR #6102", "PR #6563", "PR #4048", vllm_omni/diffusion/models/hunyuan_image3/pipeline_hunyuan_image3.py, vllm_omni/diffusion/models/hunyuan_image3/hunyuan_image3_tokenizer.py, vllm_omni/diffusion/models/hunyuan_image3/request_layout.py, tests/diffusion/models/hunyuan_image3/test_hunyuan_image3_step_execution.py, tests/diffusion/models/hunyuan_image3/test_image_kv_cache_manager.py] +sources: [incidents/painterly/_index.md, hf-alignment-pitfalls.md, vllm_omni/diffusion/models/hunyuan_image3/prompt_utils.py, vllm_omni/model_extras/hunyuan_image3.py, vllm_omni/model_extras/registry.py, "PR #6094", "vllm_omni/diffusion/models/hunyuan_image3/hunyuan_image3_transformer.py", "PR #6306", "PR #6102", "PR #6563", "PR #4048", vllm_omni/diffusion/models/hunyuan_image3/pipeline_hunyuan_image3.py, vllm_omni/diffusion/models/hunyuan_image3/hunyuan_image3_tokenizer.py, vllm_omni/diffusion/models/hunyuan_image3/request_layout.py, tests/diffusion/models/hunyuan_image3/test_hunyuan_image3_step_execution.py, tests/diffusion/models/hunyuan_image3/test_image_kv_cache_manager.py, "PR #7021"] --- # HunyuanImage3 开发规则 @@ -44,6 +44,8 @@ sources: [incidents/painterly/_index.md, hf-alignment-pitfalls.md, vllm_omni/dif | `layering` | shared serving 或模型 adapter 分层 | `HY3-1g`, `HY3-2f`, `HY3-6i` | | `image` | 图片数量、processor、VAE、条件图 | `HY3-2b`, `HY3-2c`, `HY3-2d`, `HY3-5a`, `HY3-5b`, `HY3-5c`, `HY3-6c`, `HY3-6d` | | `size-ratio` | size=auto、ratio、batch ratio | `HY3-4a`, `HY3-4b`, `HY3-4c`, `HY3-4d`, `HY3-6g` | +| `moe-loader` | `get_expert_mapping`、expert remapping、fused-MoE loader 对接 | `HY3-9a` | + | `randomness` | seed、RNG、复现 | `HY3-5d`, `HY3-5e`, `HY3-5f`, `HY3-6h` | | `alignment-residual` | prompt、token、stop 已验证后仍有真实输出差异 | `HY3-3d`, `HY3-5b`, `HY3-7a`, `HY3-7b`, `HY3-7c`, `HY3-7d`, `HY3-7e` | | `paged-kv` | Hunyuan scheduler-paged KV、Q/K/V layout、prefill/denoise span 或 mixed attention | `HY3-5h`, `HY3-5i` | @@ -161,3 +163,10 @@ sources: [incidents/painterly/_index.md, hf-alignment-pitfalls.md, vllm_omni/dif - 强制：model 仅提供 32Q/8KV Q/K/V layout 与每 sequence 的 `full_attn_spans`；runner 从 Scheduler snapshot 激活 rows。first prefill 写整个 allocated sequence，后续 denoise 只从 prefix offset 写 target；paged forward 前清除 model-owned dense prompt state。 - 禁止：model 分配 Scheduler blocks、激活 Worker runtime、复用 cross-request prefix，或在 paged path 接受 imported AR KV/negative CFG；不得把 request-mode 覆盖外推为 `denoise_step`、Ring 或 AllGather。 - 验收：覆盖 full-allocation prefill、prefix-offset target update、32Q/8KV、full-attention spans 与 dense prompt-state clear；正式证据限 GPU/NPU request execution，其他 execution/parallel contracts另测。^[PR #6563] + +## HY3-9a — `get_expert_mapping` 只返回上游 flat list，本地 remapping 必须分方法 + +- 触发：修改 HunyuanImage3 AR 或 DiT loader 的 `get_expert_mapping`、MoE expert params mapping、checkpoint `gate_proj`/`up_proj` remapping，或与 vLLM fused-MoE loader 的对接。 +- 强制：公开 `get_expert_mapping()` 必须返回 flat `list[tuple[str, str, int, str]]`（非 MoE 返回 `[]`），供上游 loader 消费；Hunyuan checkpoint 的 `gate_proj`/`up_proj` → `gate_and_up_proj` 分片 remapping 只能放在独立的 `_get_expert_weights_remapping()`，由 `load_weights` 显式取用。AR 与 diffusion transformer 两侧同一合同。 +- 禁止：让 `get_expert_mapping()` 再返回 `(mapping, remapping)` 二元组，或把 Omni-local remapping 塞进上游期望的 public API。 +- 验收：对 `HunyuanModel` 与 `HunyuanImage3Model` 断言 public mapping 为 flat list、local remapping 字典独立，并覆盖非 MoE 空返回。^[PR #7021] diff --git a/knowledge/repos/vllm-omni/models/minicpm-o-4-5/_index.md b/knowledge/repos/vllm-omni/models/minicpm-o-4-5/_index.md index 7ab2d062c..20ac364fc 100644 --- a/knowledge/repos/vllm-omni/models/minicpm-o-4-5/_index.md +++ b/knowledge/repos/vllm-omni/models/minicpm-o-4-5/_index.md @@ -51,7 +51,8 @@ Online serving CI 的 core suite 固定 async chunk，expansion 分开覆盖 syn fixture 不再覆盖 eager/KV cap 并固定 active-speech turn window；验证证据边界见 MCPMO-5b。 描述直达源码与模型专有门禁见 [rules](rules.md#direct-代码快速入口)；并发 duplex 的 ragged -Talker chunk、shipping profile 与 chat 隔离见 [native duplex 规则](rules-duplex.md)；Code2Wav +Talker chunk、full-attention 容量 rollover、shipping profile 与 chat 隔离见 [native duplex 规则](rules-duplex.md) +；Code2Wav state、batch cap 与 NPU residual limits 见 [Code2Wav 并发批处理规则](rules-code2wav-batching.md)。新模型语义验证见 [model validation](../../review/guides/model-validation.md)。 diff --git a/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules-duplex.md b/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules-duplex.md index c8ec0d7a3..261f292e9 100644 --- a/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules-duplex.md +++ b/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules-duplex.md @@ -1,10 +1,10 @@ --- title: "MiniCPM-o 4.5 native duplex 规则" created: 2026-09-04 -updated: 2026-09-06 +updated: 2026-09-08 type: rule tags: [vllm-omni, models, model-executor] -sources: ["PR #6318", "PR #6346", "PR #6404", "PR #6458", "PR #6619", "PR #6630", "PR #6678", "PR #6626", "PR #6767", "PR #6821", vllm_omni/deploy/minicpmo_4_5.yaml, vllm_omni/model_executor/models/minicpmo_4_5/minicpmo_4_5_omni_tts.py, vllm_omni/model_executor/stage_input_processors/minicpmo_4_5_omni.py, tests/model_executor/models/minicpmo_4_5/test_talker_batching.py, vllm_omni/deploy/minicpmo_4_5_2gpu.yaml, vllm_omni/deploy/minicpmo_4_5_3gpu.yaml, vllm_omni/deploy/minicpmo_4_5_8x4090.yaml, examples/online_serving/minicpmo/realtime_duplex_demo.py, vllm_omni/experimental/fullduplex/client.py, vllm_omni/model_executor/models/minicpmo_4_5/duplex/adapter.py, vllm_omni/model_executor/models/minicpmo_4_5/duplex/session.py, vllm_omni/model_executor/models/minicpmo_4_5/duplex/stage0.py, vllm_omni/entrypoints/duplex/realtime_input.py, vllm_omni/entrypoints/duplex/runtime_adapter.py, vllm_omni/entrypoints/duplex/runtime_bridge.py, vllm_omni/entrypoints/duplex/serving.py, vllm_omni/entrypoints/duplex/session_runner.py, vllm_omni/experimental/fullduplex/video_stacking.py, tests/config/test_config_factory.py, tests/engine/duplex/test_duplex_deploy_config.py, tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py, tests/entrypoints/openai_api/test_duplex_handler.py, tests/examples/test_minicpmo_realtime_duplex_simple_demo.py, "PR #6529", vllm_omni/entrypoints/duplex/protocol.py, vllm_omni/entrypoints/duplex/realtime_state.py, tests/entrypoints/openai/test_duplex_protocol.py] +sources: ["PR #6318", "PR #6346", "PR #6404", "PR #6458", "PR #6619", "PR #6630", "PR #6678", "PR #6626", "PR #6767", "PR #6821", vllm_omni/deploy/minicpmo_4_5.yaml, vllm_omni/model_executor/models/minicpmo_4_5/minicpmo_4_5_omni_tts.py, vllm_omni/model_executor/stage_input_processors/minicpmo_4_5_omni.py, tests/model_executor/models/minicpmo_4_5/test_talker_batching.py, vllm_omni/deploy/minicpmo_4_5_2gpu.yaml, vllm_omni/deploy/minicpmo_4_5_3gpu.yaml, vllm_omni/deploy/minicpmo_4_5_8x4090.yaml, examples/online_serving/minicpmo/realtime_duplex_demo.py, vllm_omni/experimental/fullduplex/client.py, vllm_omni/model_executor/models/minicpmo_4_5/duplex/adapter.py, vllm_omni/model_executor/models/minicpmo_4_5/duplex/session.py, vllm_omni/model_executor/models/minicpmo_4_5/duplex/stage0.py, vllm_omni/entrypoints/duplex/realtime_input.py, vllm_omni/entrypoints/duplex/runtime_adapter.py, vllm_omni/entrypoints/duplex/runtime_bridge.py, vllm_omni/entrypoints/duplex/serving.py, vllm_omni/entrypoints/duplex/session_runner.py, vllm_omni/experimental/fullduplex/video_stacking.py, tests/config/test_config_factory.py, tests/engine/duplex/test_duplex_deploy_config.py, tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py, tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py, tests/entrypoints/openai_api/test_duplex_handler.py, tests/examples/test_minicpmo_realtime_duplex_simple_demo.py, "PR #6529", vllm_omni/entrypoints/duplex/protocol.py, vllm_omni/entrypoints/duplex/realtime_state.py, tests/entrypoints/openai/test_duplex_protocol.py, "PR #6799"] confidence: high --- @@ -106,3 +106,10 @@ confidence: high 把此 ordering fix 外推为通用 realtime/reliability guarantee。 - 验收：zero-audio reserve、partial ACK→commit→response-end→later ACK 原位扩展、unreserved stale reject、 delete cleanup、两种 truncate cap 与 checkpoint-before-commit 顺序均须覆盖。^[PR #6529] ^[PR #6821] + +## MCPMO-4i — full_attention Talker 必须保留上下文直至容量溢出才 rollover + +- 触发：修改 MiniCPM-o Talker `attention_type`、Stage 1 streaming session update、`meta.next_stage_prompt_len`、condition sequence metadata，或 shipping `minicpmo_4_5.yaml` 的默认 attention 策略。 +- 强制：默认遵循 checkpoint 声明的 `full_attention`；在该模式下，只要「下一 condition + codec-generation reserve」仍落在由 runtime `max_model_len` 与 `tts_config.max_position_embeddings` 导出的有效上限内，就必须保留已累积的 Talker KV/history 并追加；仅在严格溢出时才重建受支持的 one-previous-condition 窗口，释放旧 KV/encoder 与 connector watermark，重置异步 replacement state，并让请求重新进入 admission。精确贴合上限时允许 append，之后继续累积直至下次溢出。显式 `sliding_recompute` 仍在每个 condition 边界重算；正整数 `meta.next_stage_prompt_len` 是 scheduler 的权威 placeholder 长度，合法 Talker condition 即使缺少 legacy `ids.prompt` 也必须先 append 真实 prompt 再推进 sequence/lifecycle。 +- 禁止：把无条件 per-condition recompute 当作生产默认；硬编码 `4096` 当作有效上限；在未溢出时丢弃可保留的 Talker history；或把本路径写成已支持物理 KV preemption 后的增量 full-attention replay。 +- 验收：覆盖 fit/exact-fit/overflow、producer-owned 与 receiver-owned sequencing 兼容、`next_stage_prompt_len` 驱动的 append，以及显式 `sliding_recompute` 仍按边界重算；不得用 capacity-triggered rollover 的通过外推为已修复无关 perf 回归。^[PR #6799] diff --git a/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules.md b/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules.md index 4ae05fa88..ef2aa0c99 100644 --- a/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules.md +++ b/knowledge/repos/vllm-omni/models/minicpm-o-4-5/rules.md @@ -24,7 +24,8 @@ confidence: high | TTS/Code2Wav/TRT、CUDA Graph | `MCPMO-1b`–`MCPMO-1e`；[CUDA graphs](rules-cuda-graphs.md) | | batch、runtime info、reference-audio handoff | `MCPMO-3a`–`MCPMO-3e`；[Code2Wav batching](rules-code2wav-batching.md) | -| native duplex、sampling budget、shipping profile、playback ACK | `MCPMO-4a`–`MCPMO-4h`；[duplex](rules-duplex.md) | +| native duplex、sampling budget、shipping profile、playback ACK、Talker full-attention rollover | `MCPMO-4a`–`MCPMO-4i`；[duplex](rules-duplex.md) | + | accuracy/performance/online-serving evidence | `MCPMO-5a`、`MCPMO-5b` | ## 完整代码路由 diff --git a/knowledge/repos/vllm-omni/models/minimax-h3/_index.md b/knowledge/repos/vllm-omni/models/minimax-h3/_index.md index 07217ddb4..b654a7d81 100644 --- a/knowledge/repos/vllm-omni/models/minimax-h3/_index.md +++ b/knowledge/repos/vllm-omni/models/minimax-h3/_index.md @@ -154,9 +154,11 @@ execution，不能代替 retained accuracy gate、Buildkite H100 result 或其 ## 审查入口 H3 input matrix/media ingress，以及 text-encoder completeness、online FP8 的 component namespace、loader 顺序、joint quality 与 offload 边界见 -[media rules](rules-media.md) 与 [MiniMax H3 rules](rules.md#direct-代码快速入口)；checkpoint transform、quantized loader 与 +[media rules](rules-media.md)（含 `text_conditioning/v1` schema）与 [MiniMax H3 rules](rules.md#direct-代码快速入口) +；checkpoint transform、quantized loader 与 text-encoder fused-source 完整性正文见 [loading rules](rules-loading.md)；DLO、consumer/H100/ROCm -部署和硬件证据正文见 [deployment rules](rules-deployment.md)；conditioned VAE 确定性、modular task 选择与 request 级 Cache-DiT/TeaCache/sigma schedule/Turbo LoRA 生命周期见 [缓存与任务生命周期规则](rules-cache-task.md)。 +部署和硬件证据正文见 [deployment rules](rules-deployment.md)；conditioned VAE 确定性、modular task 选择与 request 级 Cache-DiT/TeaCache/sigma schedule/Turbo LoRA 生命周期、alias metadata 同构与 Turbo artifact 合同见 [缓存与任务生命周期规则](rules-cache-task.md)。 + H3 VAE decoder 的 model-local eager dispatch、remote-code eligibility、exactness guards、selective FP16 materialization、keyframe encode 的 cuDNN/TF32 pin 与 compile/spatial-parallel fallback 见 [VAE eager-ops rules](rules-vae-ops.md) diff --git a/knowledge/repos/vllm-omni/models/minimax-h3/rules-cache-task.md b/knowledge/repos/vllm-omni/models/minimax-h3/rules-cache-task.md index 7675c4e68..969a0c607 100644 --- a/knowledge/repos/vllm-omni/models/minimax-h3/rules-cache-task.md +++ b/knowledge/repos/vllm-omni/models/minimax-h3/rules-cache-task.md @@ -1,10 +1,10 @@ --- title: "MiniMax H3 缓存与任务生命周期规则" created: 2026-09-04 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, models, diffusion] -sources: ["PR #5703", "PR #5720", "PR #5810", "PR #5837", "PR #5840", "PR #5853", "PR #5991", "PR #6476", "PR #6550", "PR #6666", "PR #6714", "PR #6909", vllm_omni/diffusion/models/minimax_h3/batched_packing.py, vllm_omni/diffusion/models/minimax_h3/fasth3.py, vllm_omni/diffusion/models/minimax_h3/lora.py, vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py, vllm_omni/diffusion/models/minimax_h3/npu/lora.py, vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py, vllm_omni/diffusion/models/minimax_h3/quality_policy.py, vllm_omni/diffusion/models/minimax_h3/vae.py, vllm_omni/diffusion/cache/cachedit/runtime.py, vllm_omni/diffusion/sched/sigma_schedule.py, vllm_omni/diffusion/worker/diffusion_worker.py, vllm_omni/entrypoints/openai/video_api_utils.py, tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py, tests/diffusion/models/minimax_h3/test_minimax_h3_fasth3.py, tests/diffusion/models/minimax_h3/test_minimax_h3_lora.py, tests/diffusion/models/minimax_h3/test_minimax_h3_native_lora.py, tests/diffusion/models/minimax_h3/test_minimax_h3_parallel.py, tests/diffusion/models/minimax_h3/test_minimax_h3_step_execution.py, tests/entrypoints/openai_api/test_video_api_utils.py] +sources: ["PR #5703", "PR #5720", "PR #5810", "PR #5837", "PR #5840", "PR #5853", "PR #5991", "PR #6476", "PR #6550", "PR #6666", "PR #6714", "PR #6909", vllm_omni/diffusion/models/minimax_h3/batched_packing.py, vllm_omni/diffusion/models/minimax_h3/fasth3.py, vllm_omni/diffusion/models/minimax_h3/lora.py, vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py, vllm_omni/diffusion/models/minimax_h3/npu/lora.py, vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py, vllm_omni/diffusion/models/minimax_h3/quality_policy.py, vllm_omni/diffusion/models/minimax_h3/vae.py, vllm_omni/diffusion/cache/cachedit/runtime.py, vllm_omni/diffusion/sched/sigma_schedule.py, vllm_omni/diffusion/worker/diffusion_worker.py, vllm_omni/entrypoints/openai/video_api_utils.py, tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py, tests/diffusion/models/minimax_h3/test_minimax_h3_fasth3.py, tests/diffusion/models/minimax_h3/test_minimax_h3_lora.py, tests/diffusion/models/minimax_h3/test_minimax_h3_native_lora.py, tests/diffusion/models/minimax_h3/test_minimax_h3_parallel.py, tests/diffusion/models/minimax_h3/test_minimax_h3_step_execution.py, tests/entrypoints/openai_api/test_video_api_utils.py, "PR #7162", "PR #7062"] confidence: high --- @@ -186,11 +186,15 @@ Cache-DiT、TeaCache、distilled sigma schedule 与 Turbo LoRA 的生命周期 ## MMH3-2j — Turbo LoRA 只接受精确发布 artifact，并与 H3 task、sampling 和 offload lifecycle 共同门禁 - 触发：MiniMax-H3 LightX2V Turbo 的 dynamic LoRA loader、artifact conversion、request task/steps/flow shift，或与 CPU、standard layerwise、distributed layerwise offload (DLO) 的组合。 -- 强制：仅识别声明文件名的 LightX2V Turbo v1.0 safetensors，且 metadata 必须为 `key_format=minimax-h3-diffusers`、rank/alpha `128/128`；任何未知 tensor、重复/不完整 A/B pair、unsupported target 或不精确的 global A/B shape 都在 binding 前失败。转换必须把 Diffusers token-refiner/main-block names 映射为 native H3 names，恢复 FC1 `[gate; up]` 行序为 model-owned packed weights，并通过 `stacked_params_mapping` 将独立 Q/K/V 接到 packed QKV。 -- 强制：Turbo 仅在非零 scale 的实际 active recognized adapter 时限制请求；只支持 FL2VA/T2VA，要求五个 sigma points（四次 denoiser evaluation）、video `flow_shift=6` 与 audio `audio_flow_shift=3`。每次真实 load 都替换该 client ID 的 Turbo classification，避免 eviction 后同 ID generic adapter 被误分类。 +- 强制：只识别文件名可解析为 `TurboSpec` 的 LightX2V Turbo safetensors（artifact 矩阵、alpha 解析与目录选择由 MMH3-2p 定义：不再要求单一 v1.0 文件名或 `alpha==128`），metadata 必须为 `key_format=minimax-h3-diffusers`、rank `128`；任何未知 tensor +、重复/不完整 A/B pair、unsupported target 或不精确的 global A/B shape 都在 binding 前失败。转换必须把 Diffusers token-refiner/main-block names 映射为 native H3 names，恢复 FC1 `[gate; up]` 行序为 model-owned packed weights，并通过 `stacked_params_mapping` 将独立 Q/K/V 接到 packed QKV。 +- 强制：Turbo 仅在非零 scale 的实际 active recognized adapter 时限制请求；task、sigma points 与 video/audio flow shift 按**实际加载** artifact 的 `TurboSpec` 校验（`fl2v` 服务 FL2VA/T2VA，`ref2v` 只在 Ref2VA-only server 服务 Ref2VA，见 MMH3-2p；PR #6550 的 v1.0 4-step/768p artifact 对应五个 sigma points、`flow_shift=6`、`audio_flow_shift=3`）。每次真实 load + 都替换该 client ID 的 Turbo classification，避免 eviction 后同 ID generic adapter 被误分类。 - 强制：DLO + active Turbo 可用时，DLO 只 stream base blocks；manager 注入的 request-switchable A/B sidecars 必须留在 compute device，且任何因更大 LoRA rank 触发的 wrapper reallocation 后仍重放该 placement。固定 sidecar HBM 是此组合的预算，不得把它算作 base-block DLO lifecycle 或 collective payload。 -- 禁止：将 exact Turbo artifact 的损坏 metadata 静默交给 generic fallback；将任意 H3 PEFT checkpoint、prefusion、multi-LoRA composition 或 Ref2VA 宣称为此功能支持；在 model-level CPU 或 standard layerwise offload 下激活 Turbo。不得为动态 sidecar 重建 DLO host shards、随每个 block/denoise step stream A/B，或让 LoRA 进入 DLO AllGather；没有 active LoRA 的 DLO 不得进入此 loader path。现有 HWR eligibility 仍拒绝 `lora_path`，AllGather 也不走该 BF16 HWR path，不能将 DLO+Turbo evidence 扩展为 HWR support。 -- 验收：覆盖 file/metadata/alpha/target/pair/shape rejection、QKV and FC1 packing、scale-zero/generic same-ID/Ref2VA lifecycle、steps/shifts errors、CPU/standard-layerwise rejection，以及 DLO admission、resident A/B device placement 和 rank-driven reallocation 后的 placement；真实 artifact evidence 必须另绑定 exact checkpoint、task、topology、sampling 和 video/audio output。DP validation 至少以同 wave 的并发 request 覆盖所声称 DP topology；DLO no-AllGather/AllGather evidence 只证明对应 base-weight transport，sidecars 不参与 collective。PR #6550 报告的 0.911 GiB/TP2-rank 与 1.355 GiB/TP1-rank 只绑定 rank-128、312-target 的该 Turbo artifact，不是任意 adapter 的容量保证。转换后的 native mixed-rank PEFT artifact 不属于本 PR 的 supported contract；其 generic fallback rank/scale mismatch 是 post-merge follow-up，不可用本页的 Turbo success 证明安全。^[PR #6476] ^[PR #6550] +- 禁止：将 exact Turbo artifact 的损坏 metadata 静默交给 generic fallback；将任意 H3 PEFT checkpoint、prefusion、multi-LoRA composition，或 combined/`transformers_ref` 拓扑下的 Ref2VA 宣称为此功能支持（Ref2VA-only server 上的 `ref2v` artifact 准入见 MMH3-2p）； +在 model-level CPU 或 standard layerwise offload 下激活 Turbo。不得为动态 sidecar 重建 DLO host shards、随每个 block/denoise step stream A/B，或让 LoRA 进入 DLO AllGather；没有 active LoRA 的 DLO 不得进入此 loader path。现有 HWR eligibility 仍拒绝 `lora_path`，AllGather 也不走该 BF16 HWR path，不能将 DLO+Turbo evidence 扩展为 HWR support。 +- 验收：覆盖 file/metadata/alpha/target/pair/shape rejection、QKV and FC1 packing、scale-zero/generic same-ID/Ref2VA lifecycle、steps/shifts errors、CPU/standard-layerwise rejection，以及 DLO admission、resident A/B device placement 和 rank-driven reallocation 后的 placement；真实 artifact evidence 必须另绑定 exact checkpoint、task、topology、sampling 和 video/audio output。DP validation 至少以同 wave 的并发 request 覆盖所声称 DP topology；DLO no-AllGather/AllGather evidence 只证明对应 base-weight transport，sidecars 不参与 collective。PR #6550 报告的 0.911 GiB/TP2-rank 与 1.355 GiB/TP1-rank 只绑定 rank-128、312-target 的该 Turbo artifact，不是任意 adapter 的容量保证。转换后的 native mixed-rank PEFT artifact 不属于本 PR 的 supported contract；其 generic fallback rank/scale mismatch 是 post-merge follow-up，不可用本页的 Turbo success 证明安全。^[PR #6476] ^[PR #6550] ^[PR #7062] + ## MMH3-2k — H3 step execution 必须保持请求状态、attention 文档和 rank-0 prepare 隔离 @@ -212,3 +216,18 @@ Cache-DiT、TeaCache、distilled sigma schedule 与 Turbo LoRA 的生命周期 [Diffusion rules](../../components/diffusion/rules.md)。 部署与容量证据见 [部署与证据规则](rules-deployment.md)；共享 diffusion 缓存机制见 [diffusion 规则](../../components/diffusion/rules.md)。 + +## MMH3-2o — modular alias metadata 必须与 canonical H3 implementation 全字段同构 + +- 触发：新增或修改 MiniMax-H3 的 alias class、`model_index.json` root routing、`DiffusionModelMetadata` 能力字段，或 recipe 依赖 repo-id 与 partition path 等价。 +- 强制：凡是解析到同一 `MiniMaxH3Pipeline` 实现的 alias，都必须与 canonical entry 保持所有 runtime-selected capability 字段同构，不只 admission 上限，还包括 backend default gate 会读取的字段（如 `attention_mask_free`）以及 packed-sequence / mixed-reference 相关能力。repo root index 若解析到 alias，则 reviewer/debugger 必须先按 alias metadata 检查默认 backend 与 request contract，而不能假设它自动继承 canonical row。 +- 禁止：只复制 review 点名的单个字段；把“同一 Python pipeline class”当成 metadata 自动等价；或用 partition-specific smoke 证明 repo-root serving 行为。 +- 验收：对 root repo id、partition subdir 和 modular alias 分别断言 metadata parity、默认 attention backend resolution 与 supported task/admission contract 一致；任一字段 drift 都应在 contract test 中直接暴露。^[PR #7162] + +## MMH3-2p — Turbo filename 编码 sampler 合同，目录不得静默挑文件 + +- 触发：修改 MiniMax-H3 LightX2V Turbo LoRA loader、artifact 选择、`TurboSpec`、request task/steps/`flow_shift`/`audio_flow_shift`、alpha 解析，或 Ref2VA/combined topology 下的 Turbo 准入。 +- 强制：只服务 Diffusers-PEFT 布局；从文件名解析 `TurboSpec`（`fl2v`/`ref2v`、denoise steps、768p 与否决定 video shift 6.0/12.0、audio shift 3.0、rank 128）。`num_inference_steps`、`flow_shift`、`audio_flow_shift` 与 `_resolve_task` 必须按**实际加载**的 artifact 校验：`fl2v` 只服务 `t2va`/`fl2va`，`ref2v` 只服务 `ref2va`；`sigma_points = denoise_steps + 1`。alpha 以 safetensors metadata 为准；缺省时用 LightX2V 参考默认 `8` 并 warning，不得硬钉 `128`。 +- 强制：`--lora-path` 指向单文件，或只含一个可解析 Turbo artifact 的目录；多候选必须 fail closed。`ref2v` 只能挂在 Ref2VA-only server；combined/`transformers_ref` 拓扑不得绑定只注入 `transformer` 的 Turbo adapter。ComfyUI 融合 QKV/SwiGLU 导出按名拒绝。 +- 禁止：继续要求单一 `4step_v1.0_768p` 文件名或 `alpha==128`；在目录里静默挑选；把未声明 alpha 当 rank；或让 Ref2VA 请求跑到未蒸馏 DiT 的 few-step schedule。 +- 验收：覆盖多 artifact 文件名矩阵、缺/错 alpha、多文件目录拒绝、task/steps/shift 负向门禁、Ref2VA-only vs combined 拒绝，以及 Diffusers-layout packing；不得把单一 artifact 通过外推为全矩阵质量证据。^[PR #7062] diff --git a/knowledge/repos/vllm-omni/models/minimax-h3/rules-media.md b/knowledge/repos/vllm-omni/models/minimax-h3/rules-media.md index 960b56c7d..ee85c712f 100644 --- a/knowledge/repos/vllm-omni/models/minimax-h3/rules-media.md +++ b/knowledge/repos/vllm-omni/models/minimax-h3/rules-media.md @@ -1,10 +1,10 @@ --- title: "MiniMax H3 媒体输入与精度规则" created: 2026-09-02 -updated: 2026-09-05 +updated: 2026-09-08 type: rule tags: [vllm-omni, models, diffusion] -sources: ["PR #5752", "PR #5829", "PR #5885", "PR #5978", "PR #6555", "PR #6688", .buildkite/cuda/test-nightly.yml, .buildkite/cuda/test-ready.yml, vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py, vllm_omni/model_executor/models/minimax_h3/reference_video.py, vllm_omni/model_executor/stage_input_processors/minimax_h3.py, vllm_omni/engine/stage_runtime.py, vllm_omni/entrypoints/openai/api_server.py, vllm_omni/entrypoints/openai/serving_video.py, vllm_omni/entrypoints/openai/video_api_utils.py, vllm_omni/inputs/data.py, tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py, tests/engine/test_async_omni_engine_stage_init.py, tests/e2e/accuracy/minimax_h3/test_minimax_h3_i2va_ref2va_similarity.py, tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py, tests/entrypoints/openai_api/test_video_server.py, tests/entrypoints/openai_api/test_video_api_utils.py, "PR #6064", "PR #6813", "PR #6824"] +sources: ["PR #5752", "PR #5829", "PR #5885", "PR #5978", "PR #6555", "PR #6688", .buildkite/cuda/test-nightly.yml, .buildkite/cuda/test-ready.yml, vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py, vllm_omni/model_executor/models/minimax_h3/reference_video.py, vllm_omni/model_executor/stage_input_processors/minimax_h3.py, vllm_omni/engine/stage_runtime.py, vllm_omni/entrypoints/openai/api_server.py, vllm_omni/entrypoints/openai/serving_video.py, vllm_omni/entrypoints/openai/video_api_utils.py, vllm_omni/inputs/data.py, tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py, tests/engine/test_async_omni_engine_stage_init.py, tests/e2e/accuracy/minimax_h3/test_minimax_h3_i2va_ref2va_similarity.py, tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py, tests/entrypoints/openai_api/test_video_server.py, tests/entrypoints/openai_api/test_video_api_utils.py, "PR #6064", "PR #6813", "PR #6824", "PR #6720"] confidence: high --- @@ -133,3 +133,10 @@ confidence: high hits; chunked/prefix-cache token tags; malformed bridge payload/request mismatch; artifact reuse and cleanup; resolver/local partition paths; Stage 0-only TP alias precedence/rejection; encoder-free DLO; explicit inline isolation; and regular/Turbo default sampling profiles. ^[PR #5885] + +## MMH3-2q — `minimax_h3.text_conditioning/v1` 必须在 Stage 0→1 边界 fail closed + +- 触发：修改 MiniMax-H3 `MiniMaxH3TextConditioning`、`text_encoder2diffusion` bridge、`OmniPayload` handoff，或 Stage 0/1 对 hidden states / token roles 的消费。 +- 强制：公开 schema id 为 `minimax_h3.text_conditioning/v1`。语义载荷：`hidden_states` 为 contiguous strided `[tokens, 5120]` `bfloat16`；`token_tags` 为 contiguous strided `[tokens]` `int64` 且仅含 `0/1`。Stage-wire `OmniPayload` 路径必须经 `from_omni_payload`：从 `hidden_states.output` 取 tensor，从 `meta.token_role_ids` 取 `[tokens, 1]` `int64` contiguous strided，再 squeeze 成语义 tags；任一类型/shape/dtype/layout 不符都在 adapter 边界失败。 +- 禁止：放宽为任意 float/int dtype 或非 contiguous layout；绕过 schema validator 直接塞 positional tensors；把该 H3-only hardening 写成已落地的通用 `StagePortSpec`。 +- 验收：CPU 测试覆盖合法 payload、错误 dtype/layout/shape、缺失字段，以及 encoder ownership（仅 owning replica 持有，disaggregated diffusion stage 缺席）；不改动既有 serializer 与 launch path 时仍须保持负向门禁。^[PR #6720] diff --git a/knowledge/repos/vllm-omni/models/minimax-h3/rules.md b/knowledge/repos/vllm-omni/models/minimax-h3/rules.md index b8e67f7eb..a9817ddef 100644 --- a/knowledge/repos/vllm-omni/models/minimax-h3/rules.md +++ b/knowledge/repos/vllm-omni/models/minimax-h3/rules.md @@ -18,8 +18,9 @@ confidence: high |---|---| | component quantization、weight loading、HWR | `MMH3-1a`、`MMH3-1f`；[loading](rules-loading.md) | | attention/RoPE/backend、encoder process state | `MMH3-1c`–`MMH3-1e`、`MMH3-1g`、`MMH3-1k`、`MMH3-1n` | -| media ingress、reference tasks、split text encoder | `MMH3-2a`、`MMH3-2b`、`MMH3-2l`；[media](rules-media.md) | -| modular task、cache policy、schedules、LoRA/FastH3 | `MMH3-2c`–`MMH3-2n`；[cache/task](rules-cache-task.md) | +| media ingress、reference tasks、split text encoder、text-conditioning schema | `MMH3-2a`、`MMH3-2b`、`MMH3-2l`、`MMH3-2q`；[media](rules-media.md) | +| modular task、cache policy、schedules、LoRA/FastH3、alias metadata、Turbo artifact | `MMH3-2c`–`MMH3-2p`；[cache/task](rules-cache-task.md) | + | DLO、hardware recipes、CI/perf evidence | `MMH3-3a`–`MMH3-3m`；[deployment](rules-deployment.md) | | VAE kernels、materialization、keyframe encode determinism、fallback | `MMH3-4c`、`MMH3-4d`；[VAE ops](rules-vae-ops.md) | | NPU text-MLP packed SwiGLU、quant dispatch、DiT activation | `MMH3-4a`、`MMH3-4b` | diff --git a/knowledge/repos/vllm-omni/models/qwen-image/_index.md b/knowledge/repos/vllm-omni/models/qwen-image/_index.md index d8a7bbb00..3f6b3d442 100644 --- a/knowledge/repos/vllm-omni/models/qwen-image/_index.md +++ b/knowledge/repos/vllm-omni/models/qwen-image/_index.md @@ -48,5 +48,6 @@ sources: ["PR #2783", vllm_omni/diffusion/lora/loader.py, vllm_omni/diffusion/mo - 审查 qwen_image 任一变体的 CFG-parallel、分布式 VAE 或 DMD2 改动——五变体 共栈,单变体改动先扫其余四个;共享 mixin 见 [Diffusion 组件](../../components/diffusion/_index.md)。 -- 审查 accuracy、regional compile 或 FlashAttention 确定性时见 +- 审查 accuracy、regional compile、FlashAttention 确定性或 Edit `txt_seq_lens` 时见 + [Qwen-Image 规则](rules.md)。 diff --git a/knowledge/repos/vllm-omni/models/qwen-image/rules.md b/knowledge/repos/vllm-omni/models/qwen-image/rules.md index 127e44867..de22ce215 100644 --- a/knowledge/repos/vllm-omni/models/qwen-image/rules.md +++ b/knowledge/repos/vllm-omni/models/qwen-image/rules.md @@ -1,10 +1,10 @@ --- title: "Qwen-Image 实现规则" created: 2026-09-02 -updated: 2026-09-04 +updated: 2026-09-08 type: rule tags: [vllm-omni, models, diffusion] -sources: ["PR #5887", tests/e2e/accuracy/test_qwen_image.py, "PR #6110", "vllm_omni/diffusion/models/qwen_image/qwen_image_transformer.py"] +sources: ["PR #5887", tests/e2e/accuracy/test_qwen_image.py, "PR #6110", "vllm_omni/diffusion/models/qwen_image/qwen_image_transformer.py", "PR #5586"] confidence: high --- @@ -16,6 +16,8 @@ confidence: high |---|---| | accuracy/repeatability gate | `QWENIMG-1a` | | compiled complex RoPE | `QWENIMG-1b` | +| Edit `txt_seq_lens`/RoPE width、negative CFG length | `QWENIMG-1c` | + ## QWENIMG-1a — accuracy gate 必须先证明同 seed repeatability @@ -34,3 +36,11 @@ confidence: high - 强制：对 `vid_freqs` 和 `txt_freqs` 使用 `torch.real()`/`torch.imag()` 后再转换为 query 对应 dtype；即使问题由 MUSA 暴露，也保持 CUDA 与其他平台使用同一等价函数式表达式。 - 禁止：在该编译路径恢复 `freqs.real`/`freqs.imag` 属性链，或仅用 `--enforce-eager` 掩盖重复 tensor alias guard；不得把该模型级修复泛化为所有 diffusion RoPE caller 已具备同等保障。 - 验收：以 legacy 属性表达式作为 reference，验证函数式路径的值与 dtype 精确一致；在目标 MUSA 上以 Qwen Image 默认编译配置验证不再出现 `Duplicate tensors found`，并在 CUDA smoke 中确认输出 parity。^[PR #6110] + +## QWENIMG-1c — edit path 的 `txt_seq_lens` 必须来自 padded embed width，不得来自 mask token count + +- 触发：修改 Qwen-Image Edit pipeline 的 prompt encode、RoPE table length、negative prompt wiring，或 request-batch padding。 +- 强制：传给 RoPE / `diffuse()` 的 `txt_seq_lens` 与 `negative_txt_seq_lens` 必须由实际 prompt embed 的 padded width 推导，而不是 `prompt_embeds_mask.sum()` 的 valid-token 数。mask 只表达语义 token 有效区，不能缩短需要为 padded encoder width 构造的 text frequency table。 +- 强制：有 negative prompt 且启用 true CFG 时，正负两侧都使用各自 embed width；无 negative prompt 时 negative length 必须是 `None`，不能伪造一个来自正分支 mask 的长度。 +- 禁止：在 Edit path 回退到 `mask.sum()`，即使别的 Qwen-Image pipeline 仍正确；或仅以 helper test 证明 RoPE table 正确而不锁定实际 `forward()` call site。 +- 验收：以 padded embeds 但较短 valid-token mask 的 request 覆盖 Edit `forward()`，精确断言传入 `diffuse()` 的长度等于 padded width；同时覆盖有/无 negative prompt 的 CFG 分支，防止单侧回归。^[PR #5586] From 96eae2390a64468ade5e649c21fbc3507870515f Mon Sep 17 00:00:00 2001 From: tzhouam Date: Thu, 10 Sep 2026 15:48:34 +0800 Subject: [PATCH 2/2] knowledge: keep the configuration quick map deliverable The extra Direct-table row for VOMNI-CFG-1s pushed the configuration owner's quick map past the size Direct can deliver inline, so every review routed there paid a file read (test_direct_routes_title_body_before_changed_files: knowledge_file_reads 3 != 2). Fold the ID into the existing `strict-normalization` cell instead; the rule itself is unchanged. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_018MAFmwP3u5qAMDgHfM6LFi --- knowledge/repos/vllm-omni/components/configuration/rules.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/knowledge/repos/vllm-omni/components/configuration/rules.md b/knowledge/repos/vllm-omni/components/configuration/rules.md index c7aac4265..9555efbf2 100644 --- a/knowledge/repos/vllm-omni/components/configuration/rules.md +++ b/knowledge/repos/vllm-omni/components/configuration/rules.md @@ -20,7 +20,7 @@ sources: ["claude-workflow-starter-private@296ea45", "PR #4281", "PR #5031", "PR | PR 描述在做什么 | 精确规则组 | 第一批 live 源码 | |---|---|---| -| strict schema、unknown field、alias、flat→nested、structured/legacy/direct parity、typed projection | `strict-normalization`：`VOMNI-CFG-1a`–`1h` | `vllm_omni/config/stage_config.py::{build_stage_runtime_overrides,strip_parent_engine_args}` → `vllm_omni/config/omni_config.py::{_build_diffusion_config_projection,VllmOmniConfig.from_pipeline_config}` → `vllm_omni/engine/stage_init_utils.py::{build_engine_args_dict,build_engine_args_dict_from_omni_stage_config}` | +| strict schema、unknown field、alias、flat→nested、structured/legacy/direct parity、typed projection | `strict-normalization`：`VOMNI-CFG-1a`–`1h`、`1s` | `vllm_omni/config/stage_config.py::{build_stage_runtime_overrides,strip_parent_engine_args}` → `vllm_omni/config/omni_config.py::{_build_diffusion_config_projection,VllmOmniConfig.from_pipeline_config}` → `vllm_omni/engine/stage_init_utils.py::{build_engine_args_dict,build_engine_args_dict_from_omni_stage_config}` | | deploy YAML、`base_config`、pipeline/stage overlay、pipeline-owned alias/hook、headless/offline parity、最终逐 stage config | `deploy-topology`：`CONF-3a`, `CONF-4b`, `CONF-5a`, [CONF-5i](rules-topology-profiles.md#conf-5i-pipeline-专属-alias-与-hook-必须在通用配置层声明归一并消费), [CONF-5j](rules-topology-profiles.md#conf-5j-pipelineconfig-在构造时必须验证终端-topology) | `vllm_omni/config/stage_config.py::{resolve_deploy_yaml,load_deploy_config,normalize_pipeline_cli_overrides,merge_pipeline_deploy,build_stage_runtime_overrides,_build_engine_args}` → `vllm_omni/config/config_factory.py::StageConfigFactory.create_from_model` → `vllm_omni/engine/stage_init_utils.py::_resolve_model_path` | | composable strategy、axis、routing、load balancing、`strategy-config` | `composable-strategy`：`CONF-4a` | `vllm_omni/config/composable_parallel/strategy_loader.py::{parse_strategy_specs,load_strategy_specs}` → `translator.py::translate_strategy_stack` → `apply.py::apply_strategy_specs` → `config_factory.py::{StageConfigFactory._apply_strategy_specs,StageConfigFactory._reconcile_strategy_with_cli}` | | `gpu_memory_utilization`、`kv_cache_memory_bytes`、多 stage 共卡、小显存 OOM | `deploy-memory`：`CONF-1a`, `CONF-2a` | `vllm_omni/config/stage_config.py::{build_stage_runtime_overrides,_build_engine_args}` → `vllm_omni/config/omni_config.py::{_build_runtime_config,_build_parallel_config,VllmOmniConfig.from_pipeline_config}` | @@ -28,7 +28,7 @@ sources: ["claude-workflow-starter-private@296ea45", "PR #4281", "PR #5031", "PR | pipeline `sampling_constraints`、`StageConfig.to_omegaconf()`、runtime stage config | `stage-config-propagation`：`VOMNI-CFG-1o` | `merge_pipeline_deploy` → `StageConfig.to_omegaconf()` → `engine.stage_configs` | | stage transport、`requires_full_payload_input`、topology projection 或 override rejection | `stage-transport`：`VOMNI-CFG-1p` | `stage_config.py::{StagePipelineConfig,_build_engine_args}` → `omni_config.py::_build_model_config` → `engine/arg_utils.py::OmniEngineArgs` | | SymmMem Ulysses transport | [VOMNI-CFG-1q](rules-diffusion-parallel-transport.md) + `DIFF-4x` | deploy → parallel config → Ulysses | -| global stage engine args 的 owner 校验、`diffusion_offload_config` 跨进程 mapping、compile/offload gate | `strict-normalization`：`VOMNI-CFG-1s` | `stage_config.py` 规范化 → ownership 校验 → 按 stage `execution_type` 投影 → offload parser | + | HF cache snapshot path、空 `config.json`、name-based pipeline fallback | [`model-reference-routing`](rules-model-reference-routing.md#conf-7a-模型引用解析必须物化对象存储配置并只从受控名称组件匹配)：`CONF-7a` | `config_factory.py::{_name_match_candidate,StageConfigFactory._try_infer_model_type}` → `pipeline_registry.py::OMNI_PIPELINES` → `tests/config/test_config_factory.py::TestNameMatchCandidateSnapshotPaths` |