Skip to content

[Klaud Cold] Add H200 Qwen3.8-27B-FP8 vLLM TP1 native MTP 1k1k / 新增 H200 Qwen3.8-27B-FP8 vLLM TP1 原生 MTP 1k1k 配方 - #3293

Open
functionstackx wants to merge 2 commits into
mainfrom
feat/qwen38-27b-fp8-h200-vllm-mtp
Open

functionstackx wants to merge 2 commits into
mainfrom
feat/qwen38-27b-fp8-h200-vllm-mtp

Conversation

@functionstackx

Copy link
Copy Markdown
Collaborator

Add qwen3.827b-fp8-h200-vllm-mtp: Qwen3.8-27B-FP8 (Qwen/Qwen3.8-27B-FP8, fp8 e4m3 weights with dynamic activation scaling) on a single H200 (141 GB), served by vLLM nightly-cd10ed6f9f6b37a8ace9cf380007e66fe12ec0c3 with TP1 and the checkpoint's native MTP head (mtp_num_hidden_layers=1, the mtp.* tensors ship in the repo). No external drafter and no --enforce-eager. Single-turn 1k1k only, concurrency 1-128. Sibling of the bf16 DSpark recipe in the qwen3.8-27b PRs; same qwen3.827b model-prefix, distinguished by precision.

Serve shape: --speculative-config '{"method":"mtp","num_speculative_tokens":3}' (the setting the vLLM recipe page gives for this model; it reports fp8 MTP acceptance 0.771), fp8 KV cache, --max-num-seqs sized to the sweep point (floor 16, 256 for accuracy evals: vLLM's default 1024 exceeded the GDN cache blocks next to the weights on H100 in run 35357364404), --no-enable-prefix-caching so every request prefills its full prompt, --language-model-only (text-only serving of Qwen3_5ForConditionalGeneration), --reasoning-parser qwen3, --tool-call-parser qwen3_xml, and the matrix context as --max-model-len. Prompts go through the chat template so MTP acceptance reflects real text rather than random tokens; acceptance is real. The MODELS.md / MODELS_zh.md rows for Qwen3.8-27B land with the bf16 H100 PR (#3260).

Validation: bash -n, YAML parse, exact-key matrix generation, and changelog append (no deletions) pass. GPU validation pending; full-sweep-fail-fast is set on this PR.

One PR per SKU.

中文

新增 qwen3.827b-fp8-h200-vllm-mtp:Qwen3.8-27B-FP8(Qwen/Qwen3.8-27B-FP8,fp8 e4m3 权重、动态激活缩放)在单张 H200(141 GB)上由 vLLM nightly-cd10ed6f9f6b37a8ace9cf380007e66fe12ec0c3 以 TP1 提供服务,并使用 checkpoint 自带的原生 MTP 头mtp_num_hidden_layers=1,仓库内含 mtp.* 张量)。无外部草稿模型,也不使用 --enforce-eager。仅单轮 1k1k,并发 1-128。与 qwen3.8-27b 各 PR 中的 bf16 DSpark 配方为姊妹配方;沿用 qwen3.827b model-prefix,以精度区分。

服务参数:--speculative-config '{"method":"mtp","num_speculative_tokens":3}'(vLLM recipe 页面为该模型给出的设置;其报告 fp8 MTP 接受率 0.771),fp8 KV cache,--max-num-seqs 按并发点设定(下限 16,精度评测为 256:运行 35357364404 中 vLLM 默认的 1024 超过 H100 上权重旁的 GDN cache block 数),--no-enable-prefix-caching 使每个请求完整 prefill 其提示词,--language-model-only(仅文本服务 Qwen3_5ForConditionalGeneration),--reasoning-parser qwen3--tool-call-parser qwen3_xml--max-model-len 使用矩阵上下文。提示词经 chat template 处理,使 MTP 接受率反映真实文本而非随机 token;接受率为真实值。 Qwen3.8-27B 的 MODELS.md / MODELS_zh.md 行随 bf16 H100 PR(#3260)提交。

验证:bash -n、YAML 解析、精确 key 矩阵生成、changelog 追加(无删除)均通过。GPU 验证待完成;本 PR 已添加 full-sweep-fail-fast

每个 SKU 一个 PR。

🤖 Generated with Claude Code

… 1k1k recipe on H200

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@functionstackx functionstackx added full-sweep-fail-fast qwen3.8-27b Qwen3.8-27B (bf16) vLLM TP1 DSpark 1k1k recipes and their eager variants labels Sep 19, 2026
@functionstackx
functionstackx force-pushed the feat/qwen38-27b-fp8-h200-vllm-mtp branch from 43f9940 to c51e82a Compare September 19, 2026 19:55
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast;仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This review covers commit 43f9940, which is no longer the latest commit on this pull request; later commits are not covered by it.

@github-actions

Copy link
Copy Markdown
Contributor

@functionstackx

Copy link
Copy Markdown
Collaborator Author

/reuse-sweep-run 35465800643

Sync with origin/main after the green sweep run 35465800643; the reuse gate
authorizes that run on this head.

在绿色 sweep 运行 35465800643 之后与 origin/main 同步;reuse gate 在此 head 上授权该运行。

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast qwen3.8-27b Qwen3.8-27B (bf16) vLLM TP1 DSpark 1k1k recipes and their eager variants

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant