[Klaud Cold] Add H100 Qwen3.8-27B bf16 vLLM TP1 DSpark 1k1k / 新增 H100 Qwen3.8-27B bf16 vLLM TP1 DSpark 1k1k 配方 - #3260
functionstackx wants to merge 3 commits into
Conversation
…k1k recipe on H100 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ea812c9 to
25721af
Compare
|
Thanks for the contribution!
中文感谢你的贡献！
|
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35359530151 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35359530151 |
…isable prefix caching Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline finding, I checked whether the changelog's pr-link: TBD would fail CI on this PR: validate_perf_changelog.py's CLI entrypoint (main -> validate_matrix_compatible_change) never calls compare_entries/validate_added_pr_link — only run-sweep.yml's "Validate perf-changelog matrix" step invokes that path, and it skips the pr-link check entirely. So the placeholder doesn't block this PR's CI; it would still need to become a real link (or the XXX convention) via prepare_perf_changelog_merge.py before merge.
Extended reasoning...
A CONFIRMED finding was already surfaced for this PR (the EVAL_ONLY branch in the new script never calls append_lm_eval_summary, unlike the sibling RUN_EVAL path), so per policy this run cannot approve and the inline comment already signals a human should look. I independently traced the pr-link: TBD candidate that was flagged as a possible CI-failure: reading infx/workflows/validate_perf_changelog.py end-to-end, main() only calls validate_matrix_compatible_change, which calls validate_generated_config — compare_entries/validate_added_pr_link (the function that rejects non-canonical, non-placeholder pr-links) is never reached from that entrypoint, and run-sweep.yml invokes exactly that entrypoint. The pr-link validation only runs later, at merge time, via utils/prepare_perf_changelog_merge.py (invoked by utils/merge_with_reuse.sh, not wired into any GitHub Actions workflow), where TBD (not in PR_LINK_PLACEHOLDERS = {"XXX", ".../pull/XXX"}) would need to be corrected before/at merge but does not fail this PR's own CI checks. This is worth recording since it contradicts a plausible-looking claim in the PR's own conventions summary that the placeholder "fails the repo's CI changelog validator."
| if [[ "${EVAL_ONLY:-false}" == true ]]; then | ||
| run_eval --framework lm-eval --port "$PORT" |
There was a problem hiding this comment.
🔴 EVAL_ONLY (golden-AL/accuracy) CI runs of this recipe will always fail after this merges, unlike every other fixed-seq-len *_mtp.sh script. The EVAL_ONLY branch calls run_eval --framework lm-eval --port "$PORT" but never calls append_lm_eval_summary, so results*.json/meta_env.json never get staged into the job's working directory from the lm-eval temp dir. Every sibling fixed-seq-len script (e.g. dsr1_fp8_b300.sh, qwen3.5_fp8_h100_mtp.sh) always pairs run_eval with append_lm_eval_summary. Fix: call append_lm_eval_summary right after run_eval in the EVAL_ONLY branch (line 76), matching every sibling recipe.
Extended reasoning...
run_eval() (benchmark_lib.sh:2892) only auto-calls append_lm_eval_summary when EVAL_ONLY=true AND scenario_is_agentic=1 (benchmark_lib.sh:3053). scenario_is_agentic comes from IS_AGENTIC, which benchmark-tmpl.yml:138 sets to '0' for scenario-type fixed-seq-len (this recipe's scenario). So for this script's EVAL_ONLY path, append_lm_eval_summary never runs, automatically or explicitly. run_lm_eval writes results into a mktemp dir stored in EVAL_RESULT_DIR (benchmark_lib.sh:2047,2114), not cwd; only append_lm_eval_summary -> stage_eval_artifacts (benchmark_lib.sh:2407) copies results*.json/meta_env.json into $(pwd). benchmark-tmpl.yml:310-316 then runs ls results*.json and exits 1 with 'Eval-only run failed: no results*.json files found' when EVAL_ONLY=true and none exist. Downstream, the 'Upload eval results' step (benchmark-tmpl.yml:428-444) uses if-no-files-found: error when inputs.eval-only, and validate_scores.py (line 448) has nothing to read. The author apparently copied this EVAL_ONLY/else structure from the agentic dsv41flash_fp4_h100_vllm_mtp.sh script, where the auto-append…
Verification: normal. The new script's EVAL_ONLY path stages no eval artifacts, so eval-only CI runs of this recipe fail the workflow's cwd result check. At qwen3.827b_bf16_h100_vllm_mtp.sh:75-96 the control flow is: `if [[ "${EVAL_ONLY:-false}" == true ]]; then run_eval --framework lm-eval --port "$PORT" (line 76) else run_benchmark_serving ...; if RUN_EVAL: run_eval + append_lm_eval_summary (94-95)…
|
/reuse-sweep-run 35359530151 |
Sync with origin/main after the green sweep run 35359530151; the reuse gate authorizes that run on this head. 在绿色 sweep 运行 35359530151 之后与 origin/main 同步；reuse gate 在此 head 上授权该运行。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
functionstackx
added
the
qwen3.8-27b
1 participant
Add
qwen3.827b-bf16-h100-vllm-mtp: Qwen3.8-27B in bf16 (Qwen/Qwen3.8-27B) on a single H100 (80 GB), served by vLLMnightly-cd10ed6f9f6b37a8ace9cf380007e66fe12ec0c3with TP1 and the RadixArk DSpark drafterDoopeworld/Qwen3.8-27B-DSpark-vLLM(byte-identical weights toRadixArk/Qwen3.8-27B-DSpark; onlyarchitecturesis changed so vLLM routes it toQwen3DSparkForCausalLM). Single-turn 1k1k only, concurrency 1-128.Serve shape:
--speculative-config '{"method":"dspark","model":"Doopeworld/Qwen3.8-27B-DSpark-vLLM","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'(seven tokens is the drafter's trained block size; its card measured k=4 and k=6 slower, and probabilistic sampling ~23% faster than greedy),--kv-cache-dtype fp8,--language-model-only(text-only serving ofQwen3_5ForConditionalGeneration),--reasoning-parser qwen3,--tool-call-parser qwen3_xml, and the matrix context as--max-model-len. Adaptive verification stays off because vLLM's GDN attention backend rejects it for this hybrid architecture. Prompts go through the chat template so draft acceptance reflects real text rather than random tokens; acceptance is real (no golden AL exists yet for this drafter). This PR also adds the Qwen3.8-27B rows to MODELS.md / MODELS_zh.md; the H200 PR references them.Validation:
bash -n, YAML parse, exact-key matrix generation, and changelog append (no deletions) pass. GPU validation pending;full-sweep-fail-fastis set on this PR.One PR per SKU: h100 and h200 are opened separately.
中文
新增
qwen3.827b-bf16-h100-vllm-mtp：bf16 的 Qwen3.8-27B（Qwen/Qwen3.8-27B）在单张 H100（80 GB）上由 vLLMnightly-cd10ed6f9f6b37a8ace9cf380007e66fe12ec0c3以 TP1 提供服务，并使用 RadixArk 的 DSpark 草稿模型Doopeworld/Qwen3.8-27B-DSpark-vLLM（权重与RadixArk/Qwen3.8-27B-DSpark逐字节一致，仅修改architectures以便 vLLM 路由到Qwen3DSparkForCausalLM）。仅单轮 1k1k，并发 1-128。服务参数：
--speculative-config '{"method":"dspark","model":"Doopeworld/Qwen3.8-27B-DSpark-vLLM","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'（7 为草稿模型训练时的 block size；其模型卡实测 k=4、k=6 更慢，probabilistic 采样比 greedy 快约 23%），--kv-cache-dtype fp8，--language-model-only（仅文本服务Qwen3_5ForConditionalGeneration），--reasoning-parser qwen3，--tool-call-parser qwen3_xml，--max-model-len使用矩阵上下文。由于 vLLM 的 GDN attention 后端不支持该混合架构的自适应验证，该选项保持关闭。提示词经 chat template 处理，使草稿接受率反映真实文本而非随机 token；接受率为真实值（该草稿模型尚无黄金 AL）。 本 PR 同时在 MODELS.md / MODELS_zh.md 中新增 Qwen3.8-27B 行；H200 PR 引用之。验证：
bash -n、YAML 解析、精确 key 矩阵生成、changelog 追加（无删除）均通过。GPU 验证待完成；本 PR 已添加full-sweep-fail-fast。每个 SKU 一个 PR：h100 与 h200 分别提交。
🤖 Generated with Claude Code