Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题与结果
Qwen3-Next/Qwen3.5 的线性注意力在 speculative verify 时会为每个候选 token 保存完整 SSM state,状态写入和显存占用随 MTP 宽度增长。本 PR 增加默认关闭的
--enable_replayssm:GDN FP32 使用单份 checkpoint 和接受历史,GDN BF16/KDA MTP 使用紧凑 raw-input cache,避免保存候选状态快照。实现
k/v与 FP32gate/beta按原生递推重放,派生 delta 不进入长期状态。验证
34 passed, 4 skipped,覆盖 accepted prefix、不同窗口、CUDA Graph、循环 flush、CPU cache、PD page、请求复用以及 Compact/native 对比。实验记录:260920-170629-*。1.06x / 1.09x / 1.16x;Replay state/history 约占原生 speculative SSM snapshot 的 44%。实验记录:260920-165530-*。未覆盖与评审重点