fix: 修复推理引擎 batch decode 中多项正确性与并发问题

- scheduler: decode 分组由幂次分桶改为精确 next_pos,消除 KV cache 位置错乱
- task: activate() 加锁操作 active_tasks,消除数据竞争
- engine: wait_completion 加超时,防止分配失败时永久死锁
- sample: TopKStrategy 向量化为 per-sample threshold,尊重各 task 的 top_k
- cache: Storage.write/gather 中 -1 页改用 mask 处理,防数据污染
- executor: prefill 逐 task 循环改为单次 tensor 调用
This commit is contained in:
2026-05-14 21:31:39 +08:00
parent f0339022c1
commit e3382f6bb5
7 changed files with 60 additions and 23 deletions
+2 -2
View File
@@ -48,12 +48,12 @@ def test_top_k_skip_when_zero():
def test_top_k_batch_tensor():
"""When top_k is a batch tensor, max element governs k for all rows."""
"""Each row respects its own top_k."""
logits = torch.tensor([[0.1, 0.5, 0.3], [0.9, 0.2, 0.1]])
s = TopKStrategy(top_k=torch.tensor([2, 1]))
result = s.apply(logits.clone(), filter_value=-1e9)
assert (result[0] > -1e9).sum() == 2
assert (result[1] > -1e9).sum() == 2
assert (result[1] > -1e9).sum() == 1
def test_top_p_nucleus_filtering():