fix: 修复推理引擎 batch decode 中多项正确性与并发问题

- scheduler: decode 分组由幂次分桶改为精确 next_pos,消除 KV cache 位置错乱
- task: activate() 加锁操作 active_tasks,消除数据竞争
- engine: wait_completion 加超时,防止分配失败时永久死锁
- sample: TopKStrategy 向量化为 per-sample threshold,尊重各 task 的 top_k
- cache: Storage.write/gather 中 -1 页改用 mask 处理,防数据污染
- executor: prefill 逐 task 循环改为单次 tensor 调用
This commit is contained in:
2026-05-14 21:31:39 +08:00
parent f0339022c1
commit e3382f6bb5
7 changed files with 60 additions and 23 deletions
+1 -3
View File
@@ -126,9 +126,7 @@ class InferenceScheduler:
pos_groups: Dict[int, List[Task]] = {}
for t in self._task_mgr.get_active_tasks():
chunk = t.next_pos // self._page_cache.page_size
key = chunk if chunk <= 1 else 1 << (chunk.bit_length() - 1)
pos_groups.setdefault(key, []).append(t)
pos_groups.setdefault(t.next_pos, []).append(t)
if pos_groups:
best_key = max(pos_groups, key=lambda k: len(pos_groups[k]))