fix: 修复文档多处不准确 + inference scheduler 越界 bug + SchedulerCallback 回调阶段修正
文档 (6 个文件): - design.md: 15+ 处修正 — persistent_key_values→paged_cache, MLA 字段重写, Server/ParallelSetup 不存在类移除, 关系箭头方向修复, SchedulerCallback 阶段修正等 - dataflow.md: 重写数据流图和描述, 修复训练回调顺序、 数据键名、MLA 归属、MetricTracker 等错误 - introduction.md: 层数 32→24, MLP 图双 Linear 修正, 默认值/响应字段/health 端点修复 - params.md: 补充 grpo 及 4 个 GRPO 参数 - README.md / README-zh-CN.md: generate.py 补全必需参数, 删除重复注释, HuggingFace 声明修正 代码 (2 个文件): - scheduler.py: n_pages 池加 page_size 余量防止越界; decode 前预分配页 - train_callback.py: SchedulerCallback 从 on_step_end 改 回 on_batch_end (按 batch 步进学习率)
This commit is contained in:
@@ -105,7 +105,9 @@ class InferenceScheduler:
|
||||
n_kv_heads = config.n_kv_heads
|
||||
head_dim = config.dim // config.n_heads
|
||||
n_layers = config.n_layers
|
||||
n_pages = (max_batch_size * self.max_seq_len + page_size - 1) // page_size
|
||||
n_pages = (
|
||||
max_batch_size * (self.max_seq_len + page_size) + page_size - 1
|
||||
) // page_size
|
||||
|
||||
self.page_cache = PagedCache(
|
||||
n_layers,
|
||||
@@ -279,6 +281,9 @@ class InferenceScheduler:
|
||||
tasks = sorted(tasks, key=lambda t: t.task_id)
|
||||
batch_sz = len(tasks)
|
||||
|
||||
for t in tasks:
|
||||
self._maybe_alloc_page(t, start_pos)
|
||||
|
||||
input_ids = torch.zeros(batch_sz, dtype=torch.long, device=self.device)
|
||||
for i, t in enumerate(tasks):
|
||||
input_ids[i] = t.output_ids[-1] if t.output_ids else t.prompt_ids[-1]
|
||||
|
||||
Reference in New Issue
Block a user