docs : 三轮深度验证修复文档与代码不一致
- architecture.md: 修正 unwrap_model 返回类型、Config Optional 标注、方法签名错误、类名错误 - training.md: 补充 on_error 回调、修正训练循环顺序、补全策略参数、model.safetensors - inference.md: 修正 GenerationRequest 参数顺序、async 语法、KVCache 描述、temperature 约束 - dataflow.md: 补充 Store.load/fetch 流程、修正可选参数默认值 - README/params: 多 GPU 示例补全 --parallel_mode、文档表补充 preprocessing.md - preprocessing.md: Chat 模式算法补全 BOS token 步骤
This commit is contained in:
@@ -147,10 +147,11 @@ For instruction mode, keys are `"prompt"` and `"response"`.
|
||||
|
||||
For each message in the `messages` array:
|
||||
|
||||
1. Render through the chat template for that single message
|
||||
2. Encode the rendered text, record token span `(start, end, role)`
|
||||
3. Concatenate all spans -- special tokens from the chat template naturally prevent BPE merging across message boundaries
|
||||
4. Fill `loss_mask` from the mask rules
|
||||
1. Prepend BOS token (position 0, always masked)
|
||||
2. Render through the chat template for that single message
|
||||
3. Encode the rendered text, record token span `(start, end, role)`
|
||||
4. Concatenate all spans — special tokens from the chat template naturally prevent BPE merging across message boundaries
|
||||
5. Fill `loss_mask` from the mask rules
|
||||
|
||||
**Multi-turn example**:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user