docs : 三轮深度验证修复文档与代码不一致

- architecture.md: 修正 unwrap_model 返回类型、Config Optional 标注、方法签名错误、类名错误
- training.md: 补充 on_error 回调、修正训练循环顺序、补全策略参数、model.safetensors
- inference.md: 修正 GenerationRequest 参数顺序、async 语法、KVCache 描述、temperature 约束
- dataflow.md: 补充 Store.load/fetch 流程、修正可选参数默认值
- README/params: 多 GPU 示例补全 --parallel_mode、文档表补充 preprocessing.md
- preprocessing.md: Chat 模式算法补全 BOS token 步骤
This commit is contained in:
2026-05-30 21:41:06 +08:00
parent 31ae2deeba
commit 1c2ff05a6d
8 changed files with 219 additions and 91 deletions
+11 -7
View File
@@ -12,7 +12,7 @@ RoPE is applied **before** KV cache write, not after — otherwise position enco
## KVCache System
Six classes working together:
Six classes (plus two helpers) working together:
```
KVCache (facade)
@@ -43,7 +43,8 @@ KVCache (facade)
BaseSamplingStrategy (ABC)
├── TemperatureStrategy
├── TopKStrategy
── TopPStrategy
── TopPStrategy
└── SamplingPipeline
```
`SamplingPipeline` composes them: Temperature → Top-K → Top-P → softmax → multinomial.
@@ -73,7 +74,9 @@ Adding a protocol = one builder file, no handler subclassing needed.
InferenceEngine
├── generate(prompt, stream, ...) → str | List[str] | Generator
├── generate_with_request(req) → same
── generate_async(prompt, ...) → AsyncGenerator
── generate_async(prompt, ...) → AsyncGenerator
├── get_stats() → Dict
└── shutdown()
```
`GenerateResult` uses `Condition` for non-streaming (`wait_completion()`) and `Event` for streaming (`wait()`). Stream callback is `cb(token)`.
@@ -124,9 +127,9 @@ Supports `stop_sequences` and streaming via `event: content_block_delta`.
| Param | Type | Default | Description |
|-------|------|---------|-------------|
| `messages` | List[dict] | required | Chat messages (role, content) |
| `temperature` | float | 1.0 | Sampling temperature (>= 0.0) |
| `top_p` | float | 1.0 | Nucleus threshold |
| `top_k` | int | 50 | Top-k count |
| `top_p` | float | 1.0 | Nucleus threshold |
| `temperature` | float | 1.0 | Sampling temperature (> 0.0) |
| `max_tokens` | Optional[int] | None | Max generation length |
| `stream` | bool | False | Stream output |
@@ -142,7 +145,8 @@ engine.generate("Hello", stream=True) # -> Generator[str]
engine.generate(["A", "B"], stream=True) # -> Generator[Tuple[int, str]]
# Async
await engine.generate_async("Hello", ...) # -> AsyncGenerator[str]
async for token in engine.generate_async("Hello", ...): # -> AsyncGenerator[str]
print(token)
```
> Document Update Time: 2026-05-28
> Document Update Time: 2026-05-30