ViperEkura/AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs : sync 6 doc files to actual code

Browse Source 
- architecture.md: removed TrainConfig.log_interval, split KVCache into
  PageCache/ContiguousCache with CacheView/PageCacheView/ContiguousCacheView,
  added JsonlStore, fixed GradientCheckpointingCallback type,
  CheckpointCallback typo, ProgressBarCallback hooks
- training.md: added position_ids to SFT keys, fixed callback hook table,
  removed merged ValidationCallback
- inference.md: documented ContiguousCache default vs PageCache paged
- dataflow.md: added JsonlStore to storage backends and format detection
- params.md: removed nonexistent --log_interval
- preprocessing.md: updated timestamp
This commit is contained in:
ViperEkura
2026-07-05 19:35:18 +08:00
parent bbe6ff2d8f
commit abb96996f8
6 changed files with 109 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files
assets/docs/training.md
+8 -8
View File
@@ -80,13 +80,13 @@ on_train_end
| `on_train_begin` | Before training starts | `GradientCheckpointingCallback` |
| `on_epoch_begin` | Start of each epoch | `ProgressBarCallback` |
| `on_batch_begin` | Every batch | — |
| `on_optimizer_step` | Every accumulation window | `GradientClippingCallback`, `MetricLoggerCallback`, `ValidationCallback` |
| `on_batch_end` | Every batch | `CheckpointCallback`, `MetricLoggerCallback`, `ProgressBarCallback` |
| `on_epoch_end` | End of each epoch | `ProgressBarCallback` |
| `on_error` | On exception during training | `CheckpointCallback`, `MetricLoggerCallback` |
| `on_train_end` | Training ends (always via finally) | `CheckpointCallback`, `MetricLoggerCallback`, `GradientCheckpointingCallback` |
| `on_optimizer_step` | Every accumulation window | `GradientClippingCallback`, `MetricCallback`, `ProgressBarCallback` |
| `on_batch_end` | Every batch | `CheckpointCallback` |
| `on_epoch_end` | End of each epoch | `MetricCallback`, `ProgressBarCallback` |
| `on_error` | On exception during training | `CheckpointCallback`, `MetricCallback` |
| `on_train_end` | Training ends (always via finally) | `CheckpointCallback`, `MetricCallback`, `GradientCheckpointingCallback` |
Default callbacks (in order): `gradient_checkpointing` (activation checkpointing, optional), `checkpoint` (safetensors, rank-0), `validation` (periodic validation on val_dataset), `metric_logger` (JSONL, rank-0), `progress_bar` (tqdm), `gradient_clipping`.
Default callbacks (in order): `gradient_checkpointing` (activation checkpointing, optional), `checkpoint` (safetensors, rank-0), `metric` (JSONL + validation, rank-0), `progress_bar` (tqdm), `gradient_clipping`.
## Strategies
@@ -108,7 +108,7 @@ $$
L_{\text{SFT}} = -\sum_{t=P+1}^{P+L} \log P(s_t \mid s_{\lt t}; \theta)
$$
Keys: `input_ids`, `target_ids`, `loss_mask`. Optional: `label_smoothing`.
Keys: `input_ids`, `target_ids`, `loss_mask`, `position_ids`. Optional: `label_smoothing`.
### DPO (Direct Preference Optimization)
@@ -214,4 +214,4 @@ nohup python scripts/tools/train.py \
Full parameter reference at [params.md](params.md).
> Document Update Time: 2026-05-30
> Document Update Time: 2026-07-05