ViperEkura 8999ca89b8 feat: add JSONL dataset store with on-the-fly tokenization ... - Add JsonlStore registered under "jsonl" in astrai/dataset/storage.py - Reuse PipelineConfig schema for JSONL dataset configuration - Update detect_format to recognize JSONL directories and files - Move save_h5/load_h5/save_bin/load_bin to astrai/serialization - Split astrai/serialization.py into checkpoint/dataset submodules - Add tests for JSONL detection, seq/SFT stores, and config roundtrip		2026-07-04 15:42:33 +08:00
..
config	refactor: merge validation into MetricCallback, simplify progress bar to optimizer steps	2026-07-03 21:43:08 +08:00
dataset	feat: add JSONL dataset store with on-the-fly tokenization	2026-07-04 15:42:33 +08:00
inference	fix: 使用 threading.Event 替代裸 bool，补全公共 API	2026-06-18 15:38:35 +08:00
model	feat : GPT-2 residual scaling weight init	2026-06-25 15:08:31 +08:00
parallel	refactor : grad_norm 指标简化，clip_grad_norm 移至 executor	2026-06-30 14:59:43 +08:00
preprocessing	feat: add JSONL dataset store with on-the-fly tokenization	2026-07-04 15:42:33 +08:00
serialization	feat: add JSONL dataset store with on-the-fly tokenization	2026-07-04 15:42:33 +08:00
tokenize	refactor : Pipeline 去除去重，ids 重命名为 sequence，泛型透传	2026-05-31 15:14:27 +08:00
trainer	fix: scheduler division-by-zero, loss_mask bool	2026-07-03 22:04:55 +08:00
__init__.py	chore : 升级 torch 2.11.0+cu128，移除自定义 Muon，修复 gloo device_id	2026-06-27 16:10:37 +08:00
factory.py	fix: ForwardRef._evaluate Python 3.12 compatibility	2026-07-03 18:41:19 +08:00
protocols.py	refactor: 重构训练后端为 Executor 模式	2026-05-24 20:35:44 +08:00