ViperEkura 8999ca89b8 feat: add JSONL dataset store with on-the-fly tokenization ... - Add JsonlStore registered under "jsonl" in astrai/dataset/storage.py - Reuse PipelineConfig schema for JSONL dataset configuration - Update detect_format to recognize JSONL directories and files - Move save_h5/load_h5/save_bin/load_bin to astrai/serialization - Split astrai/serialization.py into checkpoint/dataset submodules - Add tests for JSONL detection, seq/SFT stores, and config roundtrip		2026-07-04 15:42:33 +08:00
..
data	feat: add JSONL dataset store with on-the-fly tokenization	2026-07-04 15:42:33 +08:00
inference	refactor : 压缩测试代码，消除重复	2026-06-19 14:54:39 +08:00
module	refactor : 压缩测试代码，消除重复	2026-06-19 14:54:39 +08:00
parallel	feat: 增加server, 并且修改测试单元	2026-04-02 15:05:07 +08:00
trainer	refactor : replace iteration with consumed_samples	2026-06-30 18:42:42 +08:00
conftest.py	refactor : replace iteration with consumed_samples	2026-06-30 18:42:42 +08:00