AstrAI/tests
ViperEkura e220413035 feat: support raw JSON files in dataset pipeline and JsonlStore
- detect_format now recognizes .json directories as jsonl store
- JsonlStore loads .json arrays and dicts alongside .jsonl
- tokenizer_path defaults to dataset dir when omitted
- Pipeline._iter_items handles .json files (arrays/single dict)
- Tests: detect_format, seq load, self-contained dataset dir
2026-07-17 12:20:03 +08:00
..
data feat: support raw JSON files in dataset pipeline and JsonlStore 2026-07-17 12:20:03 +08:00
inference perf: replace paged KV cache with contiguous ContiguousCache, decode all groups 2026-07-05 11:34:36 +08:00
module refactor : 压缩测试代码,消除重复 2026-06-19 14:54:39 +08:00
parallel feat: 增加server, 并且修改测试单元 2026-04-02 15:05:07 +08:00
trainer feat: add --resume flag to decouple weight loading from training resumption 2026-07-16 14:23:23 +08:00
conftest.py refactor : replace iteration with consumed_samples 2026-06-30 18:42:42 +08:00