ViperEkura e220413035 feat: support raw JSON files in dataset pipeline and JsonlStore ...

- detect_format now recognizes .json directories as jsonl store
- JsonlStore loads .json arrays and dicts alongside .jsonl
- tokenizer_path defaults to dataset dir when omitted
- Pipeline._iter_items handles .json files (arrays/single dict)
- Tests: detect_format, seq load, self-contained dataset dir

2026-07-17 12:20:03 +08:00

..

data

feat: support raw JSON files in dataset pipeline and JsonlStore

2026-07-17 12:20:03 +08:00

inference

perf: replace paged KV cache with contiguous ContiguousCache, decode all groups

2026-07-05 11:34:36 +08:00

module

refactor : 压缩测试代码，消除重复

2026-06-19 14:54:39 +08:00

parallel

feat: 增加server, 并且修改测试单元

2026-04-02 15:05:07 +08:00

trainer

feat: add --resume flag to decouple weight loading from training resumption

2026-07-16 14:23:23 +08:00

conftest.py

refactor : replace iteration with consumed_samples

2026-06-30 18:42:42 +08:00