ViperEkura e220413035 feat: support raw JSON files in dataset pipeline and JsonlStore ... - detect_format now recognizes .json directories as jsonl store - JsonlStore loads .json arrays and dicts alongside .jsonl - tokenizer_path defaults to dataset dir when omitted - Pipeline._iter_items handles .json files (arrays/single dict) - Tests: detect_format, seq load, self-contained dataset dir		2026-07-17 12:20:03 +08:00
..
conftest.py	refactor: split mask builder by single/multi output	2026-07-08 21:18:34 +08:00
test_checkpoint.py	refactor : replace iteration with consumed_samples	2026-06-30 18:42:42 +08:00
test_dataset.py	feat: support raw JSON files in dataset pipeline and JsonlStore	2026-07-17 12:20:03 +08:00
test_preprocess_builder.py	refactor: split mask builder by single/multi output	2026-07-08 21:18:34 +08:00
test_preprocess_config.py	refactor : 清理工厂和配置系统中的死代码与冗余抽象	2026-06-07 11:39:50 +08:00
test_preprocess_pipeline.py	refactor : 压缩测试代码，消除重复	2026-06-19 14:54:39 +08:00
test_sampler.py	chore: 将data 模块命名为dataset	2026-04-04 16:16:27 +08:00