AstrAI/tests/data
ViperEkura a1ea26d367 fix: rewrite GRPO data pipeline for offline record-level access
- process_list_field returns List[List[int]] preserving per-response boundaries
- GRPODataset rewritten to record-level __getitem__ (no windowing/stride)
- grpo_collate_fn pads variable-length responses into [B, G, R] tensors
- JsonlStore detects nested List[List[int]] and stores List[Tensor] per record
- Store._normalize skips nested-list keys from cumsum bookkeeping
- Pipeline._flush handles nested lists without cross-record flattening
- Export grpo_collate_fn from astrai.dataset
- 6 new GRPO tests + 2 updated builder tests, 114 total pass
2026-07-17 14:34:41 +08:00
..
conftest.py refactor: split mask builder by single/multi output 2026-07-08 21:18:34 +08:00
test_checkpoint.py refactor : replace iteration with consumed_samples 2026-06-30 18:42:42 +08:00
test_dataset.py fix: rewrite GRPO data pipeline for offline record-level access 2026-07-17 14:34:41 +08:00
test_preprocess_builder.py fix: rewrite GRPO data pipeline for offline record-level access 2026-07-17 14:34:41 +08:00
test_preprocess_config.py refactor : 清理工厂和配置系统中的死代码与冗余抽象 2026-06-07 11:39:50 +08:00
test_preprocess_pipeline.py feat: implement bfd_split packing strategy 2026-07-17 12:38:56 +08:00
test_sampler.py chore: 将data 模块命名为dataset 2026-04-04 16:16:27 +08:00