AstrAI/astrai/dataset
ViperEkura a1ea26d367 fix: rewrite GRPO data pipeline for offline record-level access
- process_list_field returns List[List[int]] preserving per-response boundaries
- GRPODataset rewritten to record-level __getitem__ (no windowing/stride)
- grpo_collate_fn pads variable-length responses into [B, G, R] tensors
- JsonlStore detects nested List[List[int]] and stores List[Tensor] per record
- Store._normalize skips nested-list keys from cumsum bookkeeping
- Pipeline._flush handles nested lists without cross-record flattening
- Export grpo_collate_fn from astrai.dataset
- 6 new GRPO tests + 2 updated builder tests, 114 total pass
2026-07-17 14:34:41 +08:00
..
__init__.py fix: rewrite GRPO data pipeline for offline record-level access 2026-07-17 14:34:41 +08:00
dataset.py fix: rewrite GRPO data pipeline for offline record-level access 2026-07-17 14:34:41 +08:00
sampler.py fix: reset sampler iter at epoch end so progress bar shows total after first epoch 2026-07-04 06:35:55 +08:00
storage.py fix: rewrite GRPO data pipeline for offline record-level access 2026-07-17 14:34:41 +08:00