fix: rewrite GRPO data pipeline for offline record-level access
- process_list_field returns List[List[int]] preserving per-response boundaries - GRPODataset rewritten to record-level __getitem__ (no windowing/stride) - grpo_collate_fn pads variable-length responses into [B, G, R] tensors - JsonlStore detects nested List[List[int]] and stores List[Tensor] per record - Store._normalize skips nested-list keys from cumsum bookkeeping - Pipeline._flush handles nested lists without cross-record flattening - Export grpo_collate_fn from astrai.dataset - 6 new GRPO tests + 2 updated builder tests, 114 total pass
This commit is contained in:
@@ -180,9 +180,24 @@ class Pipeline:
|
||||
dt = _STR_TO_DTYPE.get(
|
||||
self.config.output.dtype.get(key, "int32"), torch.int32
|
||||
)
|
||||
tensors[key] = [
|
||||
torch.tensor(list(chain.from_iterable(ids_list)), dtype=dt)
|
||||
]
|
||||
# GRPO multi-response keys store List[List[int]] per record
|
||||
# (responses/masks). Rewards store List[float] per record.
|
||||
# Both produce List[Tensor] (one tensor per record), but
|
||||
# responses need inner flattening while rewards do not.
|
||||
if ids_list and isinstance(ids_list[0], list):
|
||||
tensors[key] = [
|
||||
torch.tensor(
|
||||
list(chain.from_iterable(ids))
|
||||
if ids and isinstance(ids[0], list)
|
||||
else ids,
|
||||
dtype=dt,
|
||||
)
|
||||
for ids in ids_list
|
||||
]
|
||||
else:
|
||||
tensors[key] = [
|
||||
torch.tensor(list(chain.from_iterable(ids_list)), dtype=dt)
|
||||
]
|
||||
|
||||
if mode == "continuous" and original_sequences:
|
||||
pos_ids = self._position_id.generate(keys.get("sequence", []))
|
||||
|
||||
Reference in New Issue
Block a user