AstrAI/astrai
ViperEkura a1ea26d367 fix: rewrite GRPO data pipeline for offline record-level access
- process_list_field returns List[List[int]] preserving per-response boundaries
- GRPODataset rewritten to record-level __getitem__ (no windowing/stride)
- grpo_collate_fn pads variable-length responses into [B, G, R] tensors
- JsonlStore detects nested List[List[int]] and stores List[Tensor] per record
- Store._normalize skips nested-list keys from cumsum bookkeeping
- Pipeline._flush handles nested lists without cross-record flattening
- Export grpo_collate_fn from astrai.dataset
- 6 new GRPO tests + 2 updated builder tests, 114 total pass
2026-07-17 14:34:41 +08:00
..
config fix : SFT pipeline position_ids default & doc boundary preservation 2026-07-04 15:59:11 +08:00
dataset fix: rewrite GRPO data pipeline for offline record-level access 2026-07-17 14:34:41 +08:00
extension refactor: stride-based attn interface with layout and causal mask 2026-07-14 21:34:42 +08:00
inference refactor: clean up inference design patterns 2026-07-05 11:41:54 +08:00
model perf: replace paged KV cache with contiguous ContiguousCache, decode all groups 2026-07-05 11:34:36 +08:00
parallel feat : add distributed checkpoint via executor checkpoint_context 2026-07-13 12:27:09 +08:00
preprocessing fix: rewrite GRPO data pipeline for offline record-level access 2026-07-17 14:34:41 +08:00
serialization feat: add --resume flag to decouple weight loading from training resumption 2026-07-16 14:23:23 +08:00
tokenize refactor : Pipeline 去除去重,ids 重命名为 sequence,泛型透传 2026-05-31 15:14:27 +08:00
trainer feat: add --resume flag to decouple weight loading from training resumption 2026-07-16 14:23:23 +08:00
__init__.py chore: bump version to 1.3.9 2026-07-12 21:04:20 +08:00
factory.py fix: ForwardRef._evaluate Python 3.12 compatibility 2026-07-03 18:41:19 +08:00
protocols.py refactor: 重构训练后端为 Executor 模式 2026-05-24 20:35:44 +08:00