AstrAI/astrai
ViperEkura 9bcd696580 fix: token-level ratio and prompt masking in GRPO strategy
- Mask prompt tokens to 0 so their logprobs excluded from ratio/KL
- Switch to token-level ratio + PPO clipping via reduction='none'
- Slice response token logprobs from full sequence output
- Replace k3 KL estimator with non-negative k1 estimator
- Fix epsilon from finifo.eps (~1e-38) to 1e-8
- Remove unused 'reduction' param from GRPOStrategy.__init__
- Clarify offline batch semantics in docstring
- Add 11 unit tests for masking, advantage, KL, sync, clipping
- Sync training.md and architecture.md docs
2026-07-12 21:24:09 +08:00
..
config fix : SFT pipeline position_ids default & doc boundary preservation 2026-07-04 15:59:11 +08:00
dataset feat: add JSONL dataset store with on-the-fly tokenization 2026-07-04 15:42:33 +08:00
extension feat: wire up paged decode CUDA kernel to Python extension 2026-07-11 18:40:49 +08:00
inference refactor: clean up inference design patterns 2026-07-05 11:41:54 +08:00
model perf: replace paged KV cache with contiguous ContiguousCache, decode all groups 2026-07-05 11:34:36 +08:00
parallel fix: auto-assign free port in spawn_parallel_fn to avoid EADDRINUSE 2026-07-12 19:21:56 +08:00
preprocessing refactor: split mask builder by single/multi output 2026-07-08 21:18:34 +08:00
serialization fix: support single .h5 file loading in load_h5 2026-07-07 00:11:16 +08:00
tokenize refactor : Pipeline 去除去重,ids 重命名为 sequence,泛型透传 2026-05-31 15:14:27 +08:00
trainer fix: token-level ratio and prompt masking in GRPO strategy 2026-07-12 21:24:09 +08:00
__init__.py chore: bump version to 1.3.9 2026-07-12 21:04:20 +08:00
factory.py fix: ForwardRef._evaluate Python 3.12 compatibility 2026-07-03 18:41:19 +08:00
protocols.py refactor: 重构训练后端为 Executor 模式 2026-05-24 20:35:44 +08:00