- Mask prompt tokens to 0 so their logprobs excluded from ratio/KL - Switch to token-level ratio + PPO clipping via reduction='none' - Slice response token logprobs from full sequence output - Replace k3 KL estimator with non-negative k1 estimator - Fix epsilon from finifo.eps (~1e-38) to 1e-8 - Remove unused 'reduction' param from GRPOStrategy.__init__ - Clarify offline batch semantics in docstring - Add 11 unit tests for masking, advantage, KL, sync, clipping - Sync training.md and architecture.md docs |
||
|---|---|---|
| .. | ||
| config | ||
| dataset | ||
| extension | ||
| inference | ||
| model | ||
| parallel | ||
| preprocessing | ||
| serialization | ||
| tokenize | ||
| trainer | ||
| __init__.py | ||
| factory.py | ||
| protocols.py | ||