ViperEkura
2c7a71a9c0
- Split single ref_model into old_model (importance sampling ratio) and ref_model (frozen KL regularizer) - Move ref_model/old_model creation from strategy __init__ to TrainContextBuilder, pass as explicit parameters - Remove periodic sync_ref_model + sync_interval; add sync_old_model for external rollout loop to call - DPOStrategy also receives ref_model from builder - Fix std to use unbiased=False (population std per GRPO paper) - Remove redundant tests (test_grpo_kl_zero_at_init, test_grpo_no_sync_interval_param) - Remove --grpo_sync_interval CLI arg |
||
|---|---|---|
| .. | ||
| conftest.py | ||
| test_callbacks.py | ||
| test_early_stopping.py | ||
| test_grpo_strategy.py | ||
| test_train_strategy.py | ||
| test_trainer.py | ||