AstrAI

Files

T

ViperEkura 2c7a71a9c0 refactor: separate old policy and ref model in GRPO strategy

- Split single ref_model into old_model (importance sampling ratio) and ref_model (frozen KL regularizer)
- Move ref_model/old_model creation from strategy __init__ to TrainContextBuilder, pass as explicit parameters
- Remove periodic sync_ref_model + sync_interval; add sync_old_model for external rollout loop to call
- DPOStrategy also receives ref_model from builder
- Fix std to use unbiased=False (population std per GRPO paper)
- Remove redundant tests (test_grpo_kl_zero_at_init, test_grpo_no_sync_interval_param)
- Remove --grpo_sync_interval CLI arg

2026-07-14 20:03:45 +08:00

conftest.py

refactor : replace iteration with consumed_samples

2026-06-30 18:42:42 +08:00

test_callbacks.py

refactor : 清理工厂和配置系统中的死代码与冗余抽象

2026-06-07 11:39:50 +08:00

test_early_stopping.py

refactor : replace iteration with consumed_samples

2026-06-30 18:42:42 +08:00

test_grpo_strategy.py

refactor: separate old policy and ref model in GRPO strategy