fix: keep async rollouts version-consistent

- serialize shared-model optimizer updates with generation - reject future or over-lagged rollout results after asynchronous scoring - close cache publication races - persist policy versions in online checkpoints
2026-09-03 12:01:24 +08:00
parent ce2f9d13b3
commit 587b0ee046
16 changed files with 531 additions and 49 deletions
@@ -94,8 +94,7 @@ class Trainer:

                        if executor.sync_gradients:
                            self._call_callbacks("before_optimizer_step", context)
-                            context.optimizer.step()
-                            context.strategy.on_optimizer_step()
+                            context.strategy.optimizer_step(context.optimizer)
                            context.optimizer.zero_grad()

                            if context.scheduler: