fix: resolve audited dispatch, kernel, and rollout bugs
- re-register the linear family with the operator dispatcher (ASTR_OPS / op_backend / resolve) - fix bf16 gemv misaligned-address faults and element mispairing for offset weights - reject misaligned bf16_swiglu inputs with a clear error and fall back in the backend gate - make the rollout reuse decision, validation, and return atomic under one policy snapshot - add the documented post-scoring rollout version check - derive live+1 under the scheduler lock in optimizer_step via apply_weight_update(None, ...) - reject rollout_max_policy_lag below rollout_interval - 1 at config time - sync gemv stream-test inputs before switching streams; drop dead loader imports
This commit is contained in:
@@ -234,4 +234,15 @@ class TrainConfig(BaseConfig):
|
||||
f"numbers of forward passes and desynchronize the "
|
||||
f"ddp/fsdp collectives, deadlocking NCCL"
|
||||
)
|
||||
if (
|
||||
self.rollout_max_policy_lag is not None
|
||||
and self.rollout_max_policy_lag < self.rollout_interval - 1
|
||||
):
|
||||
raise ValueError(
|
||||
f"rollout_max_policy_lag={self.rollout_max_policy_lag} "
|
||||
f"cannot be below rollout_interval - 1 = "
|
||||
f"{self.rollout_interval - 1}: the replay cache reuses "
|
||||
f"rollouts up to that lag, so a tighter bound guarantees "
|
||||
f"a fatal RolloutVersionError mid-training"
|
||||
)
|
||||
return self
|
||||
|
||||
Reference in New Issue
Block a user