fix: broadcast ref/old model state_dict for FSDP

- Add broadcast_state_dict to sync state_dict from rank-0 to all ranks
- Fix create_ref_model returning None on non-rank-0 under FSDP
- Fix sync_old_model only updating old_model on rank-0 under FSDP
- Split skip_no_cuda/skip_no_kernel markers and hoist to top-level conftest
- Add distributed tests for broadcast_state_dict and create_ref_model
This commit is contained in:
2026-07-31 08:32:22 +08:00
parent 28d1bd07cf
commit 738cb8f128
9 changed files with 248 additions and 17 deletions
+2
View File
@@ -165,6 +165,8 @@ class FakeTokenizer:
class FakeExecutor:
"""Executor stub tracking ``sync_gradients`` and providing ``unwrap_model``."""
use_distributed = False
def __init__(self, sync_gradients=True):
self._sync_gradients = sync_gradients