fix: broadcast ref/old model state_dict for FSDP

- Add broadcast_state_dict to sync state_dict from rank-0 to all ranks
- Fix create_ref_model returning None on non-rank-0 under FSDP
- Fix sync_old_model only updating old_model on rank-0 under FSDP
- Split skip_no_cuda/skip_no_kernel markers and hoist to top-level conftest
- Add distributed tests for broadcast_state_dict and create_ref_model
This commit is contained in:
2026-07-31 08:32:22 +08:00
parent 28d1bd07cf
commit 738cb8f128
9 changed files with 248 additions and 17 deletions
+2
View File
@@ -7,6 +7,7 @@ from astrai.parallel.executor import (
FSDPExecutor,
GradientState,
NoneExecutor,
broadcast_state_dict,
create_ref_model,
)
from astrai.parallel.setup import (
@@ -34,4 +35,5 @@ __all__ = [
"DDPExecutor",
"FSDPExecutor",
"create_ref_model",
"broadcast_state_dict",
]