ViperEkura 8206afefd9 fix: FSDP clip_grad_norm and default reshard_after_forward=False ...

- FSDP params are DTensors sharded across ranks
- torch.nn.utils.clip_grad_norm_ computes LOCAL norm only
- Each rank would clip by a different factor, causing gradient divergence
- Fix: compute local norm, all-reduce squared sum, sqrt for global norm
- Default reshard_after_forward=False (forward then backward makes reshard redundant)
- Reduces per-step time by ~19% (1033ms to 839ms on 2xL20)

2026-07-29 23:27:10 +08:00

..

__init__.py

refactor: replace FSDP with FSDP2 as default parallel backend

2026-07-29 23:09:37 +08:00

executor.py

fix: FSDP clip_grad_norm and default reshard_after_forward=False

2026-07-29 23:27:10 +08:00

module.py

refactor: 优化参数传递，清理导入样式

2026-04-03 22:06:32 +08:00

setup.py

refactor: move signal_handler from parallel/ to top-level for broader reuse

2026-07-28 10:36:17 +08:00