AstrAI/astrai/serialization
ViperEkura b092316385 feat : add distributed checkpoint via executor checkpoint_context
- Add checkpoint_context context manager to BaseExecutor with entry/exit barrier
- Add _gather_state_dict hook overridden per executor (template method)
- DDPExecutor skips unwrap on non-rank-0 to avoid redundant state_dict gather
- FSDPExecutor uses rank0_only=True to reduce memory on non-writers
- Remove redundant rank-0 guard from Checkpoint.save and manual barrier from Callback
2026-07-13 12:27:09 +08:00
..
__init__.py feat: add JSONL dataset store with on-the-fly tokenization 2026-07-04 15:42:33 +08:00
checkpoint.py feat : add distributed checkpoint via executor checkpoint_context 2026-07-13 12:27:09 +08:00
dataset.py fix: support single .h5 file loading in load_h5 2026-07-07 00:11:16 +08:00