ViperEkura 8c052c99ee feat: add optional FlashAttention (FA2/FA3) backend ...

- add FlashAttnBackend (ATTN_BACKEND.FLASH) using flash_attn_func with KV-cache gather + GQA, mirroring TorchNativeBackend
- add flash_attn_available() probe gated on compute capability plus a real-kernel smoke test, cached at first use
- lazy-import flash-attn via importlib so it stays an optional dependency, raising clear errors when unusable
- add 'flash' optional extra (flash-attn>=2.6) and export the new backend

2026-08-05 15:27:26 +08:00

..

config

feat: add MoE auxiliary loss metrics

2026-08-02 06:30:43 +08:00

dataset

refactor: remove H5 storage backend in favor of mmap bin

2026-07-29 12:50:27 +08:00

extension

feat: add optional FlashAttention (FA2/FA3) backend

2026-08-05 15:27:26 +08:00

inference

refactor: share prefill+decode step between scheduler paths

2026-08-03 13:45:27 +08:00

model

feat: add MoE auxiliary loss metrics

2026-08-02 06:30:43 +08:00

optim

fix: use torch.optim.AdamW in ManoAdamW instead of NAdamW

2026-08-01 09:20:54 +08:00

parallel

fix: broadcast ref/old model state_dict for FSDP

2026-07-31 08:32:22 +08:00

preprocessing

refactor: remove H5 storage backend in favor of mmap bin

2026-07-29 12:50:27 +08:00

serialization

refactor: remove H5 storage backend in favor of mmap bin

2026-07-29 12:50:27 +08:00

tokenize

refactor: simplify BaseFactory and separate ModelFactory from AutoModel

2026-07-30 09:38:20 +08:00

trainer

refactor: emit strategy metrics as floats

2026-08-02 06:38:28 +08:00

__init__.py

chore: bump version to 1.3.12

2026-08-01 09:22:16 +08:00

factory.py

refactor: simplify BaseFactory and separate ModelFactory from AutoModel

2026-07-30 09:38:20 +08:00

protocols.py

refactor: 重构训练后端为 Executor 模式

2026-05-24 20:35:44 +08:00

signal_handler.py

refactor: move signal_handler from parallel/ to top-level for broader reuse

2026-07-28 10:36:17 +08:00