ViperEkura 81788faef4 perf: use flash_attn_with_kvcache for contiguous cache decode ...

- Decode with contiguous cache uses flash_attn_with_kvcache instead of materializing full KV via gather + flash_attn_func
- _backend_supports allows FlashAttnBackend for decode (q_len==1) even with explicit mask
- Decode speedups vs TorchNative (B=1,4,8,16 mean): cuda 1.55x, flash 1.40x, torch_native 1.00x
- Read K/V directly from flat pool via cache_batch_idx + cache_seqlens, zero-copy view reshape

2026-08-07 18:21:02 +08:00

..

config

feat: add moe auxloss and metrics

2026-08-05 18:12:28 +08:00

dataset

refactor: remove H5 storage backend in favor of mmap bin

2026-07-29 12:50:27 +08:00

extension

perf: use flash_attn_with_kvcache for contiguous cache decode

2026-08-07 18:21:02 +08:00

inference

refactor: unify greedy check with _is_greedy helper

2026-08-07 14:14:15 +08:00

model

refactor: stateless MoE routing with grouped dispatch

2026-08-05 18:42:12 +08:00

optim

fix: use torch.optim.AdamW in ManoAdamW instead of NAdamW

2026-08-01 09:20:54 +08:00

parallel

fix: broadcast ref/old model state_dict for FSDP

2026-07-31 08:32:22 +08:00

preprocessing

fix : correct prefill sampling and record alignment

2026-08-05 22:20:29 +08:00

serialization

refactor: remove H5 storage backend in favor of mmap bin

2026-07-29 12:50:27 +08:00

tokenize

refactor: simplify BaseFactory and separate ModelFactory from AutoModel

2026-07-30 09:38:20 +08:00

trainer

refactor: stateless MoE routing with grouped dispatch

2026-08-05 18:42:12 +08:00

__init__.py

chore: bump version to 1.3.12

2026-08-01 09:22:16 +08:00

factory.py

refactor: simplify BaseFactory and separate ModelFactory from AutoModel

2026-07-30 09:38:20 +08:00

protocols.py

refactor: 重构训练后端为 Executor 模式

2026-05-24 20:35:44 +08:00

signal_handler.py

refactor: move signal_handler from parallel/ to top-level for broader reuse

2026-07-28 10:36:17 +08:00