ViperEkura 81788faef4 perf: use flash_attn_with_kvcache for contiguous cache decode ...

- Decode with contiguous cache uses flash_attn_with_kvcache instead of materializing full KV via gather + flash_attn_func
- _backend_supports allows FlashAttnBackend for decode (q_len==1) even with explicit mask
- Decode speedups vs TorchNative (B=1,4,8,16 mean): cuda 1.55x, flash 1.40x, torch_native 1.00x
- Read K/V directly from flat pool via cache_batch_idx + cache_seqlens, zero-copy view reshape

2026-08-07 18:21:02 +08:00

..

lib

refactor: move compiled kernel .so files into extension/lib

2026-07-31 15:36:32 +08:00

__init__.py

refactor : use factory for attention backends

2026-08-05 15:37:22 +08:00

attention_backend.py

perf: use flash_attn_with_kvcache for contiguous cache decode

2026-08-07 18:21:02 +08:00

attention_ops.py

perf: move decode split partials to InferenceWorkspace

2026-08-06 19:12:09 +08:00

loader.py

feat: SGLang-style paged attention kernels replace page-table path

2026-08-01 15:41:25 +08:00

rotary_backend.py

refactor: unify rotary embedding interface and update docs

2026-07-31 16:52:25 +08:00

rotary_ops.py

refactor: unify rotary embedding interface and update docs

2026-07-31 16:52:25 +08:00