ViperEkura d0c5debbab perf: preload V in decode split-kv shared mem and cache partial tensors ...

- Preload V into shared memory alongside K to eliminate per-element KV address lookups in the inner softmax/accum loop (doubles smem)
- Cache split-KV partial tensors (o_part, ml_part) with static tensors instead of per-call allocation in both decode and paged-decode paths
- Force is_causal=True in CUDA decode backend (decode is always causal)

2026-08-06 18:27:00 +08:00

..

kernels

perf: preload V in decode split-kv shared mem and cache partial tensors

2026-08-06 18:27:00 +08:00

tests

refactor: unify paged and contiguous attention kernels via KVSource policy

2026-08-05 14:06:13 +08:00

__init__.py

feat: add optional CUDA kernel system (csrc/) + fused GQA decode attention

2026-07-06 12:09:58 +08:00

CMakeLists.txt

build: migrate CUDA kernel build to CMake

2026-08-04 21:27:22 +08:00