ViperEkura 7ba43a7c6f perf: add split-K (FlashDecoding) to decode MMA kernel ... Decode has only batch*kv_head independent tasks, so the grid was tiny (e.g. 16 blocks) leaving most SMs idle (ncu: 0.04 waves/SM, 11% DRAM). - Partition KV across gridDim.z blocks emitting unnormalised (O, m, l) partials, reduced by a new combine kernel - Choose split count to fill the device (~2 blocks/SM), capped by tile count and 32; fall back to single-pass direct-write when batch*kv_head already saturates the SMs - Refactor decode dispatch into named helpers, de-duplicate scalar fallback Result: now DRAM-bound at 63% (99->543 GB/s), 2.1-2.5x over torch SDPA in the low-parallelism regime, on par at high parallelism		2026-07-10 11:43:18 +08:00
..
kernels	perf: add split-K (FlashDecoding) to decode MMA kernel	2026-07-10 11:43:18 +08:00
tests	perf: reduce MMA kernel registers, switch to static smem	2026-07-10 00:39:47 +08:00
__init__.py	feat: add optional CUDA kernel system (csrc/) + fused GQA decode attention	2026-07-06 12:09:58 +08:00
build.py	perf: add fast-math and vectorization nvcc/cxx build flags	2026-07-07 22:28:32 +08:00