AstrAI/csrc/kernels
ViperEkura 88f8dca2c2 perf: enlarge prefill KV tile to BC=32 for D<=128
- Kernel is latency-bound (25% occupancy), not compute/bandwidth-bound
- BC=16 wasted a cp.async wait + barrier + loop overhead per tiny tile
- Double KV tile to BC=32 for D<=128; D=256 stays 16 (64KB > 48KB smem cap)
- Retune MIN_BLOCKS per head_dim (32->6, 64->4, 128->3, 256->2)
- Result: ~6-8% faster on L20, 0.93-1.20x vs torch SDPA, correctness unchanged
2026-07-10 17:43:23 +08:00
..
gqa_common.cuh fix: correct prefill mask index, unify GQA kernel interface 2026-07-06 17:21:23 +08:00
gqa_decode_attn.cu perf: add split-K (FlashDecoding) to decode MMA kernel 2026-07-10 11:43:18 +08:00
gqa_decode_attn.cuh fix: correct prefill mask index, unify GQA kernel interface 2026-07-06 17:21:23 +08:00
gqa_decode_attn_mma.cuh perf: add split-K (FlashDecoding) to decode MMA kernel 2026-07-10 11:43:18 +08:00
gqa_mma_utils.cuh perf: pipeline prefill MMA kernel (double-buffered K/V + packed stores) 2026-07-10 12:19:14 +08:00
gqa_prefill_attn.cu perf: enlarge prefill KV tile to BC=32 for D<=128 2026-07-10 17:43:23 +08:00
gqa_prefill_attn.cuh perf: reduce MMA kernel registers, switch to static smem 2026-07-10 00:39:47 +08:00
gqa_prefill_attn_mma.cuh perf: enlarge prefill KV tile to BC=32 for D<=128 2026-07-10 17:43:23 +08:00