- Kernel is latency-bound (25% occupancy), not compute/bandwidth-bound - BC=16 wasted a cp.async wait + barrier + loop overhead per tiny tile - Double KV tile to BC=32 for D<=128; D=256 stays 16 (64KB > 48KB smem cap) - Retune MIN_BLOCKS per head_dim (32->6, 64->4, 128->3, 256->2) - Result: ~6-8% faster on L20, 0.93-1.20x vs torch SDPA, correctness unchanged |
||
|---|---|---|
| .. | ||
| gqa_decode_test.cu | ||
| gqa_prefill_test.cu | ||