ViperEkura b1b65a657e perf: target 512 grid blocks for decode split-K ...

- compute_num_splits used 2*sm/base, undersplitting at large batch
- single-warp decode blocks host ~11/SM, not 1/2-SM, so B=16 got 3 splits when 8 was optimal
- Grid search on L20: bandwidth saturates near 256-512 total blocks; target 512
- Pass num_passes into base_blocks for the non-paged decode to match the paged path
- B=16 kv=2048: 0.0230->0.0157ms (-32%); paged B=16: 0.0527->0.0243ms (-54%); B=32: 0.0406->0.0241ms (-41%)

2026-08-02 16:10:40 +08:00

..

attn_common.h

feat: SGLang-style paged attention kernels replace page-table path

2026-08-01 15:41:25 +08:00

attn_decode_split_kv_mma.cuh

perf: remove split partials memset and overlap decode tile loads

2026-07-31 22:37:44 +08:00

attn_decode_split_kv.cuh

style: unify decode expf to __expf

2026-07-31 00:19:18 +08:00

attn_decode.cu

perf: launch CUDA kernels on torch's current stream

2026-08-02 13:20:14 +08:00

attn_dispatchers.cuh

perf: target 512 grid blocks for decode split-K

2026-08-02 16:10:40 +08:00

attn_entry_utils.cuh

perf: launch CUDA kernels on torch's current stream

2026-08-02 13:20:14 +08:00

attn_mma_utils.cuh

build: parametrize CUDA version for wheels and docker

2026-07-31 14:10:55 +08:00

attn_paged_decode_split_kv_mma.cuh

perf: drop redundant smem zero-init in paged decode kernel

2026-08-01 16:17:34 +08:00

attn_paged_decode_split_kv.cuh

feat: scalar paged prefill fallback and decode causal fix

2026-08-01 16:52:01 +08:00

attn_paged_decode.cu

perf: launch CUDA kernels on torch's current stream

2026-08-02 13:20:14 +08:00

attn_paged_prefill_split_q_mma.cuh

feat: SGLang-style paged attention kernels replace page-table path

2026-08-01 15:41:25 +08:00

attn_paged_prefill_split_q.cuh

feat: scalar paged prefill fallback and decode causal fix

2026-08-01 16:52:01 +08:00

attn_paged_prefill.cu

perf: launch CUDA kernels on torch's current stream

2026-08-02 13:20:14 +08:00

attn_prefill_split_q_mma.cuh

feat: unify attention backend with multi-dim mask support

2026-07-30 20:38:34 +08:00

attn_prefill_split_q.cuh

feat: unify attention backend with multi-dim mask support

2026-07-30 20:38:34 +08:00

attn_prefill.cu

perf: launch CUDA kernels on torch's current stream

2026-08-02 13:20:14 +08:00

attn_warp_utils.cuh

perf: extend MMA decode to arbitrary GQA ratio, add launch bounds, vectorize combine

2026-07-27 00:35:34 +08:00

rotary_emb.cu

perf: launch CUDA kernels on torch's current stream

2026-08-02 13:20:14 +08:00