feat: wire up paged decode CUDA kernel to Python extension

- Add attn_paged_decode wrapper in ops.py with gather fallback
- Register kernel in loader.py and export from __init__.py
- Extract test_utils.cuh shared by all attention unit tests
- Rename attn_paged_vs_contiguous.cu to attn_paged_decode_test.cu
- Refactor decode/prefill tests to use common bf16 helpers and cpu ref
- Fix k_cache dim check in attn_paged_decode.cu
This commit is contained in:
2026-07-11 18:40:49 +08:00
parent 89ece26c25
commit 2c3cef1c87
9 changed files with 510 additions and 389 deletions
+1 -1
View File
@@ -79,7 +79,7 @@ torch::Tensor attn_paged_decode(
int batch = q.size(0);
int q_head = q.size(1);
int head_dim = q.size(3);
int kv_head = k_cache.size(3);
int kv_head = k_cache.size(2);
int max_pages = page_table.size(1);
TORCH_CHECK(q.is_cuda() && page_table.is_cuda() && k_cache.is_cuda() && v_cache.is_cuda());