perf: accelerate decode linear with bf16 gemv
- add decode-shape benchmark harness - add bf16 GEMV CUDA primitive with head-dim generic kernel - dispatch decode-time linear layers to gemv for M=1 - extend gemv coverage to small decode batches
This commit is contained in:
@@ -61,6 +61,7 @@ set(KERNEL_NAMES
|
||||
attn_prefill
|
||||
attn_paged_decode
|
||||
attn_paged_prefill
|
||||
bf16_gemv
|
||||
rotary_emb
|
||||
)
|
||||
set(KERNEL_SRCS
|
||||
@@ -68,6 +69,7 @@ set(KERNEL_SRCS
|
||||
attention/prefill.cu
|
||||
attention/paged_decode.cu
|
||||
attention/paged_prefill.cu
|
||||
gemv/bf16_gemv.cu
|
||||
rotary_emb.cu
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user