refactor: remove bf16 gemm and swiglu kernels and rebuild csrc benchmarks

- delete csrc/kernels/gemm.cu and swiglu.cu and drop their CMake and setup.py registration
- remove the ops wrappers plus backend/linear.py and backend/swiglu.py so Linear and MLP call F.linear directly
- drop the four gemm and swiglu kernel test files and prune the stale cuda_kernels.md sections
- add csrc/bench benchmarks for the remaining kernels: attention decode prefill paged decode paged prefill versus single-launch SDPA references, rotary versus the torch fallback, fp8 quantize and mm_fp8 versus torch baselines
- attention, rotary_emb, and fp8_ops kernels are unchanged
This commit is contained in:
2026-09-05 01:38:10 +08:00
parent a77e35dd51
commit 6709534d64
24 changed files with 1306 additions and 3394 deletions
+7 -11
View File
@@ -1,11 +1,10 @@
"""CUDA kernel wrappers, operator dispatch, and backend selection.
Public API:
- ``attention``, ``linear``, ``swiglu``, ``apply_rotary_emb`` — op
families with safe torch fallbacks (see ``astrai.extension.backend``)
- ``attention``, ``apply_rotary_emb`` — op families with safe torch
fallbacks (see ``astrai.extension.backend``)
- ``attn_decode`` / ``attn_prefill`` / ``attn_paged_decode`` /
``attn_paged_prefill`` — direct attention kernel wrappers
- ``bf16_gemm`` / ``bf16_swiglu`` — directly callable linear/MLP kernels
- ``AttentionBackend`` / ``TorchNativeBackend`` / ``CudaBackend`` /
``FlashAttnBackend`` — attention backend strategies
- ``resolve`` / ``explain`` / ``op_backend`` / ``env_mode`` — the shared
@@ -14,6 +13,9 @@ Public API:
Layout convention: all q/k/v are ``[batch, seq_len, n_heads, head_dim]``
(blhd). Scale is always ``1/sqrt(head_dim)``. Wrapper functions call their
compiled CUDA kernels directly; fallback is the backend's responsibility.
Linear projections and dense-MLP SwiGLU run plain torch (``F.linear`` /
``Linear`` / ``MLP``); the former bf16_gemm / bf16_swiglu kernels and their
backends were removed.
"""
from astrai.extension.backend import (
@@ -27,8 +29,6 @@ from astrai.extension.backend import (
attention,
attn_backend,
get_backend,
linear,
swiglu,
)
from astrai.extension.dispatch import (
Axes,
@@ -52,9 +52,8 @@ from astrai.extension.ops import (
TensorLayout,
attn_decode,
attn_paged_decode,
attn_paged_prefill,
attn_prefill,
bf16_gemm,
bf16_swiglu,
)
__all__ = [
@@ -68,13 +67,10 @@ __all__ = [
"attention",
"attn_backend",
"get_backend",
"linear",
"swiglu",
"attn_decode",
"attn_paged_decode",
"attn_prefill",
"bf16_gemm",
"bf16_swiglu",
"attn_paged_prefill",
"is_available",
"KERNEL_NAMES",
"apply_rotary_emb",