refactor: remove bf16 gemm and swiglu kernels and rebuild csrc benchmarks
- delete csrc/kernels/gemm.cu and swiglu.cu and drop their CMake and setup.py registration - remove the ops wrappers plus backend/linear.py and backend/swiglu.py so Linear and MLP call F.linear directly - drop the four gemm and swiglu kernel test files and prune the stale cuda_kernels.md sections - add csrc/bench benchmarks for the remaining kernels: attention decode prefill paged decode paged prefill versus single-launch SDPA references, rotary versus the torch fallback, fp8 quantize and mm_fp8 versus torch baselines - attention, rotary_emb, and fp8_ops kernels are unchanged
This commit is contained in:
@@ -1,11 +1,10 @@
|
||||
"""CUDA kernel wrappers, operator dispatch, and backend selection.
|
||||
|
||||
Public API:
|
||||
- ``attention``, ``linear``, ``swiglu``, ``apply_rotary_emb`` — op
|
||||
families with safe torch fallbacks (see ``astrai.extension.backend``)
|
||||
- ``attention``, ``apply_rotary_emb`` — op families with safe torch
|
||||
fallbacks (see ``astrai.extension.backend``)
|
||||
- ``attn_decode`` / ``attn_prefill`` / ``attn_paged_decode`` /
|
||||
``attn_paged_prefill`` — direct attention kernel wrappers
|
||||
- ``bf16_gemm`` / ``bf16_swiglu`` — directly callable linear/MLP kernels
|
||||
- ``AttentionBackend`` / ``TorchNativeBackend`` / ``CudaBackend`` /
|
||||
``FlashAttnBackend`` — attention backend strategies
|
||||
- ``resolve`` / ``explain`` / ``op_backend`` / ``env_mode`` — the shared
|
||||
@@ -14,6 +13,9 @@ Public API:
|
||||
Layout convention: all q/k/v are ``[batch, seq_len, n_heads, head_dim]``
|
||||
(blhd). Scale is always ``1/sqrt(head_dim)``. Wrapper functions call their
|
||||
compiled CUDA kernels directly; fallback is the backend's responsibility.
|
||||
Linear projections and dense-MLP SwiGLU run plain torch (``F.linear`` /
|
||||
``Linear`` / ``MLP``); the former bf16_gemm / bf16_swiglu kernels and their
|
||||
backends were removed.
|
||||
"""
|
||||
|
||||
from astrai.extension.backend import (
|
||||
@@ -27,8 +29,6 @@ from astrai.extension.backend import (
|
||||
attention,
|
||||
attn_backend,
|
||||
get_backend,
|
||||
linear,
|
||||
swiglu,
|
||||
)
|
||||
from astrai.extension.dispatch import (
|
||||
Axes,
|
||||
@@ -52,9 +52,8 @@ from astrai.extension.ops import (
|
||||
TensorLayout,
|
||||
attn_decode,
|
||||
attn_paged_decode,
|
||||
attn_paged_prefill,
|
||||
attn_prefill,
|
||||
bf16_gemm,
|
||||
bf16_swiglu,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
@@ -68,13 +67,10 @@ __all__ = [
|
||||
"attention",
|
||||
"attn_backend",
|
||||
"get_backend",
|
||||
"linear",
|
||||
"swiglu",
|
||||
"attn_decode",
|
||||
"attn_paged_decode",
|
||||
"attn_prefill",
|
||||
"bf16_gemm",
|
||||
"bf16_swiglu",
|
||||
"attn_paged_prefill",
|
||||
"is_available",
|
||||
"KERNEL_NAMES",
|
||||
"apply_rotary_emb",
|
||||
|
||||
Reference in New Issue
Block a user