feat: unify attention backend with multi-dim mask support
- Add attention() functional entry delegating to active backend - GQA/MLA forward calls attention() instead of inline cache/SDPA - CUDA kernels support 2D/3D/4D mask via mask_h_stride field - CudaBackend.fwd_decode builds 2D padding mask for mixed seq_lens - KVCache.max_len precomputed in bind_tasks to avoid GPU sync - batch==1 decode short-circuits mask=None - Split tests into conftest, test_backend, test_backend_equivalence, test_kernel_mask - 440 tests pass, L20 decode 1.44-1.60x speedup vs torch native
This commit is contained in:
@@ -20,6 +20,7 @@ from astrai.extension.attention_backend import (
|
||||
AttentionBackend,
|
||||
CudaBackend,
|
||||
TorchNativeBackend,
|
||||
attention,
|
||||
attn_backend,
|
||||
get_backend,
|
||||
)
|
||||
@@ -35,6 +36,7 @@ __all__ = [
|
||||
"AttentionBackend",
|
||||
"CudaBackend",
|
||||
"TorchNativeBackend",
|
||||
"attention",
|
||||
"attn_backend",
|
||||
"get_backend",
|
||||
"attn_decode",
|
||||
|
||||
Reference in New Issue
Block a user