refactor: replace magic layout ints with TensorLayout enum

- Add TensorLayout enum (C++ + Python) to replace magic layout ints
- Add C10_CUDA_CHECK post-launch error checking to all kernel entries
- Add CUDAGuard + freqs_cis shape validation to rotary_emb.cu
- Cache SM count to eliminate per-call cudaDeviceGetAttribute
- Add DISPATCH_CAUSAL_MASK macro to deduplicate dispatcher if/else
- Convert mask type hints from X|None to Optional[X]
This commit is contained in:
2026-08-01 11:05:52 +08:00
parent 3639b50b4a
commit 7feeb0b93e
9 changed files with 91 additions and 59 deletions
+2
View File
@@ -25,6 +25,7 @@ from astrai.extension.attention_backend import (
get_backend,
)
from astrai.extension.attention_ops import (
TensorLayout,
attn_decode,
attn_paged_decode,
attn_prefill,
@@ -37,6 +38,7 @@ __all__ = [
"AttentionBackend",
"CudaBackend",
"TorchNativeBackend",
"TensorLayout",
"attention",
"attn_backend",
"get_backend",