feat: add optional CUDA kernel system (csrc/) + fused GQA decode attention
Structure: csrc/ -- .cu sources + build.py registry astrai/extension/ -- compiled .so + __init__.py (import dispatcher) setup.py -- CUDAExtension from csrc/build.py REGISTRY Control: CSRC_KERNELS=true|false env var at install time. Fallback: astrai.extension.available dict for runtime detection.
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
# Source directory for CUDA kernels — build-time only.
|
||||
# Compiled .so files live in astrAI/_ext/.
|
||||
Reference in New Issue
Block a user