refactor: extension dispatch layer with CUDA/torch fallback

- Add gqa_decode_attn/gqa_prefill_attn dispatch functions - Internal _available/__modules with underscore prefix - CUDA kernel path with F.scaled_dot_product_attention fallback - GQA head expansion in fallback path
2026-07-06 21:07:16 +08:00
parent f1cc7cedce
commit 53ed52b4b8
2 changed files with 82 additions and 5 deletions
@@ -16,7 +16,6 @@ from astrai.dataset import (
    Store,
    StoreFactory,
 )
-from astrai.extension import available
 from astrai.factory import BaseFactory
 from astrai.inference import (
    GenerationRequest,