AstrAI

Files

T

ViperEkura 3067a8e1a6 feat: unify attention backend with multi-dim mask support

- Add attention() functional entry delegating to active backend
- GQA/MLA forward calls attention() instead of inline cache/SDPA
- CUDA kernels support 2D/3D/4D mask via mask_h_stride field
- CudaBackend.fwd_decode builds 2D padding mask for mixed seq_lens
- KVCache.max_len precomputed in bind_tasks to avoid GPU sync
- batch==1 decode short-circuits mask=None
- Split tests into conftest, test_backend, test_backend_equivalence, test_kernel_mask
- 440 tests pass, L20 decode 1.44-1.60x speedup vs torch native

2026-07-30 20:38:34 +08:00

__init__.py

feat: unify attention backend with multi-dim mask support

2026-07-30 20:38:34 +08:00

attention.py

feat: unify attention backend with multi-dim mask support

2026-07-30 20:38:34 +08:00

decoder_block.py

refactor: rebuild KV cache with three-layer separation architecture

2026-07-30 17:19:06 +08:00

embedding.py

refactor : neftune_alpha 在 Embedding 构造时传入，由模型配置链路负责