AstrAI

Files

T

ViperEkura d0c5debbab perf: preload V in decode split-kv shared mem and cache partial tensors

- Preload V into shared memory alongside K to eliminate per-element KV address lookups in the inner softmax/accum loop (doubles smem)
- Cache split-KV partial tensors (o_part, ml_part) with static tensors instead of per-call allocation in both decode and paged-decode paths
- Force is_causal=True in CUDA decode backend (decode is always causal)

2026-08-06 18:27:00 +08:00

lib

refactor: move compiled kernel .so files into extension/lib

2026-07-31 15:36:32 +08:00

__init__.py

refactor : use factory for attention backends

2026-08-05 15:37:22 +08:00

attention_backend.py

perf: preload V in decode split-kv shared mem and cache partial tensors

2026-08-06 18:27:00 +08:00

attention_ops.py

feat: SGLang-style paged attention kernels replace page-table path