AstrAI

Files

T

ViperEkura 41dcf0feb9 feat: SGLang-style paged attention kernels replace page-table path

- PagedAttentionParams uses flat KV pool + req_to_token + kv_indptr/qo_indptr instead of page_table
- MMA split-KV decode and split-Q prefill kernels with indirect ragged-batch addressing
- Prefill kernel accepts 4D mask (causal-aware); decode kernel supports 2D mask
- CudaBackend is inference-only: kv_cache=None raises, no torch fallback
- benchmark.py: required --ckpt, --backend/--compare options
- Parallel build isolates build-temp/build-lib per subprocess
- Standalone test covers decode/prefill with mask, 27 cases pass

2026-08-01 15:41:25 +08:00

config

Make Nora+NAdamW the default optimizer

2026-07-31 23:16:39 +08:00

dataset

refactor: remove H5 storage backend in favor of mmap bin

2026-07-29 12:50:27 +08:00

extension

feat: SGLang-style paged attention kernels replace page-table path

2026-08-01 15:41:25 +08:00

inference

perf: reduce decode overhead in scheduler and executor