ViperEkura
/
AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
517 Commits 1 Branch 9 Tags 4.1 MiB
Commit Graph

4 Commits

Author SHA1 Message Date
ViperEkura 57729fd92d refactor: stride-based attn interface with layout and causal mask 
- Replace is_causal + causal_offset with unified causal_offset (-1 = off, >=0 = first Q pos)
- Causal and mask can now coexist (was mutually exclusive)
- Add stride-based addressing for Q/KV/O (layout-agnostic, zero-copy)
- Add layout param ("bhld"/"blhd") parsed in Python, passed as int to C++
- Support 2D [batch, kv_len] and 3D [batch, q_len, kv_len] mask
- Vectorize paged KV gather in Python fallback (was per-token Python loop)
- Extract shared helpers: compute_num_splits, alloc_split_partials, dispatch_head_dim
- Unify paged_decode entry via attn_pack_paged_params
- Update mma_softmax_tile for 3D mask with per-row qrow indexing
 2026-07-14 21:34:42 +08:00
ViperEkura 8a8550184f refactor: template AttentionParams, rename .cuh to .h 
- Convert AttentionParams to a template struct supporting arbitrary types
- Rename attn_common.cuh -> attn_common.h (no CUDA-specific code remains)
- Include standard headers explicitly in each .cuh instead of via attn_common.cuh
- Allow .h files in csrc/ via .gitignore
 2026-07-11 11:03:14 +08:00
ViperEkura b8b439b713 perf: fuse decode combine kernel to single-pass online-rescale reduction 
- Replace 3-scan loops (mstar, lstar, acc) with 1-pass online rescale
- Halves __expf calls (num_splits vs 2*num_splits) and ml_part re-reads
- Mathematically equivalent, no change to o_part traffic or output
 2026-07-11 00:24:09 +08:00
ViperEkura d923ebe38d refactor: rename gqa_* to attn_*, split-KV for all decode paths 
- Rename all csrc/kernels/gqa_*.cuh/cu to attn_*, with _split_q / _split_kv
  strategy suffix and optional _mma compute suffix
- Remove non-split MMA decode kernel, keep only split-KV path
- Convert scalar decode fallback to split-KV (o_part/ml_part + combine)
- Move combine kernel to attn_decode_split_kv.cuh (shared by both paths)
- Rename GQAParams to AttentionParams
- Update all C++ #include, PYBIND11, and Python extension references
 2026-07-10 23:35:14 +08:00