ViperEkura
/
AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
499 Commits 1 Branch 9 Tags 3.9 MiB
Commit Graph

10 Commits

Author SHA1 Message Date
ViperEkura 8a8550184f refactor: template AttentionParams, rename .cuh to .h 
- Convert AttentionParams to a template struct supporting arbitrary types
- Rename attn_common.cuh -> attn_common.h (no CUDA-specific code remains)
- Include standard headers explicitly in each .cuh instead of via attn_common.cuh
- Allow .h files in csrc/ via .gitignore
 2026-07-11 11:03:14 +08:00
ViperEkura d923ebe38d refactor: rename gqa_* to attn_*, split-KV for all decode paths 
- Rename all csrc/kernels/gqa_*.cuh/cu to attn_*, with _split_q / _split_kv
  strategy suffix and optional _mma compute suffix
- Remove non-split MMA decode kernel, keep only split-KV path
- Convert scalar decode fallback to split-KV (o_part/ml_part + combine)
- Move combine kernel to attn_decode_split_kv.cuh (shared by both paths)
- Rename GQAParams to AttentionParams
- Update all C++ #include, PYBIND11, and Python extension references
 2026-07-10 23:35:14 +08:00
ViperEkura 88f8dca2c2 perf: enlarge prefill KV tile to BC=32 for D<=128 
- Kernel is latency-bound (25% occupancy), not compute/bandwidth-bound
- BC=16 wasted a cp.async wait + barrier + loop overhead per tiny tile
- Double KV tile to BC=32 for D<=128; D=256 stays 16 (64KB > 48KB smem cap)
- Retune MIN_BLOCKS per head_dim (32->6, 64->4, 128->3, 256->2)
- Result: ~6-8% faster on L20, 0.93-1.20x vs torch SDPA, correctness unchanged
 2026-07-10 17:43:23 +08:00
ViperEkura 9027fdc546 test: bench production MMA attention path with FLOP/s and bandwidth 2026-07-10 16:58:34 +08:00
ViperEkura 85dc771460 perf: reduce MMA kernel registers, switch to static smem 
- Move Qa[KD][4] into tile loop (reload from sQ per tile)
  cutting ~32 resident registers for HEAD_DIM=128
- Replace extern __shared__ with static template-sized smem
  (no cudaFuncSetAttribute or dynamic allocation needed)
- Add __launch_bounds__ with MIN_BLOCKS param, dispatch by HEAD_DIM
  (hd=128→4, hd=64→6, hd=32→6)
- Remove dynamic smem from scalar kernel and C test
- Result: hd=128 168→128 regs, 25%→33% occupancy
 2026-07-10 00:39:47 +08:00
ViperEkura 4225518cf3 perf: add fast-math and vectorization nvcc/cxx build flags 
- centralize CXX_FLAGS/NVCC_FLAGS in csrc/build.py as single source
- add --use_fast_math, --ptxas-options=-O3,-v, --extra-device-vectorization
- add -march=native -funroll-loops host flags
- setup.py reads shared cxx_flags/nvcc_flags from registry
- sync pure-C test build commands with new flags
 2026-07-07 22:28:32 +08:00
ViperEkura ddc4bd1cf6 feat: tensor-core mma prefill with build-time dispatch 
- add register-resident flash-attention kernel using mma.sync.m16n8k16
- dispatch mma vs scalar at build time: pre-sm_80 defines
  -DASTRAI_NO_MMA, else defaults to mma
- scalar path vectorized with float4 smem loads (ld8)
 2026-07-06 20:33:24 +08:00
ViperEkura cc36530c73 perf: group-split register-blocking gqa_prefill kernel 
- one query row per group of G=8 lanes, each owning HEAD_DIM/G dims of qreg[]/acc[] in registers
- removes full 32-lane warp_reduce_sum; S dot reduces over only G lanes
- templated on <HEAD_DIM,G,ROWS,P_BC>, block=(G,ROWS)=(8,32)
- per-group shuffle mask so causal loop-bound divergence doesn't deadlock the shuffle
- update pure-C test to the templated launch
 2026-07-06 18:33:08 +08:00
ViperEkura 11fa807cfc fix: correct prefill mask index, unify GQA kernel interface 
- Fix mask indexing: batch*q_len*kv_len -> batch*kv_len
- Add csrc/kernels/gqa_common.cuh with shared GQAParams struct
- Unify decode/prefill Python API: both accept (q,k,v,mask=None,...)
- Decode now supports optional mask, is_causal, causal_offset, scale
- Rename struct fields: B->batch, Hq->q_head, Hk->kv_head, D->head_dim
- Use py::arg() for correct None/defaults handling in pybind11
- Update pure C tests and build instructions (-arch=sm_89)
 2026-07-06 17:21:23 +08:00
ViperEkura bcdd93e0eb feat: split kernel defs from bindings, add prefill tiled kernel and pure C tests 
- Split .cuh/.cu for gqa_decode_attn and gqa_prefill_attn
- gqa_prefill_attn: tiled shared-memory K/V, fused load, compute-opt, mask support
- Add pure C tests under csrc/tests/ for fast nvcc-only iteration
- Update .gitignore for build artifacts
 2026-07-06 16:14:55 +08:00