2c3cef1c87
feat: wire up paged decode CUDA kernel to Python extension
89ece26c25
feat: paged decode attention with split-KV (scalar + MMA)
2c0b5d0b5e
perf: enable MMA decode path for G=1 full attention
a4ae7d17fb
perf: increase decode split-K parallelism for short sequences
8a8550184f
refactor: template AttentionParams, rename .cuh to .h
Compare 16 commits »
2c5629b81d
docs: fix documentation errors across README and assets/docs
841a582b28
refactor: split mask builder by single/multi output
c8567a6f65
fix: exclude embedding, lm_head, bias, and norm params from Muon optimizer, use AdamW
8035be9b1f
fix: make MuonMix inherit from torch.optim.Optimizer
e9b03f4fca
perf: apply cp.async, XOR swizzle, pre-scaled Q to decode MMA kernel
Compare 13 commits »
bfc8ff6098
Fix training budget (19k steps / ~20B tokens), add Muon optimizer, correct variance scaling math, move figures to §Training Config, tighten abstract
4a143b056d
Tighten abstract, add Muon optimizer mention, update ckpt_comparison figure
5726bf6c84
Add optimizer/initialization comparison figure (ckpt_comparison)
53ed52b4b8
refactor: extension dispatch layer with CUDA/torch fallback
f1cc7cedce
feat: ldmatrix + smem padding for mma prefill kernel
ddc4bd1cf6
feat: tensor-core mma prefill with build-time dispatch
cc36530c73
perf: group-split register-blocking gqa_prefill kernel
11fa807cfc
fix: correct prefill mask index, unify GQA kernel interface
Compare 11 commits »
abb96996f8
docs : sync 6 doc files to actual code
ff13a45d95
Restructure data pipeline: replace IFD analysis with length bias appendix
bbe6ff2d8f
release : v1.3.8
db9b39b084
fix: resolve IFD token-set asymmetry and support single-token answers
849e1e00a3
refactor: clean up inference design patterns
5416c2e8fb
perf: replace paged KV cache with contiguous ContiguousCache, decode all groups
599a51f4f7
fix: reliable test timeout, separate generate/test phases, dynamic pass@k
Compare 7 commits »
f2c2b159ba
Add IFD loss ratio density analysis, weight distribution appendix, and rewrite abstract
459523f0e4
Fix IFD citation: correct author list and venue (NAACL 2024, not NeurIPS)
4a7bf7ac3f
Add GQA/SwiGLU/RoPE formula blocks, Alpaca-GPT4 citation, and improve IFD-Loss Ratio interpretation
5c915514b3
Add correlation analysis interpretation to IFD bias section
bc8d4066c1
Add IFD analysis: density/length-bias figures and appendix
9534316bac
Fix duplicate paragraph and misleading table caption
b40fcc86c7
Refine abstract, switch to Times font, remove date, fix overfull hboxes with microtype
dd412b3c9c
Add IFD analysis, appendix, and improve abstract
c7158418dd
perf: add BFD bin-packing and custom attention mask to IFD batch scoring
4d3c9341c1
refactor: rewrite IFD evaluation with clean three-layer architecture
4e508afa2d
fix : SFT pipeline position_ids default & doc boundary preservation
Compare 3 commits »
