Commit Graph
2 Commits
Author SHA1 Message Date
ViperEkura f9efb705b8 perf: output fp8 gemm in bf16 instead of fp32
- cublasLt C layout and buffer switched to CUDA_R_16BF, halving output bandwidth
- downstream ops (RMSNorm etc.) keep matching bf16 dtype, fused kernels stay
- numeric error unchanged (0.19% vs fp32 ref on quantized inputs)
2026-08-14 01:08:26 +08:00
ViperEkura a5b238dd86 feat: add fp8 training via cublasLt dispatch
- fp8_mm kernel (csrc): cublasLt fp8 e4m3 gemm, TN layout mapped zero-copy
- custom::fp8_mm custom op: meta/cuda/cpu kernels + scale-corrected bf16 autograd
- aten::linear and linear_backward dispatch on CUDA key, zero model changes
- per-tensor scale or raw cast; single-GPU smoke loss matches bf16
2026-08-14 00:39:49 +08:00