 ViperEkura
|
15862d4b56
|
perf: fuse fp8 linear fwd and bwd into single kernel calls
- fp8_linear_forward: cast + cublasLt GEMM + transpose + bias in one call
- fp8_linear_backward: scale-free, dtype derived from input tensor
- drops per-op Python dispatch (was ~6-8 launches per linear) and amax syncs
- 1024x1024 linear: 6.8x slow -> 0.67x (36.7us vs 24.8us bf16)
- small-model e2e still 1.71x slow; 15bt estimate ~0.78x (linear-heavy)
|
2026-08-14 01:24:37 +08:00 |
|
|
ViperEkura
|
f9efb705b8
|
perf: output fp8 gemm in bf16 instead of fp32
- cublasLt C layout and buffer switched to CUDA_R_16BF, halving output bandwidth
- downstream ops (RMSNorm etc.) keep matching bf16 dtype, fused kernels stay
- numeric error unchanged (0.19% vs fp32 ref on quantized inputs)
|
2026-08-14 01:08:26 +08:00 |
|
|
ViperEkura
|
a5b238dd86
|
feat: add fp8 training via cublasLt dispatch
- fp8_mm kernel (csrc): cublasLt fp8 e4m3 gemm, TN layout mapped zero-copy
- custom::fp8_mm custom op: meta/cuda/cpu kernels + scale-corrected bf16 autograd
- aten::linear and linear_backward dispatch on CUDA key, zero model changes
- per-tensor scale or raw cast; single-GPU smoke loss matches bf16
|
2026-08-14 00:39:49 +08:00 |
|